Heads up, Giving a lot of context in this one 
I’m looking at how best to handle loading/unloading of assets.
From what I can tell, scenes aren’t a great way of managing rendering assets when you have to care about memory or load times, like on mobile, because by design you can’t share assets between scenes.
Context
Let’s say I’m making a multiplayer pokemon-like world (because of course I am) which has an overworld and smaller areas, houses, caves etc to explore.
- Be on the overworld
- See a door
- Walk through a door
- Engine loads the assets required for that ‘area’
- (ideally doesn’t have to reload any already loaded assets, like tiles, player sprites etc) so that memory and load times are protected)
- GameEngine creates new Babylon ‘scene’ and inserts a mix of already loaded assets and assets needed for just this scene
- GameEngine pauses rendering the ‘outside’ scene, renders the ‘inside’ scene
- Player goes back through door to outside area
- GameEngine unloads assets that were only required for that area, keeping the shared assets like player sprites etc in memory
8,9,10. rinse, repeat
My initial approach
- initially thought I’d have multiple scenes, the overworld is a scene, a single instance of a house is a scene etc. I could definitively unload unused assets, e.g unload the house & NPC sprites when the player exits the house by calling
scene.dispose()but the player assets wouldn’t be as the player is referenced by the other scene.
This means if I wanted to use the same assets for rendering the player AND I wanted to ‘know’ I can call scene.dispose() on the house scene and all non-shared assets will eventually be unloaded, I’d take performance hit of loading & keeping multiple instances of the same assets in memory (load the player mesh & animations & textures etc each time they need to be in a new scene) - that’s a memory hit and a loading time hit for an asset that’s already loaded. That seems like a really good reason to NOT use scenes.
Maybe I’m just structuring my whole approach wrong but I wanted to have each area be it’s own independent scene because :
- It aligns with the approach of having each isolated area it’s own EntityComponentSystem-style EcsWorld, other entities don’t have to worry about stuff that’s going on in that world
- Having the areas loaded up into a shared main scene works gameplay wise BUT now I need a way of manually tracking and disposing of any and all assets created as part of that little scene.
- doing odd things like - Add an EcsContextComponent to entities which have an array of filters like [‘Outside’, ‘SceneAName’, ‘VisualInventory’] and only rendering entities from active ‘scenes’ breaks when things like physics are involved, you end up layering complexity on top of complexity, needing to apply additive physics layers too etc
The extra layer of sadness, asset caching
What’s making me extra sad about this is I was building out a nice ECS-style approach in which I have a BabylonRenderSystem that just loops through entities in an ECS world, loads the assets and keeps them in sync without the ECS having to know about babylon stuff. This all broke the moment I needed to do a scene transition because the caching layer made the assumption it could serve up assets from existing assetContainers.
private async createMeshFromType(meshType: string | undefined, componentsById: { [x: string]: IComponent }): Promise<Mesh> {
const meshcomponent = componentsById[MeshVisualComponentId] as MeshVisualComponent;
const transformComponent = componentsById[TransformComponentId] as TransformComponent;
if (meshcomponent.assetId) {
const assetInfo = this.assetLoader.getAssetInfoById<MeshAssetInfo>(meshcomponent.assetId);
const assetPath = this.assetLoader.getAssetPath(assetInfo.path);
const useCache = true;
let meshAssetContainer: babylon.AssetContainer = null as any;
if (useCache) {
if (this.assetLoader.getAssetCacheItem(`${assetPath}:assetContainer`)) {
meshAssetContainer = this.assetLoader.getAssetCacheItem(`${assetPath}:assetContainer`);
} else {
const asset = await this.assetLoader.getAssetFromPath(assetPath);
const url = URL.createObjectURL(new Blob([asset]));
meshAssetContainer = await babylon.LoadAssetContainerAsync(url, this.scene, {
pluginExtension: ".glb",
});
this.assetLoader.addAssetCacheItem(`${assetPath}:assetContainer`, meshAssetContainer);
}
} else {
meshAssetContainer = await babylon.LoadAssetContainerAsync(assetPath, this.scene);
}
const meshContainer = meshAssetContainer.instantiateModelsToScene();
return meshContainer.rootNodes[0] as Mesh;
}
}
But now it can’t just follow
- just loop through entities
- if the entity doesn’t exist, create it
- if the assets for the entity don’t exist, load them and cache the assetContainer for next time
because it now has to know about the concept of scenes and that needs to be communicated up and down the stack because the assetCache can’t simply load an asset anymore, it has to know the target scene at load time and that can’t change - this means
- my asset layer needs to know way more about what’s going on, the asset caching layer needs to respond to scene change events
- I can only ‘cache’ assets to be re-used in that scene
- I have to blow away the asset cache when the scene is disposed, because an assetContainer throws an exception when trying to load an asset from a disposed scene (which makes sense in narrow context, but not in this wider use case)
Don’t read this all as a dig at Babylon, I can’t imagine the # of counter-intuitive trade-offs that have had to be made to build this thing.
I have zero doubt that there’s an engine reason for loaded assets being hard tied to scenes, but as a user this leaves me in a pickle. I’m a bit scared and wondering if I should give up all the sweet Babylon stuff and just go back to using three.js, which does separate out asset loading from scenes three.js manual
So my question is : Help ! 
You probably want to fine-tune memory management within your manager anyway.
But I’m fine, I could cope with the current system after wasting hours understanding how it works.
