Feature request: add model loading stage

I have put lots of thought lately into how models are currently loaded as I have always wanted to load models slightly differently. I would like to support three use cases:

  1. passing models across scenes (even Engines?) without AssetContainer
  2. pre-loading model without automatically loading to scene (can optionally manipulate before adding to scene).
  3. re-using a loaded model and hydrating an in-memory cached model quickly (without instancing).

I think that I can retrofit the ObjLoader to support this as I am familiar with most of that code (not so much the textures, but at least the meshes and MTL file), but let’s look at the three.js gltfLoader:

import * as THREE from 'three';
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader';
const loader = new GLTFLoader();
const scene = new THREE.Scene();
loader.load(
    'models/file.glb',
    (gltf) => {
        // this is the part I want to cache and control myself!
        scene.add(gltf.scene)
    },
    (xhr) => {
        // this is like progress loading on existing SceneLoader
        console.log((xhr.loaded / xhr.total) * 100 + '% loaded')
    },
    (error) => { console.log(error); }
);

More generally I am asking if the community would benefit and want to be able to do that. At least for gltf (@bghgary ?) there is willingness to extend loaders. What I am proposing is a potential intermediate second stage of loading. This would allow me to preload a model and then hydrate/serialize at will. Part of the proposal is a backwards compatible opt-in “preload only” and onPreloaded callback that would allow not automatically “add to scene”. In the three gltfloader you can clone that scene object for multiple usage and I believe that object can be loaded in a different scene.

In my project react-babylonjs my most frequently asked questions revolve around models and I have trouble coming up with a clean solution especially in v3 with a move to support Suspense, so there is a fallback option while the model is loading and wanting to support showing the same model multiple times in the scene is a typical issue. This is not possible due to global caching implemented on my side currently for handling loading and Suspense (would you believe React has you throwing a promise!).

This isn’t supported OOTB currently as this would use the same Model actually and cache busting (ie: filename) would request duplicate loading (I didn’t wrap in Suspense components for simplicity).

<>
  <Model position={position1} rootUrl={'/models'} sceneFilename={'file.gltf'} />
  <Model position={position2} rootUrl={'/models/'} sceneFilename={'file.gltf'} />
</>

So with instancing I realize I can solve this and I do have recipes with that as a solution. Still I would like to support this in a way that I would (pre) load the model once and place multiple times in the scene. I feel also the other things we get for free would benefit the community as a whole. Especially pre-loading and quick hydrating models potentially across scenes. Anyway, I won’t ramble more and hope to get feedback if it would be useful.

4 Likes

I’m not totally sure I understand everything you are saying yet, but why not AssetContainer? Many of the things you are describing can be done with an AssetContainer I think.

The AssetContainer is not as “portable” in the sense that I would like to move a loaded model to another scene even on another engine instance to meet my use case. Have a look here at the comment from drcmda, who is the author of react-three-fiber ( pmndrs/react-three-fiber: :switzerland: A React renderer for Three.js (github.com)) and ( pmndrs/gltfjsx: :video_game: Turns GLTFs into JSX components (github.com) ) - it’s a long read, but has all the context that is a bit lengthy for a feature request - his first comment is here:

edit: just want to highlight selected parts of that comment to save people needing to click through:

in r3f you can re-use cached assets in any scene, even when the previous one unmounts. this is one the critical things about suspense and cache control, it’s outside the mount phases.
… the buffers and materials will always be there

but in gltfjsx’es case it could even mount multiple times, because it’s immutable.
gltfjsx especially is one of the most important assets we have in the r3f eco system, this is light years from how people have dealt with gltf before.

I haven’t read the thread yet, but I think I understand. If we do something for this, I don’t think it would be good to make this glTF specific. This can apply to any data being loaded or perhaps even created manually. @sebavan @Deltakosh @RaananW Any thoughts?

2 Likes

I agree it’s not glTF specific as it would apply to other loaders and could be used elsewhere, but I would like that cacheable intermediate format and extra stage available in loaders (and preload).

2 Likes

I love the overall idea

I am almost positive that this comment is not going to be helpful, as I cannot say what asset manager even is.

But, the reason I do not is I use Mesh sub-classes with embedded geometry in the constructor. While sub-classing capability is the main reason, storing all the meshes in JS files means it can go into any engine on the page. It IS a cacheable intermediate format, pre-parsed.

If you do not need it immediately in the scene, you can specify async or defer on the script tag. I also have a function inside the module, called defineMaterials, which takes a scene & base url for textures. You can run this as soon as scene is created, and get your textures downloaded & materials created on a different browser thread.

There is still a final shader compile, but things can be really fast when the new module.mesh('name'...) is actually executed. Just think things are getting a little complicated in non-embedded land.

5 Likes

I think your comment is helpful, because it demonstrates that others have built their own capabilities outside the framework and that others in the community may benefit by including this capability and working on the idea together and sharing ideas can make the solution more useful. What you have designed for having geometries available outside of scene/engine can be preloaded and used anywhere - much like this proposal. Thanks for sharing.

4 Likes

OK - this is great to see interest in integrating this. What is the best way to move forward? I could propose a solution for OBJ loader, but I know we will need a more general solution that supports lights/cameras/etc for more advanced loaders like glTF and .babylon etc. Here is a screen capture that shows the structure:

Addressing how the meshes are loaded with geometries and materials/textures will present an interesting challenge - looks like this in console:
image
Without investigating the code or object structure it would appear we need a way to map textures and materials as a starting point.

Also, there is an intermediate inherited class with methods like clone, which may use a composite pattern for operations.