GLTF Parser Hooks

Hey @bghgary … Do you think you can point me in the right direction for this code that splits ups the sub meshes… Maybe i can take a look at giving it an option to not split up sub meshes.

Its just I cant move forward with my Tree Exporter and its really screwing up my LOD system

:slight_smile:

The code is here: Babylon.js/glTFLoader.ts at master · BabylonJS/Babylon.js · GitHub

Basically, the idea is that we need to check if the mesh primitives can be merged (i.e. if they have the same kinds of attributes). If they do, then call VertexData.merge to merge all the primitives into a single vertex data.

1 Like

Sweet… What attributes should we be check that are equal ???

Also…How should i go about this… Add another promise type function that runs after the:
for (const primitive of primitives) loop that adds all the loadMeshPrimitivesAsync promises. And in that function we access node._primitiveBabylonMeshes array. Loop thru that and group together primitives and stick that re-grouped primitives back into node._primitiveBabylonMeshes

Something like that ???

BTW… The loadAnimationAsync interface seems fine… If you change the base function in GLTFLoader.ts … Ill just recopy. So let me know if you change loadAnimationAsync in gltfLoader.ts.

Otherwise this is working fine in my extension so far:

    /** @hidden */
    public loadAnimationAsync(context: string, animation: BABYLON.GLTF2.IAnimation): Promise<BABYLON.AnimationGroup> {
        if (this._parseScene === true && animation.extras != null && animation.extras.metadata != null) {
            console.warn("CVTOOLS: LoadAnimationAsync: " + animation.name);
            const loader:any = this._loader;
            const xanimation:any = animation;
    
            const babylonAnimationGroup = new BABYLON.AnimationGroup(animation.name || `animation${xanimation.index}`, loader._babylonScene);
            xanimation._babylonAnimationGroup = babylonAnimationGroup;
    
            const promises = new Array<Promise<any>>();
    
            BABYLON.GLTF2.ArrayItem.Assign(xanimation.channels);
            BABYLON.GLTF2.ArrayItem.Assign(xanimation.samplers);
    
            for (const channel of animation.channels) {
                const xchannel:any = channel;
                promises.push(loader._loadAnimationChannelAsync(`${context}/channels/${xchannel.index}`, context, animation, channel, babylonAnimationGroup));
            }
    
            return Promise.all(promises).then(() => {
                const metadata:any = {};
                metadata.unity = animation.extras.metadata;
                (<any>babylonAnimationGroup).metadata = metadata;
                this._parser.addAnimationGroupItem(babylonAnimationGroup);
                babylonAnimationGroup.normalize(0);
                return babylonAnimationGroup;
            });
        } else {
            return null; // Not Handled
        }
    }

I can keep track of the animationGroups as they are created… so no need to create and expose animationGroups array to the loader for just that loaded animationGroups…

I can now also get to the IAnimation.extras.metadata i need to hookup those animation groups to the state machine they where exported from in unity… Looks great to me :slight_smile:

It’s a bit more complicated than that since there is instancing, submaterials, etc. Feel free to try to figure it all out, but I can try to tackle this hopefully today or tomorrow.

This is fine as long as you are okay with calling internal functions which can change at any time.

In that case I better let you handle merging vertex data. Thanks Gary :ok_hand:

I started looking at this. It is going to be relatively hard. Will see how far I can get today.

Im kool with that… AS you can see… its really just a copy of gltfLoader loadAnimationAsync… in the last promise i TAG the animation group with unity metadata then add it to my parser’s animationGroup List.

If loadAnimationAsync changes (please let me know) ill just re-copy and apply my tiny mods at the end :slight_smile:

I don’t think it’s a good idea anymore to merge the vertex data in the glTF loader. There are too many edge cases. glTF mesh primitives can have different attributes, different byte strides, different normalized flags, different draw modes, etc. Then there is instancing to consider.

A relevant glTF issue on this topic: Intended use(s) of mesh primitives · Issue #1278 · KhronosGroup/glTF · GitHub.

Perhaps you can use Mesh.MergeMeshes in some way to achieve what you want, but it will likely have a performance hit.

Why does it split up submeshes in the first place.

How supposed to handle LOD if splitting up submeshes ???

The loader doesn’t split. It just take each mesh primitive and creates a separate mesh for each.

I’m not sure what you mean. There are multiple way to handle LODs. A naive way is that each LOD is a completely separate node tree and is turned on and off.

Well if i had a character that had 5 submeshes… I also have 3 LODs of that character.

When exported to gltf my 1 mesh (with 5 submeshes) turns into to 5 primitives.

what am i supposed to do about LOD (which i assume gets split up also)… Go back and iterate over every primitive child and figure out which pimitive is which and associate its primitive LOD… I guess…

It just seems like ALOT of mesh overhead and going back thru trying to SYNC-BACK-UP what i originally exported… That kinda sucks.

Also… I see where he GLTF Export process breaks all the meshes and submeshes down into primitives… that also sucks…

If i export a single mesh… I want a single mesh :frowning:

Looks like im gonna have to go CUSTOM GLTF and make my own meshes… serialize my own geometry data and create my own meshes in babylon from this serialized mesh data.

I know its not GLTF Spec (And your the GLTF Liaison) … But meshes and skinning kinda suck to me… Everything else is great… especially the binary encoding… I love the binary part. That is why i am considering using GLTF as my own hybrid babylon/gltf file format (by way of extensions).

Dunno… maybe will end up with two toolkits…

One will be the old .babylon file (I try and fix up the current toolkit with all the good stuff i put in the GLTF exporter)

And i work on another for ME (and maybe the community) if @Deltakosh says its ok to have my custom GLTF file format in the main Babylon Toolkit… Dunno what to do :frowning:

Yo @bghgary … Dude i cant express how much of a great freaking job you did with the whole BABYLON GLTF Importer and is API for extensions. I used other to start like PlayCanvas (which is really some some raw java script to kinda hack in load gltf models… Nothing like the STRONG API you have for GLTF in babylon)

Anyways… making extensions you can do pretty much whatever you want (WITH THE RIGHT HOOKING into the gltf import pipeline). So i made another extension call CVTOOLS_babylon_mesh. In this extension i DONT encode a Node.Skin but instead encode a streamlined BabylonSkeleton as metadata. I now also have a mode called Export Mesh Mode… It can be set to either Primitives which export normal GLTF primitives for your submeshes. Or go Babylon which will encode ALL the triangles for the Unity mesh as 1 Single primitive mesh… I then encode Babylon Submeshes array with all the sub mesh indexes and a Babylon MultiMaterial with all the sub mesh materials.

Now in my loadNodeAsync… If the node metadata as SUBMESH and MULTI-MATERIAL info… I create a Babylon.MultiMaterial with all the materials from the submeshes. Then do BABYLON.SubMesh.AddToMesh with all the sub mesh array info in the metadata

And Eureka I now have basically Native Babylon Shared Skeletons and MultiMaterial Meshes with all the GLTF beniftes like binrary encoding for all the float array.

My same Robot model with 1 simple idle animation and NO BLEND SHAPES what 30MB in .babylon files… Now that same model with its Babylon Style Meshes and Skeletons encoding in the GLTF binary stream with IDLE, WALK AND RUN animations is only 5MB.

That is why i love the GLTF format… I HATE the Skinning and the Primitives

But with your GLTF Import API… The more i work with it the better i get at turning Unity content into Babylon content. I can do my my own native babylon skinning with shared skeletons and use native babylon mesh setup with submeshes and multi materials…

Great Job On the Whole API… Kudos :slight_smile:

2 Likes

Awesome, very glad you got this working. I was going to reply that core glTF doesn’t do exactly what you want and you should probably use an extension, but you beat me to the punch. :slight_smile:

If you’re up for it, I think it would be helpful if you raise these issues on the glTF GitHub. It will be good to have input from real usage. The two issues that I think are relevant:

  1. Intended use(s) of mesh primitives · Issue #1278 · KhronosGroup/glTF · GitHub (which I mentioned earlier)
  2. https://github.com/KhronosGroup/glTF/issues/1285

Yo @bghgary need your advice again. I am handling submeshes in my extension. I encode a multi material as metadata… I wanna use pointers to the gltf material for my list of materials in multi material… but ran into a problem trying to call loadMaterialAsync more than once per node… if I call it in a loop adding the promise for each multi material it hangs and never completes… I can only successfully call loadMaterialAsync once per node… got any suggestions before I tear apart the loadMaterialAsync to try and make a light version of that function that can be called multiple times per node to load up a multi material

Can you repro in a playground? That will help me understand what you mean / what is going on.

Ill try… but im not to good at using the raw javascript to show what im trying to do in the full typescript api

You can use the typescript playground :slight_smile:
https://www.babylonjs-playground.com/ts.html

Yo @bghgary … On another note… I just download and did a build of the latest master and the gltfLoader is now using _position and _rotationQuaternion and _scaling now and my animations dont seem to work anymore

@Deltakosh made this change for perf reasons. What problem are you having?