GLTF Parser Hooks

Questions
163

Well i made a change i was going to PR into gltfLoader.ts to allow animation target override for animation groups called animationTargetOverride

    private _loadAnimationChannelAsync(context: string, animationContext: string, animation: IAnimation, channel: IAnimationChannel, babylonAnimationGroup: AnimationGroup, animationTargetOverride: Nullable<IAnimatable> = null): Promise<void> {

then at bottom:

                if (animationTargetOverride != null) {
                    animationTargetOverride.animations.push(babylonAnimation);
                    babylonAnimationGroup.addTargetedAnimation(babylonAnimation, animationTargetOverride);
                } else {
                    targetNode._babylonTransformNode!.animations.push(babylonAnimation);
                    babylonAnimationGroup.addTargetedAnimation(babylonAnimation, targetNode._babylonTransformNode!);
                }

That should put the animations directly on my animation target override which is the actual bone the channel animation was encoded to… But using my native babylon skeleton…

I was doing this before by coping the animations from the transform node that are created as the skins in gltf… but is slower… I am trying to get the loadAnimationAsync to put those animation directly on my bones instead of the transform nodes.

that should work but now playing my animation on my bones… Dunno of is because its now has an underscore or what ???

164

I need something to debug to help. I don’t how the underscore change has any effect on your change (though I’m not sure modifying a private function is a good idea).

165

Yep… That was is it… i took out the underscore and rebuild and that worked perfect.

The difference is that i pass a BONE as the animationTargetOverride property i added in the loadAnimationChannelAsync function above…

So i dont have to CUT AN COPY the while loadAnimationChannelAsync function to my extension i added that little animation target override to allow changing the animation target of that channel… Which is the Unity bone the animation was encoded from… GLTF just puts that animation on the transform node under the mesh instead of the actual bone itself… that animationTargetOvrride allows me to specify the BONE itself from the babylon skeleton i am creating instead of a gltf skin.

But… position and rotation and scaling SETTERS must be doing something when its a bone… like marking it dirty or something … dunno

166

Yep… @bghgary … The setters dont just directly update _position etc…

take a look at the bone position setters:

    /** Gets or sets current position (in local space) */
    public get position(): Vector3 {
        this._decompose();
        return this._localPosition;
    }

    public set position(newPosition: Vector3) {
        this._decompose();
        this._localPosition.copyFrom(newPosition);

        this._markAsDirtyAndCompose();
    }

    /** Gets or sets current rotation (in local space) */
    public get rotation(): Vector3 {
        return this.getRotation();
    }

    public set rotation(newRotation: Vector3) {
        this.setRotation(newRotation);
    }

    /** Gets or sets current rotation quaternion (in local space) */
    public get rotationQuaternion() {
        this._decompose();
        return this._localRotation;
    }

    public set rotationQuaternion(newRotation: Quaternion) {
        this.setRotationQuaternion(newRotation);
    }

    /** Gets or sets current scaling (in local space) */
    public get scaling(): Vector3 {
        return this.getScale();
    }

    public set scaling(newScaling: Vector3) {
        this.setScale(newScaling);
    }

If i gotta cut and copy the loadAnimationChannelAsync to go back to without the underscore… i can… :slight_smile:

Just let me know of i should because i am overriding with a bone ???

167

We will revert the underscore change. The underlying perf problem is caused by something else.

168

Sweet … ill add the PR to add support for animation target override AFTER the _ fix :slight_smile:

169

Yo @bghgary … What would be the performance impact to have a empty transform node attach to each bone of a skeleton … ???

I ask because it would be ALOT easier to attach all the orginal transform nodes to the native babylon skeleton bone… so if you had like a HAT mesh attached to the head bone… the transform node would be the parent node i attached to bone… so MOST bones would not have meshes but be empty transform nodes…

If it is a costly thing i would have to TAG each bone that have SOCKET MESH component attached to it and then attache that node to the bone… but i would have to dig further into the api to do that …

So is there a performance hit that ill will take to have an empty transform node attached to each bone ???

170

The easiest way to determine this is to profile the scene. If I understand what you are describing, the cost will be calculating the transform of the parent node. If the nodes are being animated, then the matrices will have to updated to get the right result.

171

I was trying to see if was a big performance cost to always attach the transform node to the bone for all bone or just the transform socket I actually need an attachment for.

For example, I wanna attach a weapon to the wrist bone. I should only attach the wrist transform node to the wrist bone. And not just attach all transform nodes to bones if there is not actually a child mesh you wanna stick on the bone

172

Yo @bghgary … I got another PR i need to make to expose load mesh primitive so i can create my submeshes and multi materials BEFORE the createInstance stuff:

    /**
     * @hidden Define this method to modify the default behavior when loading data for mesh primitives.
     * @param context The context when loading the asset
     * @param name The mesh name when loading the asset
     * @param node The glTF node when loading the asset
     * @param mesh The glTF mesh when loading the asset
     * @param primitive The glTF mesh primitive property
     * @param assign A function called synchronously after parsing the glTF properties
     * @returns A promise that resolves with the loaded mesh when the load is complete or null if not handled
     */
    _loadMeshPrimitiveAsync?(context: string, name: string, node: INode, mesh: IMesh, primitive: IMeshPrimitive, assign: (babylonMesh: AbstractMesh) => void): Promise<AbstractMesh>;

AND the implementation start:

    /**
     * @hidden Define this method to modify the default behavior when loading data for mesh primitives.
     * @param context The context when loading the asset
     * @param name The mesh name when loading the asset
     * @param node The glTF node when loading the asset
     * @param mesh The glTF mesh when loading the asset
     * @param primitive The glTF mesh primitive property
     * @param assign A function called synchronously after parsing the glTF properties
     * @returns A promise that resolves with the loaded mesh when the load is complete or null if not handled
     */
    public _loadMeshPrimitiveAsync(context: string, name: string, node: INode, mesh: IMesh, primitive: IMeshPrimitive, assign: (babylonMesh: AbstractMesh) => void): Promise<AbstractMesh> {
        const extensionPromise = this._extensionsLoadMeshPrimitiveAsync(context, name, node, mesh, primitive, assign);
        if (extensionPromise) {
            return extensionPromise;
        }
   ...
}

Can i make that PR ???

173

Yes. If we ever change these functions, you might have a lot of work to handle.

174

Yep… Just cut and copy really… I hook my little pieces in there… Just please let me know if you change those functions :slight_smile:

175

There isn’t any way for me to know who are using them, so that might not be realistic.

176

Yo @bghgary … Can you look over my new Submesh And Multi Material Extension.

Its working beautiful… but i had to guess at the promise chain for creating materials in a loop synchronously and pushing promises in a loop to load the material properties for each loaded material asynchronously… Thats in my new _loadMultiMaterialAsync function. Can you look that over and see if im using promise chain API as intended ???

I also will include my COPY lof a custom _loadMeshPrimitiveAsync function I HAD TO COPY to get to call my _loadMultiMaterialAsync if the metadata has a multimaterial property and after you apply the mesh geometry i create the submeshes…

Again is working, but f you can look it over for me… as you are the GLTF Guru :slight_smile:

    public _loadMeshPrimitiveAsync(context: string, name: string, node: BABYLON.GLTF2.INode, mesh: BABYLON.GLTF2.IMesh, primitive: BABYLON.GLTF2.IMeshPrimitive, assign: (babylonMesh: BABYLON.AbstractMesh) => void): Promise<BABYLON.AbstractMesh> {
        const loader:any = this._loader;
        //console.warn("CVTOOLS: LoadMeshPrimitiveAsync: " + name);

        loader.logOpen(`${context}`);

        const canInstance = (node.skin == undefined && !mesh.primitives[0].targets);

        let babylonAbstractMesh: BABYLON.AbstractMesh;
        let promise: Promise<any>;

        const instanceData = (<any>primitive)._instanceData;
        if (canInstance && instanceData) {
            babylonAbstractMesh = instanceData.babylonSourceMesh.createInstance(name);
            promise = instanceData.promise;
        }
        else {
            const promises = new Array<Promise<any>>();

            const babylonMesh = new BABYLON.Mesh(name, loader._babylonScene);

            loader._createMorphTargets(context, node, mesh, primitive, babylonMesh);
            promises.push(loader._loadVertexDataAsync(context, primitive, babylonMesh).then((babylonGeometry) => {
                return loader._loadMorphTargetsAsync(context, primitive, babylonMesh, babylonGeometry).then(() => {
                    babylonGeometry.applyToMesh(babylonMesh);
                    if (primitive.extras != null && primitive.extras.metadata != null && primitive.extras.metadata.multimaterial != null && primitive.extras.metadata.submeshes != null) {
                        //////////////////////////////////////////////////////
                        // Mackey Primitives Modifications
                        //////////////////////////////////////////////////////
                        const submeshes:any = primitive.extras.metadata.submeshes;
                        babylonMesh.subMeshes = [];
                        for (let subIndex = 0; subIndex < submeshes.length; subIndex++) {
                            const parsedSubMesh = submeshes[subIndex];
                            BABYLON.SubMesh.AddToMesh(parsedSubMesh.materialIndex, parsedSubMesh.verticesStart, parsedSubMesh.verticesCount, parsedSubMesh.indexStart, parsedSubMesh.indexCount, <BABYLON.AbstractMesh>babylonMesh);
                        }
                        //////////////////////////////////////////////////////
                    }
                });
            }));

            const babylonDrawMode = (<any>BABYLON.GLTF2.GLTFLoader)._GetDrawMode(context, primitive.mode);

            if (primitive.extras != null && primitive.extras.metadata != null && primitive.extras.metadata.multimaterial != null && primitive.extras.metadata.submeshes != null) {
                //////////////////////////////////////////////////////
                // Mackey Primitives Modifications
                //////////////////////////////////////////////////////
                const multimaterial:any = primitive.extras.metadata.multimaterial;
                const materialids:string[] = multimaterial.materials;
                const materials:BABYLON.GLTF2.IMaterial[] = [];
                const matid = multimaterial.id || (mesh.name + "_multi");
                const matname = multimaterial.name || (mesh.name + "_multi");
                const multimat = new BABYLON.MultiMaterial(matname, loader._babylonScene);
                multimat.id = matid;
                materialids.forEach((materialId) => {
                    const material:any = BABYLON.GLTF2.ArrayItem.Get(`${context}/material`, loader._gltf.materials, parseInt(materialId));
                    materials.push(material);
                });
                promises.push(this._loadMultiMaterialAsync(`/materials/${matname}`, materials, babylonMesh, babylonDrawMode, (babylonMaterials) => {
                    multimat.subMaterials = babylonMaterials;
                    babylonMesh.material = multimat;
                }));
                //////////////////////////////////////////////////////
            } else if (primitive.material == undefined) {
                let babylonMaterial = loader._defaultBabylonMaterialData[babylonDrawMode];
                if (!babylonMaterial) {
                    babylonMaterial = loader._createDefaultMaterial("__GLTFLoader._default", babylonDrawMode);
                    loader._parent.onMaterialLoadedObservable.notifyObservers(babylonMaterial);
                    loader._defaultBabylonMaterialData[babylonDrawMode] = babylonMaterial;
                }
                babylonMesh.material = babylonMaterial;
            } else {
                const material:any = BABYLON.GLTF2.ArrayItem.Get(`${context}/material`, loader._gltf.materials, primitive.material);
                promises.push(loader._loadMaterialAsync(`/materials/${material.index}`, material, babylonMesh, babylonDrawMode, (babylonMaterial) => {
                    babylonMesh.material = babylonMaterial;
                }));
            }

            promise = Promise.all(promises);

            if (canInstance) {
                (<any>primitive)._instanceData = {
                    babylonSourceMesh: babylonMesh,
                    promise: promise
                };
            }

            babylonAbstractMesh = babylonMesh;
        }

        BABYLON.GLTF2.GLTFLoader.AddPointerMetadata(babylonAbstractMesh, context);
        loader._parent.onMeshLoadedObservable.notifyObservers(babylonAbstractMesh);
        assign(babylonAbstractMesh);

        loader.logClose();

        return promise.then(() => {
            return babylonAbstractMesh;
        });
    }
    private _loadMultiMaterialAsync(context: string, materials: BABYLON.GLTF2.IMaterial[], babylonMesh: BABYLON.Mesh, babylonDrawMode: number, assign: (babylonMaterials: BABYLON.Material[]) => void = () => { }): Promise<BABYLON.Material[]> {
        const loader:any = this._loader;
        const xmaterials:any = materials;
        //console.warn("CVTOOLS: LoadMultiMaterialAsync: " + babylonMesh.name);

        xmaterials._data = xmaterials._data || {};
        let babylonData = xmaterials._data[babylonDrawMode];
        if (!babylonData) {
            loader.logOpen(`${context} ${babylonMesh.name || ""}`);

            const babylonMaterials:BABYLON.Material[] = [];
            const promises = new Array<Promise<any>>();

            materials.forEach((material) => {
                const babylonMaterial = loader.createMaterial(context, material, babylonDrawMode);
                BABYLON.GLTF2.GLTFLoader.AddPointerMetadata(babylonMaterial, context);
                loader._parent.onMaterialLoadedObservable.notifyObservers(babylonMaterial);

                promises.push(loader.loadMaterialPropertiesAsync(context, material, babylonMaterial));
                babylonMaterials.push(babylonMaterial);
            });

            babylonData = {
                babylonMaterials: babylonMaterials,
                babylonMeshes: [],
                promises: promises
            };

            xmaterials._data[babylonDrawMode] = babylonData;

            loader.logClose();
        }

        babylonData.babylonMeshes.push(babylonMesh);

        babylonMesh.onDisposeObservable.addOnce(() => {
            const index = babylonData.babylonMeshes.indexOf(babylonMesh);
            if (index !== -1) {
                babylonData.babylonMeshes.splice(index, 1);
            }
        });

        assign(babylonData.babylonMaterials);

        return Promise.all(babylonData.promises).then(() => {
            return babylonData.babylonMaterials;
        });
    }
177

Looks reasonable. If you ever support morph targets, the morph target part needs to be updated also.

178

What do I need to update in the morph targets ???

179

Yo @bghgary … I add my Submesh Index stuff after the regular Morph Target Stuff… So should be ok:

                loader._createMorphTargets(context, node, mesh, primitive, babylonMesh);
                promises.push(loader._loadVertexDataAsync(context, primitive, babylonMesh).then((babylonGeometry) => {
                    return loader._loadMorphTargetsAsync(context, primitive, babylonMesh, babylonGeometry).then(() => {
                        babylonGeometry.applyToMesh(babylonMesh);
                        //////////////////////////////////////////////////////
                        // Mackey Primitives Modifications
                        //////////////////////////////////////////////////////
                        const submeshes:any = primitive.extras.metadata.submeshes;
                        babylonMesh.subMeshes = [];
                        for (let subIndex = 0; subIndex < submeshes.length; subIndex++) {
                            const parsedSubMesh = submeshes[subIndex];
                            BABYLON.SubMesh.AddToMesh(parsedSubMesh.materialIndex, parsedSubMesh.verticesStart, parsedSubMesh.verticesCount, parsedSubMesh.indexStart, parsedSubMesh.indexCount, <BABYLON.AbstractMesh>babylonMesh);
                        }
                        //////////////////////////////////////////////////////
                    });
                }));
180

I think you’re right. I was thinking about merging mesh data, but that’s not what you are doing.