GLTF Skeletons And Transform Nodes

#1

Yo @bghgary … I got a couple questions about GLTF Skeletons…

1… What are you doing differently to a skeleton that links it with the transform nodes in the skinned mesh hierarchy ???

2… Why does playing animations on the transform nodes move the mesh. Its a transform node in the inspector, but you said something about those transform nodes are actually bones… How is that ???

Can you please point me to the code in GLTFLoader.ts that does this. I would like to see how and what you are actually doing to create a skeleton (that are linked to the transform nodes… i guess). I looked at the loadSkinAsync chain and i dont see anything that stands out :frowning:

0 Likes

#2

Here is the PR: Add support to override mesh for skeleton and link bones to transform nodes by bghgary · Pull Request #5513 · BabylonJS/Babylon.js · GitHub

0 Likes

#3

I guess my next question is Why?

Why do we have to do all that to create a skeleton… Why not just create the skeleton and put the animations on the bones.

It is to keep the TRANSFORMS in the loop so you can attach other meshes like a sword ???

0 Likes

#4

I ask because i am trying to figure out why Animations played using the standard GLTF transform nodes DONT HAVE TO BE RE-SKINED. But if i put the animations directly on the bones… The skin job has to be PERFECT or you get spaghetti models when the animation plays.

I think is has to do with the bone local matrix…In unity is the parent bone bindpose (ALREADY INVERSED) multiplied by the bone inverse… This is he way the Babylon Entities serialze skeletons… then no need to save all the inverseBindMatrices and have to update the bone matrix at runtime since each bone has is inverse local matrix setup on the bone itself.

GLTF seems to NOT be effected by this issue because the animations are encoded on the transform nodes then the transform node WORLD matrix is copied to the linked bone each frame…

This somehow does NOT get the spaghetti model… But if i go the pure skeleton route… I gotta RE-SKIN the model in Maya to get around the spaghetti model…

Have you got ANY clues as to how to handle spaghetti looking animations BESIDE re skinning the model in Maya… Use another matrix or something ???

0 Likes

#5

This is what used to happen, but it has problems that you reported. :slight_smile:

glTF skins are modeled from Unity which gets the bone transforms, etc. from nodes in the hierarchy. As seen in the issue you posted (GLTF Importer De-Parenting Skinned Meshes - Bugs - HTML5 Game Devs Forum), if the runtime doesn’t match exactly with what is expected from glTF, unexpected things might happen.

When the glTF issue Sharing skeletons between two skinned mesh · Issue #1285 · KhronosGroup/glTF · GitHub gets fixed in future versions, this will be much less of an issue, but I can’t think of a good way to fix this with glTF 2.0 and Babylon.

0 Likes

#6

Yeah… this was a tough one fo me too… I end solving the issue with metadata… for skeleton
then in my GLTF extension when serializing a skeleton, and assigning to the mesh… I check if that skeleton already exists and use that if so. But i am serializing the whole skeleton as metadata and NOT exporting a GLTF SKIN… Just the bone weights and bone indexes on the skinned mesh renderer. My extension create the skeleton just like the normal babylon parser then assign to mesh:


        private static DoParseSkeletonBones(parsedSkeleton: any, parser:BABYLON.MetadataParser, scene: BABYLON.Scene): BABYLON.Skeleton {
            const skeleton = new BABYLON.Skeleton(parsedSkeleton.name, parsedSkeleton.id, scene);
            skeleton.needInitialSkinMatrix = parsedSkeleton.needInitialSkinMatrix;
            if (parsedSkeleton.dimensionsAtRest) {
                skeleton.dimensionsAtRest = BABYLON.Vector3.FromArray(parsedSkeleton.dimensionsAtRest);
            }
            let index: number;
            for (index = 0; index < parsedSkeleton.bones.length; index++) {
                let parsedBone:any = parsedSkeleton.bones[index];
                let parentBone:any = null;
                if (parsedBone.parentBoneIndex > -1) {
                    parentBone = skeleton.bones[parsedBone.parentBoneIndex];
                }
                const localMatrix:BABYLON.Matrix = (parsedBone.matrix != null) ? BABYLON.Matrix.FromArray(parsedBone.matrix) : BABYLON.Matrix.Compose(
                    (parsedBone.scale != null) ? BABYLON.Vector3.FromArray(parsedBone.scale) : BABYLON.Vector3.One(),
                    (parsedBone.rotation != null) ? BABYLON.Quaternion.FromArray(parsedBone.rotation) : BABYLON.Quaternion.Identity(),
                    (parsedBone.translation != null) ? BABYLON.Vector3.FromArray(parsedBone.translation) : BABYLON.Vector3.Zero()
                );
                const bone = new BABYLON.Bone(parsedBone.name, skeleton, parentBone, localMatrix, null, null, index);
                if (parsedBone.id !== undefined && parsedBone.id !== null) {
                    bone.id = parsedBone.id;
                }
                if (parsedBone.length) {
                    bone.length = parsedBone.length;
                }
                if (parsedBone.metadata) {
                    bone.metadata = parsedBone.metadata;
                }
                if (parsedBone.animation) {
                    bone.animations.push(BABYLON.Animation.Parse(parsedBone.animation));
                }
            }
            // placed after bones, so createAnimationRange can cascade down
            if (parsedSkeleton.ranges) {
                for (index = 0; index < parsedSkeleton.ranges.length; index++) {
                    const data = parsedSkeleton.ranges[index];
                    skeleton.createAnimationRange(data.name, data.from, data.to);
                }
            }
            return skeleton;
        }

That way i end up with a native skeleton (NOT LINKED TO NODE TRANSFORM… Which i get rid of anyway)… When you have a native skeleton… no need for all those extra transform nodes… (Except if you wanna attach meshes like weapon to hand… which i am workin on a SOCKET mesh system for that… kinda like Unreal Sockets)

0 Likes

#7

If you can translate Unity into something that always works in Babylon, then the loader should be able to also, since glTF works like Unity for skinning. But I suspect it doesn’t always work. Perhaps I’m wrong.

0 Likes