Unity Exporter Skeletal Animation Bug

#1

The Babylon Editor Toolkit (Unity Exporter)

Has been updated.

KNOWN ISSUE:

Skeletal animation are still a problem (and have been from the beginning).

The code for creating a babylon skeleton in C# using the Babylon Entities DOES NOT take into account the bones local matrix along with the bind pose matrix.

I have been begging for help for quite some time now, to help me fix the problem.

Unfortunately i don’t know the MATH for taking into account that local matrix. I tried multiply the bone pose matrix with the local bone matrix and i tried the parent bone matrix. I cant seem to hit the right combination. Note: I am just guessing here.

So if anyone has issues with skeletal animations… That the reason why.

If there are any smart cookies out there. Please send me the fixed up code for SceneBuilder.Meshes.cs:

        private BabylonSkeleton ConvertUnitySkeletonToBabylon(SkinnedMeshRenderer skin, GameObject gameObject, float progress, ref UnityMetaData metaData)
        {
            ExporterWindow.ReportProgress(progress, "Exporting Skeleton: " + gameObject.name);

            Transform root = skin.rootBone;
            Transform[] bones = skin.bones;
            Matrix4x4[] bindPoses = skin.sharedMesh.bindposes;
            Transform transform = skin.transform;

            BabylonSkeleton babylonSkeleton = new BabylonSkeleton();
            babylonSkeleton.name = gameObject.name;
            babylonSkeleton.id = Math.Abs(GetID(transform.gameObject).GetHashCode());
            babylonSkeleton.needInitialSkinMatrix = false;
            
            // Prefilled to keep order and track parents.
            var transformToBoneMap = new Dictionary<Transform, BabylonBone>();
            for (var i = 0; i < bones.Length; i++)
            {
                Transform unityBone = bones[i];

                string exmsg = "Exporting bone: " + unityBone.name + " at index " + i.ToString();
                ExporterWindow.ReportProgress(progress, exmsg);

                var babylonBone = new BabylonBone();
                babylonBone.name = unityBone.name;
                babylonBone.index = i;
                babylonBone.metadata = new BoneMetaData() { transformPath = Tools.FormatTransformPath(unityBone, root, true) };
                transformToBoneMap[unityBone] = babylonBone;
            }
            
            // Attaches Matrix and parent.
            for (var i = 0; i < bones.Length; i++)
            {
                var unityBone = bones[i];
                var babylonBone = transformToBoneMap[unityBone];
                
                Matrix4x4 localTransform;
                
                // Unity BindPose is already inverse so take the inverse again :-)
                if (transformToBoneMap.ContainsKey(unityBone.parent))
                {
                    var babylonParentBone = transformToBoneMap[unityBone.parent];
                    babylonBone.parentBoneIndex = babylonParentBone.index;
                    localTransform = bindPoses[babylonBone.parentBoneIndex] * bindPoses[i].inverse;
                }
                else
                {
                    if (unityBone != root) {
                        UnityEngine.Debug.LogWarning("Parent bone transform '" + unityBone.parent.name + "' missing weights for child bone '" + unityBone.name);
                    }
                    babylonBone.parentBoneIndex = -1;
                    localTransform = bindPoses[i].inverse;
                }

                // Support socket meshes
                var gox = unityBone.gameObject;
                var socket = gox.GetComponent<SocketMesh>();
                if (socket != null && socket.enableSocket == true) {
                    float posX = socket.socketPosition.x, posY = socket.socketPosition.y, posZ = socket.socketPosition.z;
                    float rotX = (socket.socketRotation.x * (float)Math.PI / 180), rotY = (socket.socketRotation.y * (float)Math.PI / 180), rotZ = (socket.socketRotation.z * (float)Math.PI / 180);
                    metaData.socketList.Add(new SocketData { 
                        boneIndex = i,
                        boneName = (!String.IsNullOrEmpty(socket.socketName)) ? socket.socketName : unityBone.name, 
                        socketMesh = null, 
                        positionX = posX, 
                        positionY = posY, 
                        positionZ = posZ,
                        rotationX = rotX, 
                        rotationY = rotY, 
                        rotationZ = rotZ
                    });
                }
                
                // Transform matrix bone maps
                transformToBoneMap[unityBone].matrix = new[] {
                    localTransform[0, 0], localTransform[1, 0], localTransform[2, 0], localTransform[3, 0],
                    localTransform[0, 1], localTransform[1, 1], localTransform[2, 1], localTransform[3, 1],
                    localTransform[0, 2], localTransform[1, 2], localTransform[2, 2], localTransform[3, 2],
                    localTransform[0, 3], localTransform[1, 3], localTransform[2, 3], localTransform[3, 3]
                };
            }

            // Reorder and attach the skeleton
            babylonSkeleton.bones = transformToBoneMap.Values.OrderBy(b => b.index).ToArray();
            babylonScene.SkeletonsList.Add(babylonSkeleton);

            return babylonSkeleton;
        }

More specifically

                Matrix4x4 localTransform;
                
                // Unity BindPose is already inverse so take the inverse again :-)
                if (transformToBoneMap.ContainsKey(unityBone.parent))
                {
                    var babylonParentBone = transformToBoneMap[unityBone.parent];
                    babylonBone.parentBoneIndex = babylonParentBone.index;
                    localTransform = bindPoses[babylonBone.parentBoneIndex] * bindPoses[i].inverse;
                }
                else
                {
                    if (unityBone != root) {
                        UnityEngine.Debug.LogWarning("Parent bone transform '" + unityBone.parent.name + "' missing weights for child bone '" + unityBone.name);
                    }
                    babylonBone.parentBoneIndex = -1;
                    localTransform = bindPoses[i].inverse;
                }

The localTransform needs to take into account the actual local matrix (position, rotation and scale) of the PARENT bone (I think)

I am counting on the community to help fix this issue. Otherwise IT WILL ALWAYS be a problem with skeletal animations :frowning: