I’ve been studying the awesome playgrounds above and also deep diving into what the babylon file loader does when parsing a serialized scene. My thinking was that if I could understand how babylon was creating the mesh and skeleton from a string, then maybe I could understand how to do it with the babylon skeleton apis.
But I’m still stuck trying to understand the vertex weighting and especially the vertex matrix indexing.
First, I tried to be very precise when creating the blender file. A default cube is 2M in all dimensions, I scaled it 6M high, applied all transformations, did a loop cut 5 times to add segments, created an armature (bone), (by default it “faces” up, so I left it that way). divided that 5 times and placed it perfectly inside the cubes by setting lengths to 2 then bound mesh and armature with automatic weighting. I exported this blender scene to .babylon file using the plugin and in the playground below I pasted the raw data into the source so that we can inspect what babylon is parsing.
Next I tried to reproduce an exact mesh using the babylon mesh builder. The createBox MeshBuilder does not have an option to segment the box, but it turns out you can use a cylinder (which has an option to create segments) and reduce the tessellation to 4, which will create a box like column.
In the following playground you will see two “boxes” side by side. One is created by parsing the serialized data that comes from blender. The 2nd one is created manually using babylon mesh builder.
https://playground.babylonjs.com/#HFWCMJ#1
It turns out creating the skeleton and making it appear in the mesh isn’t the hard part. Indeed it turns out if you use an identity matrix on the 2nd bone on… then the Sphere and Spurs display does not appear. But if you pass it a translated matrix the sphere and spur display appears (and that’s what the blender export is providing the Skeleton Parse method). But it also seems like the skeleton is only responsible for some hierarchy information. It doesn’t describe how it is influencing the mesh. That heavy lifting seems to be on the mesh itself.
The .babylon serialization for a mesh contains two keys probably related to this: matricesWeights and matricesIndices.
The length of the matricesIndices array matches the total vertex count (110) of the imported mesh, so my intuition tells me that the matricesIndices should specify something regarding each vertex in the mesh, but I can’t make heads or tails of what the value means. The values are all integers, and out of the 110 length array, there are only 7 unique values used:
[...new Set(data.meshes[0].matricesIndices)] => [4, 1027, 0, 256, 131328, 197121, 262914]
But why 7? There are 5 bones, so if you wanted to specify that you were under the influence of one of those bones, you could just use index into the skeleton hierarchy. But… that wouldn’t be sufficient to indicate that a vertex is under the influence of multiple bones… for that the value needs to represent a spread of choices… maybe… I can’t figure it out. Does anyone know?
matricesWeights is also an interesting array. The length is 4 times the length of the vertex count. After watching the babylon video on skeletons, I’m told that the default weighting of bones is “up to 4” (but can be “pushed to 8”). So I’m guessing that this array is telling me the distribution of weights for each vertex on 4 bones. But which 4 bones? I have 5 bones, so how do these weights correspond to the right bones in the hierachy? That must be what the matricesIndices number is about. Also interesting is that if you look at the array in batches of 4, the last number is always 0, while the first 3 are the only ones used and always add up to nearly 1. This observation seems to correlate with the numBoneInfluences = 3 in the imported data.
What do these numbers mean?