Skeletons, bones, animation by zero

libero51 · April 19, 2022, 6:17am

Hello everybody.
This is the summary of my understanding of skeletons and bones
https://playground.babylonjs.com/#8HKA9I
The construction of the skeleton and associated mesh is done with the basic building data.
Animation is done through CPU engagement. I’d like to see the code of a similar simple GPU-managed animation, without using other programs … just code.

I still haven’t been able to export the skeleton in .babylon format correctly,
but this is a later problem.

slin · April 20, 2022, 5:49am

Hi @libero51

It is a big topic. I will try to answer.

Let me start with the CPU implementation first. If you open the inspector in your PG, and select the ‘no name’ texture under Textures. You would see something like the following.

In the PREVIEW section, you will find this texture has size width: 16, height: 1. This texture actually stores values of (n+1) transformation matrix for number of bones n. Each pixel can hold RGBA, so 4 float values. It requires 4 pixels to hold a transformation matrix. E.g. You have 3 bones. So this texture stores 3 + 1 = 4 matrix. Each matrix requires 4 pixels to store 16 float values. Thus, this texture holds 4 transformation matrix.

babylon.js javascript code generates the transformation matrix for the animation for current time t. This texture is passed as uniform to shader language. The shader code in bonesVertex.fx reads the transformation matrix for each bone by sampling the texture.

Fully GPU managed animation I guess you mean baked animation. The idea is to prepare transformation matrix for the full animation at once. For example, if your animation doesn’t have any randomness or doesn’t behave differently based on user inputs. You can prepare (‘bake’) the animation. See babylon.js documentation about baked animation. Basically, if your animation completes for 2 seconds = 120 frames. You calculate the transformation matrix for each of these 120 frames, and prepare a texture that is width 16, height 120. To have a smaller texture e.g. you might only calculate transform matrix of 60 frames (30 frames/s). It will reduce the smoothness of the animation.

Once this baked texture is prepared, there is no need for javascript to calculate transformation matrix at each frame and pass it to GPU side. Instead, javascript side only needs to provide the current time time. Vertex shader can sample the corresponding height of the texture to get the animation matrix for a particular time frame. babylon.js baked matrix vertex shader code is at:

github.com

BabylonJS/Babylon.js/blob/master/packages/dev/core/src/Shaders/ShadersInclude/bakedVertexAnimationDeclaration.fx

#ifdef BAKED_VERTEX_ANIMATION_TEXTURE
    uniform float bakedVertexAnimationTime;
    uniform vec2 bakedVertexAnimationTextureSizeInverted;
    uniform vec4 bakedVertexAnimationSettings;
    uniform sampler2D bakedVertexAnimationTexture;

    #ifdef INSTANCES
        attribute vec4 bakedVertexAnimationSettingsInstanced;
    #endif

    #define inline
    mat4 readMatrixFromRawSamplerVAT(sampler2D smp, float index, float frame)
    {
        float offset = index * 4.0;
        float frameUV = (frame + 0.5) * bakedVertexAnimationTextureSizeInverted.y;
        float dx = bakedVertexAnimationTextureSizeInverted.x;
        vec4 m0 = texture2D(smp, vec2(dx * (offset + 0.5), frameUV));
        vec4 m1 = texture2D(smp, vec2(dx * (offset + 1.5), frameUV));
        vec4 m2 = texture2D(smp, vec2(dx * (offset + 2.5), frameUV));
        vec4 m3 = texture2D(smp, vec2(dx * (offset + 3.5), frameUV));

This file has been truncated. show original

github.com

BabylonJS/Babylon.js/blob/master/packages/dev/core/src/Shaders/ShadersInclude/bakedVertexAnimation.fx

#ifdef BAKED_VERTEX_ANIMATION_TEXTURE
{
    #ifdef INSTANCES
        #define BVASNAME bakedVertexAnimationSettingsInstanced
    #else
        #define BVASNAME bakedVertexAnimationSettings
    #endif

    // calculate the current frame for the VAT
    float VATStartFrame = BVASNAME.x;
    float VATEndFrame = BVASNAME.y;
    float VATOffsetFrame = BVASNAME.z;
    float VATSpeed = BVASNAME.w;

    float totalFrames = VATEndFrame - VATStartFrame + 1.0;
    float time = bakedVertexAnimationTime * VATSpeed / totalFrames;
    // when you loop an animation after the first run (and for all subsequent loops), we wrap to frame #1, not to frame #0
    float frameCorrection = time < 1.0 ? 0.0 : 1.0;
    float numOfFrames = totalFrames - frameCorrection;
    float VATFrameNum = fract(time) * numOfFrames;

This file has been truncated. show original

This PG was created before babylon.js supports baked animation. It might be easier for you to play with and understand the shader behavior. It initially uses babylon.js javascript-based animation as a baking tool to play animation once and prepare the baked texture. Once it is done, it uses the custom shader code to apply transformation to each vertex.

libero51 · April 22, 2022, 9:06pm

Hi slin , thanks for your explanation.
I can build simple 3d structures and can coat them with a texture using a simple text editor. I am aware that such a method is not suitable for building figures with thousands of vertices and triangles.
Even just for my little solids I had to build a little 3d-2d editor that relates the 3d vertices with the corresponding ones on the 2d texture. My goal is to animate small figures with few vertices and few triangles.
I am allergic to using a digger’s bucket to drive a nail … a hammer is enough … With this philosophy and with this goal I would like to see a dozen lines of simple source code that would replace my CPU animations with a simple one animation managed by the GPU. I am aware of how much complexity there is behind the excavator …, or behind the baked texture. Yesterday I spent three hours looking for a manual to regulate a water heater managed by a small display board … An electric water heater for me is a container with a resistance and a thermal switch … that was … another excavator … I just wanted to use the hammer … eheheh.
Just kidding, for fun … or almost.
Let’s go back to the little solids
If the vertices and triangles are few, there are also relatively few animation operations managed by the CPU. I think the math coprocessor takes a few machine cycles to solve “my” simple trigonometric functions.
I don’t rule out changing my mind, but if I write some code I would like to understand what I’m doing …
For our fisher friends here is a small example with about forty vertices and triangles …

The fish…

.
Greetings to all

libero51 · June 21, 2022, 6:25pm

Hello everyone,
I have not found any examples of native skeleton animation, even a simple one.
… Blender, blender, blender … I was practically forced to use it.
I built with Blender a small skeleton with three bones connected to a mesh consisting of three cubes. I animated two bones with a simple lateral rotation and I exported everything in .babylon format.
Only three cubes stacked vertically. I would have expected 24 vertices: 4x4 = 16 + 4 side seams + 2 + 2 seams for the lower and upper faces.
Blender uses 67 vertices …
I don’t criticize Blender but it doesn’t seem geared towards effective LowPolyMesh
Trivial point of view …
I analyzed the .babylon file very patiently, and I roughly removed everything that did not serve my purpose. I replaced the geometry data of the mesh and I reduced everything in a Json datafile.
I reduced everything into a first rough playground.
Native Skeleton Animation

The structure of the transformation matrix of each frame is explained here:
Wiki Rotation Matrix
Usna Rotation Matrix

It was one of the hardest goals I’ve ever achieved … and it’s just the beginning … Now I have a method to animate my LowPoliMeshes.
Greetings

libero51 · June 24, 2022, 5:52am

Hello everyone,
I formatted the code better and added the relationship between the transformation matrix and a generic animation frame contained in a .babylon file.
Rotation around the Z axis.
Greetings

libero51 · June 25, 2022, 4:13pm

Hello everyone,
With this post I think I have achieved the goal.
Having to work in synthesis, I formatted the data columns by reducing the decimals of the source data.
It is built from scratch, the mesh, the skeleton, the bones and their animations.
I couldn’t find a direct way to proceed without using Blender.
I actually took apart a very simple animated skeleton obtained with Blender.
I made an effort to make the data structures readable.

Playground

Greetings…

Topic		Replies	Views
Baked Texture Animations with Animation Groups? Questions	37	2373	November 27, 2023
Baked animation support Feature requests	35	1552	October 11, 2021
Baked Texture Animations to Reduce CPU Usage with Many Concurrent Animations Questions	2	311	August 18, 2023
Skeletal Animation + Babylon Animation Questions	5	534	October 12, 2020
Best way to bake vertex animation? Questions	8	2026	July 6, 2020

Skeletons, bones, animation by zero

Related Topics