Did a little test to see if I could keep my instances, while still being able to choose/change different animations, and thanks to the new instancedBuffer it’s a success so far.
A 254x248 texture at about 1MB. Rendered out using the Game Development Toolkit in Houdini.
I’m using tinyexr.js, so there are some memory issues, and you might need to hard-refresh (ctrl-f5) if it fails.
Found another script capable of rendering vertex animations to BMP images from within Max. So no more Houdini middle-man and external EXR loaders needed.
Simply provide the UV index, start and end- frames and you’ll have your BMP image, model, scale and offset. (Scale and offset are used instead of a bounding box)
It takes an imported mesh with skeleton and animation and returns a VAT texture which you can save for use with your own custom shader. Once that’s done, a little test is performed against a nodematerial and instancedBuffer to create many instances with independent animations.
There are still a lot of stuff to add/tweak/test. For instance (no pun intended), the animation doesn’t consider bone_influencers > 4 since I’m not grabbing the MatricesIndicesExtraKind / MatricesWeightExtraKind yet. The VAT is also fairly low res (I think), and a lot more work is needed to make a better texture (I’m guessing exr). There is some jerkiness in the way the instances move. I’m guessing the texelfetch result from the VAT is not as high res. yada, yada, yada…
The good part is: no need to swizzle the texture, no need for 3rd party scripts/hacks etc. If you have a mesh with bone animations that is imported into the PG, you can make a VAT. I think it’s a start…
Plus I’m beat, anyone is free to try and take it to another level. I think it shouldn’t be too hard to make a VAT generator dedicated for bjs users on a url somewhere?
Very nice. Some of the jerkiness could be due to the speed being 30 instead of 60. At least that change makes it look better. I feel like EXR loading is a mess, but I failed at stealing the loader from ThreeJS, so for the time being, I use the transpiled Tinyexr.js in a worker and a modified version of the Maxscript to save to 32bit floating point EXR. Blending can’t really be done, unless all your animations are close to identical, or if you have transformation-animations between all of them.
VAT certainly has limitations, but I would like to see if I can get an actual first-person character rolling using it. If you separate your animations into for example upper-body animations and lower-body animations, you can even play multiple, and different animations at the same time.
I found some shader code on Github to rotate vertices in a specific direction, so at least I’ll have some look-at functionality.
I found a major issue with the VAT gen. After more testing, it seems like doing the diff computation in scene registerBeforeRender function is not the correct approach. There are times when the animation cb triggers b4 the scene function ends. This needs a rethink. Also, nodematerial vs ShaderMaterial perfs need study, might be a reason for jerkyiness…shrugs. I need to find time, maybe this weekend and re-do.
That’s good question. I know older games like Mario 64 had all limps separate, so that hey could be parented and animated that way. I’ve tried that with BabylonJS, but even with instances, the parent-child hierarchies make it perform very poorly. In that video the performance does seem to suffer quite a lot when it hits the 1000-2000 entities.
But ended up using secondary attributes instead of filling the matrix with the animation data. https://playground.babylonjs.com/#57P8Y8#6
You can offload the animating part to the shader as well, but I prefer this easy way of changing animations. In this example I can have 5000 instances with different animations while still hitting 50fps.
With a normal cloned rabbit mesh+skeleton, this is closer to 500-600 for me, but it’s not exactly a fair test as the instance shader is Very simple, so I’ll have to either simplify a shader for the cloned version, or perhaps just recreate it in a node material. That should be straight forward.
Now I just need to figure out how to have different bone rotations on each instance, so for example, an FPS character can bend his spine and look up and down without effecting the actual skeleton/other instances. My guess for now is, that I can send a matrix as an attribute, and then multiply it by the desired bone, if I can somehow ID that bone it is. Since I already send the matrices and indices of the bones, I guess that shouldn’t be an issue, though
@Evgeni_Popov
Do you know if something happened in the exporters between the original rabbit and the current exporters?
To simplify my issue, I downloaded the original rabbit and created a simple skeleton with the same amount of bones(4) and a 0-100 animation. But it doesn’t seem to work out of the box. https://playground.babylonjs.com/#57P8Y8#7
One thing I noticed is that I have to pass the mesh to the getTransformMatrices function. This isn’t a requirement with the original model. With other models I see slight movement in the direction of the animation, but checking the texture in the inspector reveals the transforms in the matrices to be very small, resulting in almost no pixel change. Do newer versions use delta values or something?
I’m using 3Ds max, and tried exporting with most features as well as with both optimized and unoptimized animations.
WOW, you solved it. This was what I wanted back in April. This is SO much better, no need for scale/offsets, no need for frame remapping and the image file size is ridiculously small. runAnim per frame causes render flickering which is annoying but otherwise, as a VAT generator for skinned meshes, its great ! I’m using this, many thanks @Raggar !
what’s left:
instances + nav mesh + walk/attack/spawn/death cycles + logic = WARCRAFT 3 !!! on the web, no less…rawr !!!
On 2nd tots, since the boneTexture alr exists in bjs, I think it would be even simpler to inject vec4 animParams into bonesVertex.fx and bonesDeclaration.fx, duh !
Using boneTextures + instances with instanceBuffers + navmesh + code grabbed from anywhere = frankenstein, lol. Increase size of crowd and navmesh as desired. I was able to run 500 on my system at 60 fps w/o problems. (Wait for the boneTexture to be computed, all the meshes will turn white then the correct crowd nav will kick in)
Observed issues:
a) When switching tabs, animations don’t seem to refresh for all instances.
b) Clustering at large crowd size is expected. I guess there’s an inherent limit to navmesh in terms of pathfinding. Maybe switching to flow fields might be better.
@Deltakosh Hi boss, what do we have in terms of environment assets available for testing ? A multilevel maze would be nice for navmesh and crowds.
Has anyone tried to save the baked animation texture as image and then use it directly? I.e. bake offline and avoid baking during loading phase.
I tried the following, but no luck. My model disappeared:
Open Babylon Inspector, find the baked texture. Click Edit. It will open another tab for texture image editing. There is a save button to save the texture as image.
Directly create texture
const texture = new BABYLON.Texture( "url to the saved texture image from step 1", scene);
I also tried to set these two properties to be the same I used for RawTexture. Doesn’t make any difference. What is the difference between RawTexture and Texture?
texture.updateSamplingMode(
BABYLON.Constants.TEXTURE_NEAREST_SAMPLINGMODE
);
(texture._texture as any).type = BABYLON.Constants.TEXTURETYPE_FLOAT;
This was what I wanted back in 
but if someone wants to donate one it could be cool!