Blender 2.9+ and Shape key animations

[gryff]

I have been co-operating with @JCPalmer on getting shape key animations exported from Blender 2.9+ . So far - nothing. But Jeff and I have found out that if exported as a GLB file and loaded into the sandbox, the animations will work. We need that in the Babylon exporter.

Now if the GLB file is exported from the sandbox as a .babylon file and that file is viewed then you can see the animations for the three shape keys show up as a morphTargetManager array of animations. How do you get that whole array to play at once? (see image blow).

Jeff thinks it needs some kind of modifications to babylon - here to allow for the complete array to play (as I understand him). So any thoughts from the powers that be (@Deltakosh ) that could help?

I hate to keep nagging Jeff!

Stay Safe All, gryff

From the exported GLB-> Babylon file (note the three animation components of the array)

[Deltakosh]

Adding glb master @bghgary

[JCPalmer]

I now have data coming out Blender from the same .blend file, which is largely in the same format as an Inspector generated .babylon file of a corresponding GLB which had morph targets with animation.

I am going to switch to something else for now. I do not need to manually try to debug it, if someone is going to put a beginAnimation() at the MorphTargetManager level.

Should note that the picture of the layout above is not exactly what is coming. There in each target, the named animations do not match, but they are all named ‘KeyAction’ in my file. I can provide a zip file, untested.

[bghgary]

I’m not sure I follow what is happening completely, but I think you are trying to play animations with a .babylon file. Animations from a glTF loads into Babylon using animation groups. Does that help in playing all of the animations at once?

[JCPalmer]

Maybe, but where would an animationGroups which had all the grouped animations go? I could probably do either inside a MorphTargetManager or at top level in the JSON file.

[JCPalmer]

How about at the mesh which has the morph target manager assigned to it as the location to write the animation groups? That would be ideal.

[bghgary]

@JCPalmer

where would an animationGroups which had all the grouped animations go?

Animation groups are serialized at the scene level. Is that what you mean?

[JCPalmer]

Yes.

[JCPalmer]

Going the animationGroups route, I am writing a file which passes JSON parse, but gets a t is undefined in the sand box. If I leave the animationGroups off, it displays. Here is a simplified version of the file:

{
"producer":{"name":"Blender","version":"2.93.0 Beta","exporter_version":"2.93-alpha","file":"shapekeys07.babylon"},

"meshes":[
{
     "name":"Male_Head","id":"Male_Head",
     ,"positions":[-0.0371,0.0138,-0.0832]
     ,"normals":[-0.113,0.265,-0.958]
     ,"uvs":[0.443,0.56,0.434,0.574]
     ,"colors":[0.792,0.792,0.792]
     ,"indices":[0,1,2]
     ,"subMeshes":[{"materialIndex":0,"verticesStart":0,"verticesCount":2686,"indexStart":0,"indexCount":7446}]
     ,"instances":[],"morphTargetManagerId":554892
}
],

"morphTargetManagers":[
{
     "id":554892,"targets":[
     {
          "name":"head-open","id":661325,"influence":0
          ,"positions":[-0.0371,0.0138,-0.0832]
     },{
          "name":"head-wide","id":562828,"influence":0
          ,"positions":[-0.0371,0.0138,-0.0832]
     },{
          "name":"head-upper","id":883942,"influence":0
          ,"positions":[-0.0371,0.0138,-0.0832]
     }
     ]
}
],

"animationGroups":[
{
     "name":"KeyAction","from":1,"to":250,"targetedAnimations":
     [
          {"name":"KeyAction","property":"influence","dataType":0,"framePerSecond":24,"loopBehavior":1,"targetId":661325,"keys":[
               {"frame":1,"values":[0]},
               {"frame":250,"values":[0]}
          ]
          },
          {"name":"KeyAction","property":"influence","dataType":0,"framePerSecond":24,"loopBehavior":1,"targetId":562828,"keys":[
               {"frame":1,"values":[0]},
               {"frame":250,"values":[0]}
          ]
          },
          {"name":"KeyAction","property":"influence","dataType":0,"framePerSecond":24,"loopBehavior":1,"targetId":883942,"keys":[
               {"frame":1,"values":[0]},
               {"frame":250,"values":[0]}
          ]
          }
     ]
}
]
}

Does anything obviously look wrong?

[bghgary]

I don’t see anything obvious. Can you attach a babylon file so I can try?

[JCPalmer]

I just found something. The Id in morphTarget is a string not a number. I followed the pattern for the manager, which is number. Manually, I changed, but then I also did the manager to text too. Still failed.

Give me a few to change code to a string, which I will give meaningful stuff, not random numbers.

[JCPalmer]

shapekeys07.zip (160.5 KB)

I also switched to an html page, where I do get a load failure

<html>
<head>
    <meta charset="UTF-8">
    <link rel="shortcut icon" href="favicon.ico" />
    <script src="https://cdn.babylonjs.com/babylon.max.js"></script>
    <style>
         html, body   { width: 100%; height: 100%; margin: 0; padding: 0; overflow: hidden; } 
         #renderCanvas{ width: 100%; height: 100%; } 
    </style>
</head>
<body>
<canvas id="renderCanvas"></canvas>
<script>
var canvas = document.getElementById("renderCanvas");
var engine = new BABYLON.Engine(canvas, true);

var scene = new BABYLON.Scene(engine);
var url = "./"; 
        
BABYLON.SceneLoader.Append(url, "shapekeys07.babylon", scene);
scene.executeWhenReady(function () {
            
// Attach camera to canvas inputs
scene.activeCamera.attachControl(canvas);

const animGroup = scene.getAnimationGroupByName("KeyAction");
console.log(animGroup);

//Play the animation  
animGroup.start(true, 1.0, animGroup.from, animGroup.to, false);

// Once the scene is loaded, register a render loop
engine.runRenderLoop(function() {
      scene.render();
});
});

    //Resize
    window.addEventListener("resize", function () {
        engine.resize();
    });
</script>
</body>
</html>

The console message for the group does not show any targetedAnimations.

I am done for the day.

[bghgary]

This doesn’t load because the targetedAnimations is supposed to be an array of a TargetedAnimation object, which has two members animation and target.

Here is an example babylon file I exported from a minimal morph target animation sample glTF model from Khronos.

scene.babylon.zip (65.3 KB)

[JCPalmer]

Ok, this part is complete. @gryff , see the html above for how to play an animation group from your scene. In sandbox, you can just hit the play button on the page footer.

[gryff]

@JCPalmer : well I downloaded your file yesterday and @bghgary’s file too. I tried both in the sandbox - the morphing cube in @bghgary 's example works fine. I tried the shapekeys example I get the dreaded t is undefined message.

I downloaded your file again - in case you had changed it - but I still get that message.

In the mean time, I’m contemplating lipsyncing a piece of opera from the 1950’s

Stay safe Jeff, gryff

[JCPalmer]

I was now planning to create a tests/shapekeyAnimation in the exporter repository, and put this in it unless you object. This will give me a permanent test for this. I am in the process of cleaning up that .blend.

I have not transferred the albedo overloading from the source code generator yet, so the skin color has not been fine tuned.

Doing that will give me a test of that as well. Here is a working export of the scene, so far.
shapekeyAnimation.zip (320.8 KB)

[gryff]

@JCPalmer

I was now planning to create a tests/shapekeyAnimation in the exporter repository, and put this in it unless you object.

No problem - go right ahead Jeff , Anything that will help the BJS community

I will have several additional files you can also have soon - including a pangram, that will have all the letters of the alphabet, and hopefully the opera piece I mentioned above.

I was reading earlier today that Blender is hopefully moving to Blender 3.0 by the fall (and Blender 2.93 full release by end of this month). You are going to be a busy guy this summer.

Stay Safe Jeff, gryff

[JCPalmer]

I am hoping to finish up for 2.93 exporter this week. I am not planning on supporting 3.0. If this works for that fine, but the next release I care about would be 3.03 LTS.

[gryff]

@JCPalmer : well tried out your latest file above - works fine in the sandbox with no sound. I tried adding sound to your web page code above and it plays - though I would probably adjust my file a little.

The major issue is the animation picked up all the shapekeys set up for each phoneme at the end of the time line. It seemed to ignore the number of frames associated with the animation and sound file length(~120 frames) and went all the way to 250 even though those extra frames are just there for my ease in creating the animation. Did you do that deliberately?

But looking forward to the latest edition of the exporter

Now off to work with some opera!

Stay Safe Jeff, gryff

[JCPalmer]

That has been the way it has been done for a long time, at least 5 years. What the scene has for the start stop range is ignored. I think it came about because different actions have different lengths, so exporter examines the data to find range for each action.

At any rate, not going to change. I just removed those extra frames at the end from the .blend. I also published the directory to the dev branch. There is an index.html that runs the sound as well. You cannot directly load pages from a git hub repo, but it looks like this: