I am currently doing 2D Pose Estimation task in Deep Learning (Similar to motion capture) and I would love to visualize it on a humanoid using Babylon JS. I am new to Babylon JS but I have previous experience with Unity 3D. There are lot of resources for pose retargeting using the GUI but not much for doing it in code. I did google it but nothing significant came up. I hope someone could help me with how I should go about it.
Create a rigged avatar with the same skeleton the Pose Estimation framework is using. Make sure they have the same rest poses. I work from Blender 'cos the exporter from there is good In code you can dynamically apply new animations to your avatar like this:
- Load .babylon avatar (mesh and skeleton)
- Load .babylon animation file (mesh, skeleton and animations)
- Copy animation - avatar.skeleton.copyAnimationRange(animator.skeleton, “Dancing”, true);
- Playback animation - avatar.skeleton.beginAnimation(“Dancing”,true,1)
Ideally you’d want the animator file to only have skeleton and animations but the Blender exporter won’t allow exporting without a mesh, so export as usual, then you will have to remove the mesh data inside the .babylon file manually (or you could just edit the Blender exporter if you have those skills)
I clean out the mesh so it looks like this:
Come to think of it, maybe you can simply remove all vertices from the mesh within Blender instead of doing so within the text file. I haven’t tried it so I’m not sure if it’ll mess with the weighting of the skeleton or not, but its worth a shot.
that was brilliant.
I’m heading getting back to blender skeleton rig exports soon. Created a “Surfing”.
“Dancing” && “Surfing” istypeof cool
So that was a helpful concept. Thx.
Blender exporter is excellent. Can’t wait to get back in that pipeline.
as for MOTION-CAPTURE… people keep asking about this!
That is three times in my randomness counter.
So using my ONE phone-a-friend …bat-signal. (>8<)
has anyone interfaced KINECT into BABYLON.skeletons (yet)?
I googled it and it looks like there were some projects - maybe that would be a good start.
It would be remarkable. : )