I’ve tested Babylon’s WebXR Hand Tracking feature and it work pretty well. I saw you can use custom hands for it and I want to do that with Ready Player Me avatars.
The avatars don’t have a specific mesh for the hands, so I pass in the mesh for the body. But they do have a very similar skeleton for the hands as the one used in the Hand Tracking API. The only bones missing are the metacarpals that I set to an undefined string to ignore them.
I’ve created a Playground with my implementation which can be tested with the Immersive Web Emulator (GitHub - meta-quest/immersive-web-emulator: Browser extension that emulates Meta Quest devices for WebXR development. Lead: Felix Zhang (fe1ix@meta.com)). When clicking enter WebXR and trying to use hands, all the bones go quite crazy 
Am I doing anything wrong?
Playground: Babylon.js Playground
Hello! I think the missing metacarpals are going to be a problem, because they’re in the middle of the hierarchy Pinging @RaananW who knows all about XR if he has any ideas for this
I checked the hand model in Blender, and moving those joints doesn’t affect the hand, so I’m not sure what the metacarpals are for. I was hoping that it could just skip over those bones and apply the transforms to the other bones.
in order to get that to work correctly, the hands need to fit the WebXR hands spec, which might be different from the model you are trying to attach it to. the WebXR specs have an exact definition of what the model should look like. In babylon you will just need to map the right transforn nodes with the ones we expect.
The babylon hand meshes are available as a .glb file in our assets library ( Assets/meshes/HandMeshes at master · BabylonJS/Assets (github.com)) so you can get an idea as to how the hand mesh is expected to be constructed.
1 Like