VR Hand Finger Rotation and 3D Hand Model Sync

Questions
1

Hi Babylon.js Community,

Every time I encounter challenges beyond my understanding, I always come here for help! I’ve been working with WebXR hands for a while now and have run into a tricky issue with finger rotations.

I know I can replace my hands using jointMeshes: { handMeshes: { left, right } }, but I have a different goal and took a slightly different approach. I imported a 3D hand model (the same one Babylon.js provides) and applied some modifications to it. Specifically, I rotated it by 180° on the Z-axis and -90° on the X-axis.

I set the rotationQuaternion of the 3D hand mesh to match the VR hand’s wrist. The 3D hand’s fingers follow the positions of the VR hand’s fingers just fine up to this point. However, my issue comes when I try to handle the rotation of the 3D fingers. When I bend the VR hand’s wrist, the VR fingers’ rotation seems to be adding to the 3D fingers’ rotation, causing the fingers to appear “crushed” or distorted.

I believe my problem is with how I’m applying the rotationQuaternion, and I’m struggling with handling local vs. world rotations. I think I need to subtract the VR wrist rotation from the 3D finger rotation and isolate just the X-axis rotation for the fingers, but I’m not sure how to do that properly.

I know this is quite a complex issue, but I’m running out of ideas and would really appreciate any guidance. I’ll include a playground for reference if needed.

Thanks so much in advance!

2

Hello :slight_smile:

I didn’t take time yet to have a look at your PG with VR headset, but still I’m surprised about this :

This is ok, if custom hand model is needed, using the default one as a basis and modifying it, it the best way to my mind :+1:

But here I don’t get the thing :thinking: Why would you do so, and aren’t all your issues coming from here ?

++
Tricotou

1 Like
3

May I ask - what is the end goal? What is wrong with the hands we provide? or what exactly are you trying to achieve with the transformed mesh(es)?

1 Like
4

I rotated them in blender, because when I import the model in babylonjs and set the meshes[1].rotationQuaternion of the model to the hand.getJointMesh(wrist).rotationQuaternion. the 3d hand is backwards. So instead of rotating it in babylon. I rotated it on blender.

5

To be able to use it for multiplayer. I thought this is the easy way of doing it. Because in here I am only copying 11 positions or rotations coords from the vr hand. Which is the upper part of the vrhand, the fingers and the wrist. And then applying those positions and rotations to an imported 3d model that I can view to other players when they join the server. Before I did some test like completely setting the 3d model bones by saying skeletons[0].bones[0].getTranNode().position = hand.getJointMesh[wrist].absolutePos but this one is not working well I can only see the mesh if the vrcamera is still looking on the imported 3d model default position in the world so What I did instead of the skeleton bones I used the meshes[1].position = vrWrist.getAbsPos