WebXR Hand Tracking Error on Safari/Vision Pro

Actually, i have created this small playground and ran it on the Apple Vision Pro device immersive mode: https://playground.babylonjs.com/#ZI9AK7#4163

It seems that in a older version of babylonjs the hand tracking was almost working. I mean the hands motion was ok, there was also a ray coming out of the hand (one at a time) but there was no pinching detection and when the pinching happened the ray would move a bit together with the fingers (not so stable). See my video for version 5.71.1: https://www.youtube.com/watch?v=6NEa0xqwtI4

For the latest version of babylon the implementation is much different, mostly the way you have described it above in this long thread :slight_smile: It is almost nice because it simulates the way the headset works but using this behavior it is really hard to hover objects and when you pinch your hand and keep it like that … the cursor goes all over the screen. It works the best when you use the “Head tracking”. The “Eye tracking” is really hard to control at least with my eyes and for the “Wrist” and “Finger” tracking the cursor is not so accurate on the directions it selects. See my video for version 7.26.3: https://www.youtube.com/watch?v=Gh3pJYMso9U

So, to bypass all these experimental trackings from Apple, it would be really nice to have a alternative control for the headset. Like you did have in the older versions but have also pinching working :sweat_smile: of course.

I am a developer, but not so specialized in this 3d world to go so deep into the inner workings babylon js but if i can provide some feedback to make this happen, i would be happy to. :vulcan_salute: