Hand Tracking Not Showing as Available Feature

I’m trying to test hand tracking on Oculus Browser with my Oculus Quest 2. I’ve enabled the webxr-hands flag.
When I run the following playground (https://playground.babylonjs.com/#X7Y4H8#16) on version 5.0 alpha, I get the error:


I found the part of the code that was causing the error, so I commented that part out. Then when I ran it, my hands appeared. However, it seemed to me like the tracking wasn’t as good as when I’m in my Oculus home environment.

I tried to build and run this locally: I created a project with webpack and es6, similar to RaananW’s starter (GitHub - RaananW/babylonjs-webpack-es6: Babylon.js basic scene with typescript, webpack, es6 modules, editorconfig, eslint, hot loading and more. Will even make coffee if you ask nicely.), and added the following:


When I try to run on my Quest, it says the feature isn’t found:

When I console.log the available features from the FeaturesManager, I get the following:

Why isn’t hand-tracking showing up as a feature? I’m still using version 5.0 alpha locally. Thanks in advance!

As an extra note, other hand-tracking samples seem to work fine, such as this one from the immersive-web group: Immersive Session with hands

Maybe @syntheticmagus can help ? otherwise you ll have to wait for @RaananW to come back from vacation in a week :slight_smile:

Thanks for the ping, Seb! I don’t know exactly what’s going on with this, but I do repro it. Raanan will definitely be able to help once he gets back, but I’ll try to take another look sooner than that and post here again if I get more information about what’s going on.

2 Likes

Thanks a ton @syntheticmagus and @RaananW will be back next Monday so nw :slight_smile:

1 Like

Since you are using an es6 build (awesome :sunglasses:) you get the hand feature tree shaken out. You will need to include it for building (I.e. Reference it or import it in your project).

I have to say that I find it odd that the feature doesn’t work out of the box. What version of Babylon did you try? Playground-latest? A few changes were made to the hand tracking, but they were fully tested. It might be an issue with the loaded mesh. Would be really great if you can send me a reproduction playground so I can test exactly what’s wrong. Thanks!

2 Likes

Thanks Raanan! Importing the feature worked locally, I’m able to see my hands with version 5.0.0-alpha.35. I think there might be something wrong with the meshes, like you said:


My hands show up just fine, but as you can see, some skinned meshes appear on the ground. They don’t do anything but sit there. I also got this error:

As for the playground, I was using Playground-latest. The playground I was testing on is the sole hand-tracking example on this documentation page.

hmmm. I need to check why this is not working.
The error you are seeing is due to the node material compilation failing for some reason. I have been testing that and made sure it works, so it will be very important for me to understand why it fails in your project.

1 Like

Is there any way I can help with that?

2 Likes

If you could share the project (even a private repo on github or a zip file), it will be great

1 Like

Here it is:
hands-test.zip (103.8 KB)

1 Like

Thanks!

So, you are missing the Node Material import. I will need to investigate why it is not automatically included when we are using the node material in the mesh, but until then you will need to manually include @babylonjs/core/Materials/Node
About the example in the docs - I have just updated it. There was an error in the playground itself. I improved the hand meshes visibility as well in a PR that was merged not a few hours ago, so it should all work as expected now :slight_smile: (if you are using npm - in the next alpha release)

1 Like