WebXR (replace controllers to default hands)

Questions
1

Hello to the creators of Babylon!

The documentation well describes how to interact with controllers, buttons, events, etc.It all works very well!

The only thing I would like to know is if there is some kind of standard hand mesh (not HAND TRACKING) with standard positions (fist, index finger, relaxed hand) and if there is, how can I replace my controllers with this mesh.

And another question. Can I change my controllers at will. For example, if I have oculus controllers, can I voluntarily change them to controllers like pico?

Do I need to use this repo somehow?

2 Likes
2

I do not think we provide default hands meshes to replace the controllers.

Adding @RaananW who is way more versed than me in WebXR :slight_smile:

Also @PirateJC as it might end up being a nice doc upgrade.

2 Likes
3

Sadly - not yet. We have an issue tracked here - WebXR option to display hand mesh instead of controllers · Issue #9089 · BabylonJS/Babylon.js (github.com)

You can force a controller to load with a parameter in the options (forceControllerProfile). changing on demand will require a bit more work on your side. The init function is this:

Babylon.js/webXRInputSource.ts at master · BabylonJS/Babylon.js (github.com)

1 Like
4

Hi @RaananW

Got a problem with WebXR controllers meshes. We got controllers meshes instead of hand meshes in version 5.34.0. Is there a fix for that yet?

Regards

5

Hi! :slight_smile:

sorry, just to be sure - you’re saying that older versions show hands, but 5.34.0 shows controllers instead?

6

Hi @RaananW

I do not know about previous versions but you were mentioning a bug in it.

ATM hands model does not show up for me. I see motion controllers but they work like hands. And the version I’m using is 5.34.0 and I use module imports.

On this playground:

I can see hands but not in my project and everything is setup more less the same.

Do I need to import something to have them?

Regards
Peter

7

Can you share the code? are you enabling the hand tracking feature?

xr.baseExperience.featuresManager.enableFeature(BABYLON.WebXRFeatureName.HAND_TRACKING, "latest", {
        xrInput: xr.input,
        jointMeshes: {
            enablePhysics: true
        }
    });
8

Hi @RaananW

This is exactly how I initialize it and I got hand functionality but I see controllers rotated180deg Z axis and moving like my hands are moving.

I also imported the entire @babylonjs/core into the project just not to miss anything and still nothing. Do I need to import something else?

Regards
Peter

9

no, nothing else needs to be added.

Would be great if you opened a new topic for that and if you could share the project with me so I can debug this. Of course, of you can.

10

Hi @RaananW

I got it working by importing the “@babylonjs/loaders” in my projects :confused: I’m using the import module in my project.

Now I got another one. The shader for the hands is not working. I got default Standard material and I can’t make it run as the playground this semitransparent purple once.

I got this:

image
image914Ă—648 29.3 KB

instead of this:

image
image1161Ă—833 44.8 KB

and they are not rigged at all. My fingers do not move.

What else do I need to import to make this run?

So what Do I do wrong now?

The file with the shader is requested and loaded in the network tab. No error or warning. Just silently is not applying it.

Regards
Peter

11

That feels like you are loading the wrong models and not enabling the hand support feature.
You will get the default hands from the webxr controller library if you don’t enable the hands module before starting the session.
I’ll write it again - would be great if you opened a new thread. Oh, and would be great to see the project or the code you are using. Playground would be wonderful.

1 Like
12

That was it. Thanks :slight_smile:

2 Likes