HandConstraintBehavior does not work

Following the HandConstraintBehavior documentation in about most basic scene possible, I observe that the mesh just quickly flies out of the viewport; does not starting following a hand.

Tested with Meta immersive emulator Chrome extension with both hands visible; tested with Quest with one (either) hand at a time.

FWIW, PR 14907 with its current two commits does not affect the behavior.

@RaananW :person_shrugging:

Will sadly have to wait a little while. I’ll check that after the release :slight_smile:

1 Like

Could you share a playground where it doesn’t work? seems to be working correctly.

I can only assume the default values didn’t work for you (mainly the handConstraintVisibility?)

For future reference - is this working? WebXR basic example with teleportation | Babylon.js Playground (babylonjs.com)

The first word (“Following”) of the original post links to one. Apologies if that was too cryptic. Here is that playground again. About 6000 milliseconds after the pageload I observe that mesh crate0 flies away and out of the camera field of view; that is the moment when a HandConstraintBehavior is attached to that mesh.

I briefly tested with Meta immersive web emulator extension. Red ball follows left hand. If that’s what’s expected, then it’s working.

1 Like

The default behavior is to show the mesh if your hand is facing the camera. Otherwise you can configure it differently to behave the way you want it to. The idea is to have, for example, a menu that is shown when the hand is facing you.
A simpler (and more naive) approach would be to parent the mesh to the hand mesh - this will keep the mesh in place always.

Oh. That’s good to know. I guess. (either I missed that or no documentation says or implies that)

Let’s see. I am sorry to say that based on my experience last month I am 60% of the way to giving up on BJS when it comes to webxr (and webxr features are on ice for the time being). But I am not here to criticize but to try to help. While responsiveness and attention from the core team has been excellent (and you should know you guys are awesome), the code & documentation quality has been below a certain bar, sadly (in most of the few areas of BSJ that I used, but more so in the webxr - in my experience and in my opinion; and that is only my opinion).

With that, I have a few things.

in that playground i linked above, if I turn both hands with open palm towards the camera and move them around after the crate mesh is attached to a HandConstraintBehavior (and after that mesh had flown away), the mesh does not return into the field of view and does not start following any of the hands. Is that still the expected behavior given the code in the playground? If so, well then, what should the user do with their hands for the mesh to follow a hand if the mesh is attached to a default instance of HandConstraintBehavior as that playground does?

Speaking more broadly about BJS as a combination of the code and documentation. As a BSJ user, I believe I read all the documentation there is on HandConstraintBehavior, briefly reviewed its code, then came up with that sample code to see how it works. I observed what I considered was a bizarre behavior and thought that HandConstraintBehavior must be broken; and I gave up (though I already knew that many things in BJS are broken and still had hope in it so I filed this bug – falling days behind in delivering whatever features I intended to).

It seems that either a) I missed something important or simply lack competence necessary for basic usage of BJS or b) the combination of the HandConstraintBehavior code and documentation is inadequate/broken … or c) a bit of both, I guess.

@RaananW , do you think we have more of (a) or more of (b) here? Please be honest; I promise I won’t take offense.

If there is much of (a) going on, can you point me to where I went wrong? I would like to learn. I.e. if the code I came up with is a poor first attempt at using HandConstraintBehavior, can you guide me on how I could possibly come up with a better one, given the documentation (even after an improvement)?

On the other hand, if there is much of (b) going on, i.e. if the combination of the code and documentation is basically broken and leads most reasonable users to dead ends as they try to use many of the BSJ features (which has been much of my experience, I am sorry to say), then … doesn’t that warrant an urgent refocus of the team to this aspect of the project?

With :heart: and appreciation.


Sorry, took me a little longer than I wanted to answer that. Will try answering quicker next time :slight_smile:

Let me start with this - Babylon’s WebXR support has a few helper classes, and ways to simplify your dev process. This doesn’t mean it is full in any way. There are things to improve, and things to add. But it is a wonderful XR layer, especially if you want to use Babylon as your rendering engine. And I think that’s probably the main decision point whether you want to use Our XR implementation or not.

Now, about the HandConstraint - It was an external contribution, and a very welcomed one. Could it have been better documented? For sure. Does it needs an update/improvements? sure looks like it! And I will be able to take it over and do it. It was meant as a way to attach a menu to your hand, similar to many different XR experiences. Implementing it yourself is also more than possible, and is probably quite straight forward.

Want to submit an issue on github? please do! Add the playground, explain the scenario, and I promise we will review it, prioritize it, and will probably fix it. especially if you have found a bug, which we are allergic to. If it is more of a UX question, or something that could work better - we will of course address it as well (probably a bit slower than in case of a bug)

To your last question - The class is pretty well documented - HandConstraintBehavior | Babylon.js Documentation (babylonjs.com) The doc page - as i said - could have been a bit more extended. I am not quite sure what your code is testing, as you haven’t changed any parameter from the class itself, but I might have missed something looking into that.

So I don’t think it is more of a or more of b. I think that this is an open source project, and a wonderful one. And I think we are doing everything we can to get everything to work correctly. If the lack of hand constraint or an issue with it is the reason you decide not to use babylon, it saddens me for sure, but I can understand you. If the ease of use and the rendering quality of the engine itself didn’t help, then it all boils down to these features. We are not perfect, by any means, but I think I (and the entire team) have proven that we listen, we change, we fix, and as soon as we can.