How to correctly position a mesh to the side of a webXR controller

Questions
1

@RaananW my man! Or anyone else that knows and can clarify things for me…

When entering immersive WebXR mode, I see two meshes that represent my Oculus Quest controllers. I would like to create a mesh (a box for now) on the side of the controllers where my palm would be (so there is a little offset).

Using the WebXR emulator’s initial pose, the controllers are facing straight forward and I’m able to create and place my box meshes next to the controller meshes:

This looks right:

image
image1218×398 19.7 KB

And I do this using a function like this, where the input mesh is the “controller-0-tracked-pointer-right-grip” grip mesh created by the default web xr experience:

    makeGrabAreaMesh(mesh: AbstractMesh, handedness: string) {
      let myGrabBox = MeshBuilder.CreateBox("abc", { size: 0.1 }, this.scene)
      myGrabBox.position.copyFrom(mesh.position) // <-- not entirely sure why we need this?  But without it the boxes such appear at origin.  But why does playground example below work with simple box meshes?
      
      myGrabBox.visibility = 0.5
      myGrabBox.showBoundingBox = true
      myGrabBox.setParent(mesh)
      if (handedness[0] === 'l') {
        myGrabBox.locallyTranslate(new Vector3(0.1, 0, 0))
      } else {
        myGrabBox.locallyTranslate(new Vector3(-0.1, 0, 0))
      }
      return myGrabBox
    }

Notice that I parented my box mesh to the grip mesh and at first all seems well. Except in practice, on the actual Oculus Quest, one can never start out in a perfect initial position with controllers pointing exactly forward. And so my boxes were not perfectly on the side or symmetrical with each other.

In fact, even in the emulator, if I first tweak the rotation of the controller orientation before entering immersive mode, and then when I enter immersive mode the offset is still just to the side, but didn’t track with the rotation.

Tweaked emulator left controller so it faces more left

image
image584×524 40.9 KB

Entered Immersive mode, notice box is still offset to the side but not on the side of the controller

image
image1138×402 23.5 KB

I thought that when we parent a mesh, it “inherits” whatever rotation is on the parent?

Such as in this playground, the child snaps to whatever orientation the parent is in
https://playground.babylonjs.com/#47CRKL

But that doesn’t seem to occur in my webXR situation.

Can you shed some light on what this happens, what I’m missing? How I should correctly parent my box mesh so it tracks to the side of the controller no matter what the rotation?

Thanks in advance!

2

Hi owen,

I think what you’re probably looking for is the grip, a property available on WebXRInputSource objects. This is basically just a 3D space you can parent things to, the origin and orientation of which follows the motion controller in question. There’s another one for pointer, too, if what you’re attaching is more associated with pointing at things (like a selection ray) as opposed to something you hold; but the transform between the two should be rigid for most controllers (maybe all, not sure about hands), so either will probably work for your use case. Can you give those properties a try and see if they work for you?

1 Like
3

But… I am parenting to the grip though. Here is a playground.

https://playground.babylonjs.com/#F41V6N#296

To reproduce, you need to have the WebXR browser emulator. First turn one of your hand controllers in any irregular orientation. When entering immersive mode, the expectation is that the box mesh stays glued to the controller orientation, but what actually happens is that it is in the same starting position regardless of orientation.

image
image2154×1758 330 KB

4

I see; I think I understand your question better now. The Playground helps a ton, thanks! Does this more closely do what you’re expecting?

parent mesh to a controller grip | Babylon.js Playground

If so, then I think what you were encountering was some confusion with relative pose. Child transforms are stored in the local geometric space of their parent transforms; so when a child transform’s position is BABYLON.Vector3.Right(), that means it’s one unit (in XR, meter) to the right of it’s parent, not of the global origin. When you call myGrabBox.setParent(mesh), Babylon automatically calculates the relative transform between mesh and myGrabBox and sets myGrabBox's position and rotation to be values such that, despite the reparenting, myGrabBox does not appear to move. By copying the position of mesh before the reparenting, that operation caused the parenting operation to actually set the position to BABYLON.Vector3.Zero(); but because you didn’t do the same with the rotation, the rotation ends up set with a non-zero value. You could just copy the rotation upfront the same way you did for the position, but the more direct approach will be to just set the position and rotation after the reparent operation, at which point you will be acting on the objects in local space. I arbitrarily chose zero position and identity quaternion as minimal changes, but you could set them to anything you like, and they should maintain that relationship with the controllers regardless of relative starting positions.

1 Like
5

Thanks! Your playground does exactly what I want.

I think I also misunderstood:

childMesh.setParent(parentMesh)
// to be the same thing as
childMesh.parent = parentMesh

Now I think I understand that the former will attempt to visually keep the child where it is (no “snapping” to the parent). While the latter has the effect of visually updating the child to be offset relative to the parent.

1 Like