# Position Mesh based on position and orientation of WebXR Controller

**URL:** <https://forum.babylonjs.com/t/position-mesh-based-on-position-and-orientation-of-webxr-controller/27321>\
**Category:** Questions\
**Tags:** webxr\
**Created:** [February 2, 2022, 5:55pm UTC](https://forum.babylonjs.com/t/position-mesh-based-on-position-and-orientation-of-webxr-controller/27321 "2022-02-02T17:55:45Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![vrhermit](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/vrhermit/32/35296_2.png) [@vrhermit](https://forum.babylonjs.com/u/vrhermit)\
**Post date:** [February 2, 2022, 5:55pm UTC](https://forum.babylonjs.com/t/position-mesh-based-on-position-and-orientation-of-webxr-controller/27321/1 "2022-02-02T17:55:45Z")

</div>

I have mesh that has some GUI objects attached. I setup a toggle visibility feature that will show and hide this object when pressing the Y button on a VR controller.

When I show this mesh using the Y button, I’d like to position it near to the player, relative to the left controller and based on the direction that the controller is pointing. I can place it in the exact location on the controller by using `controller.grip.position`. I thought that I might be able to use the forward vector to get a position in front of the controller, so I tried `controller.grip.forward` but that always positioned the mesh in the same direction in world space, regardless of where the controller was pointing/facing. Or to put it another way, I think the forward vector on the grip does not consider the orientation of the controller.

What is a better way to get the coordinates of a point in space relative to VR motion controller?

---

<div class="post-metadata">

**Author:** ![Deltakosh](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/deltakosh/32/26635_2.png) [@Deltakosh](https://forum.babylonjs.com/u/Deltakosh)\
**Post date:** [February 2, 2022, 6:45pm UTC](https://forum.babylonjs.com/t/position-mesh-based-on-position-and-orientation-of-webxr-controller/27321/2 "2022-02-02T18:45:08Z")

</div>

cc @RaananW

---

<div class="post-metadata">

**Author:** ![RaananW](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/raananw/32/28955_2.png) [@RaananW](https://forum.babylonjs.com/u/RaananW)\
**Post date:** [February 3, 2022, 11:59am UTC](https://forum.babylonjs.com/t/position-mesh-based-on-position-and-orientation-of-webxr-controller/27321/3 "2022-02-03T11:59:21Z")

</div>

> [@vrhermit](#):
>
> What is a better way to get the coordinates of a point in space relative to VR motion controller?

Yes, there is 🙂

The WebXRInputSource class has the function `getWorldPointerRayToRef`, which will give you a forward ray from the grip or pointer 🙂  
[https://doc.babylonjs.com/typedoc/classes/babylon.webxrinputsource#getworldpointerraytoref](https://doc.babylonjs.com/typedoc/classes/babylon.webxrinputsource#getworldpointerraytoref)

The `controller` variable is an XRInputSource, so you can use this function directly

---

<div class="post-metadata">

**Author:** ![vrhermit](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/vrhermit/32/35296_2.png) [@vrhermit](https://forum.babylonjs.com/u/vrhermit)\
**Post date:** [February 3, 2022, 2:08pm UTC](https://forum.babylonjs.com/t/position-mesh-based-on-position-and-orientation-of-webxr-controller/27321/4 "2022-02-03T14:08:45Z")

</div>

Thanks! It took me a little while to figure out how to use `getWorldPointerRayToRef` and I’m still not sure I understand it. I found this [playground](https://playground.babylonjs.com/#54YYZD#53) by searching for `getWorldPointeRayToRef`.

Here is what I did

```auto
             const tmpRay = new BABYLON.Ray(
                controller.pointer.absolutePosition,
                controller.pointer.forward,
                0.5
              );
              controller.getWorldPointerRayToRef(tmpRay, true);

              const newPosition = new BABYLON.Vector3(
                tmpRay.origin.x + tmpRay.direction.x,
                tmpRay.origin.y,
                tmpRay.origin.z + tmpRay.direction.z
              );

```

1. Create a ray using the controller pointer position and forward vector, with a length of 0.5
2. Called `controller.getWorldPointeRayToRef`
3. Used the ray origin and direction to calculate a position in space

This works for what I’m trying to do.

I am a little confused about step 2 though. What does this do? The name of the function seems to indicate that it would return a `ray`, but the documentation says it returns void and they in needs to have a ray passed _in_.

If I comment out `controller.getWorldPointerRayToRef(tmpRay, true)` my positioning code still works.

I guess I have what I need, but I don’t understand how or why it works and how `getWorldPointeRayToRef` is involved.

---

<div class="post-metadata">

**Author:** ![RaananW](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/raananw/32/28955_2.png) [@RaananW](https://forum.babylonjs.com/u/RaananW)\
**Post date:** [February 3, 2022, 3:12pm UTC](https://forum.babylonjs.com/t/position-mesh-based-on-position-and-orientation-of-webxr-controller/27321/5 "2022-02-03T15:12:05Z")

</div>

> [@vrhermit](#):
>
> I am a little confused about step 2 though. What does this do? The name of the function seems to indicate that it would return a `ray` , but the documentation says it returns void and they in needs to have a ray passed _in_ .

Yes, this is a “ToRef” function. it takes an already-existing ray and updates it. it is done this way to avoid too many garbage-collection calls every time you call this function. So you need to create an empty ray, pass it to the function, and then use the ray you have passed. And the function returns void, because the actual work is done with the variable passed to it.

---

<div class="post-metadata">

**Author:** ![vrhermit](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/vrhermit/32/35296_2.png) [@vrhermit](https://forum.babylonjs.com/u/vrhermit)\
**Post date:** [February 3, 2022, 5:10pm UTC](https://forum.babylonjs.com/t/position-mesh-based-on-position-and-orientation-of-webxr-controller/27321/6 "2022-02-03T17:10:58Z")

</div>

Interesting 🤔

From what I can tell, these two bits of code to the same thing, at least for my purpose.

## Option 1:

- Create a ray using the grip position and forward vector

```js
              const tmpRay = new BABYLON.Ray(
                controller.grip.absolutePosition,
                controller.grip.forward,
                0.25
              );

```

## Option 2:

- Create an **empty ray**
- Pass the empty ray to `getWorldPointerRayToRef` which will update its data (origin, direction, length?)

```js
             const tmpRay = new BABYLON.Ray(
                new BABYLON.Vector3(),
                new BABYLON.Vector3(),
                Infinity
              );

              controller.getWorldPointerRayToRef(tmpRay, true);

```

After using either of those options, I can calculate a position using the origin and direction on the ray.

```js
              const newPosition = new BABYLON.Vector3(
                tmpRay.origin.x + tmpRay.direction.x,
                tmpRay.origin.y,
                tmpRay.origin.z + tmpRay.direction.z
              );

```

The only difference seems to be if I need a reference or not.

For this use, I only need to calculate this position once. I don’t want to “attach an object” to the controller. Instead, I want to move an object to this position when pressing a button.

It seems like the reference would be useful for attaching something to the controller, having it updated in real time. Is there a reason to use Option 2 if I’m not doing that?

---

<div class="post-metadata">

**Author:** ![RaananW](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/raananw/32/28955_2.png) [@RaananW](https://forum.babylonjs.com/u/RaananW)\
**Post date:** [February 3, 2022, 6:13pm UTC](https://forum.babylonjs.com/t/position-mesh-based-on-position-and-orientation-of-webxr-controller/27321/7 "2022-02-03T18:13:27Z")

</div>

The difference between doing it yourself and calling the function is mainly cosmetic if everything is set “correctly”. For example, some controllers don’t have the pointer. in that case, it will still return a forward ray, but from the pointer. It also deals with RHS if needed. It is just a helper function, you can of course do it yourself 🙂

---

<div class="post-metadata">

**Author:** ![vrhermit](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/vrhermit/32/35296_2.png) [@vrhermit](https://forum.babylonjs.com/u/vrhermit)\
**Post date:** [February 3, 2022, 6:17pm UTC](https://forum.babylonjs.com/t/position-mesh-based-on-position-and-orientation-of-webxr-controller/27321/8 "2022-02-03T18:17:04Z")

</div>

> [@RaananW](#):
>
> RHS

Got it. I’ll generally do things the recommended way. I wanted to make sure I wasn’t missing anything. Thanks for your help explaining this to me.