WebXR - Grab an Object

Questions
1

Hi there. I’m trying to use Pointer events to grab an object in WebXR. I’m trying the code below, and the issue I’m seeing is that the originMesh is never anything besides ‘null’. Even if I were trying event.pointerId, it doesn’t match the controller’s id. an event’s pointerId would be something like ‘3’, and the motion controller’s pointer.id would be something like “controller-3-tracked-pointer-left-pointer”. How do I know what controller created the PointerEvent?

scene.onPointerObservable.add((pointerInfo) => {
    switch (pointerInfo.type) {
    case PointerEventTypes.POINTERDOWN:
        if (pointerInfo.pickInfo.hit && pointerInfo.pickInfo.pickedMesh) {
            if (pointerInfo.pickInfo.originMesh) {
                this.pickInfo.originMesh.addChild(pointerInfo.pickInfo.pickedMesh)
            }
        }
        break
    default: break
    }
})
1 Like
2

Here’s a Playground to demonstrate what I’m trying to do.
https://playground.babylonjs.com/#C2YPLR#1

3

Your PG has errors.

I have corrected it:

https://playground.babylonjs.com/#INITB4

I get Session mode “immersive-vr” not supported in browser in the Chrome console log: maybe it’s expected, I don’t have any VR/XR related device connected to my computer.

4

Thanks for the updated code. I’m still not seeing an originMesh, which I think might be expected. I don’t know when an originMesh would exist.

I’ve done more dwelling, and think that I might need to simplify my scene and disable all pointers besides one, and allow the user to enable the pointer by hitting the trigger of the controller they prefer to use (like SteamVR’s Home). Then I can assume the click is happening from the only enabled pointer.

5

I found a useful function, pointerSelection.getXRControllerByPointerId. I was able to use that to find the controller that triggered the event. Here’s the updated playground

https://playground.babylonjs.com/#INITB4#1

1 Like
6

That is the right function to use.
If you want to stay defensive (and maybe support mouse/touch input), make sure you are in XR before setting the parent.

I made a few changes to use our best-practices - https://playground.babylonjs.com/#INITB4#2 (setting floor meshes during construction, filtering pointer events, checking that IN_XR before using XR functions).

1 Like
7

this works great in my scene in the quest. BUT, I can’t let go or release an object. It sticks to my controller and stays “stuck”.

How can I release it?

thanks,
Dale

1 Like
8

Hi Dale!

Release the parenting when pointer up is triggered :slight_smile:

9

Thanks, @RaananW,
I know this is probably super basic. Here is my playground that works, but I can’t release things - https://playground.babylonjs.com/#9K3MRA#301

this is my feeble attempt to release the pointer - I can still pick things, but it crashes when I release. Can you point me in the right direction?

scene.onPointerObservable.add((pointerInfo) => {
if (pointerInfo.pickInfo.hit && pointerInfo.pickInfo.pickedMesh) {
var pickedMesh = pointerInfo.pickInfo.pickedMesh
let xrInput = xrHelper.pointerSelection.getXRControllerByPointerId(pointerInfo.event.pointerId)
let motionController = xrInput.motionController

        switch (pointerInfo.type) {
            case BABYLON.PointerEventTypes.POINTERDOWN:
                if (motionController) {
                    pickedMesh.setParent(motionController.rootMesh)
                }
            break;
            case BABYLON.PointerEventTypes.POINTERUP:
            // release the object from the parent
            pickedMesh.setParent(null)
            break;
        }
});

and here is the playground https://playground.babylonjs.com/#9K3MRA#303

10

The POINTERUP doesn’t have the pickedMesh passed to it, so you’ll need to save the “grabbed” mesh somewhere, then reference that to set its parent to null.
Something like this will get you started.

switch (pointerInfo.type) {
case BABYLON.PointerEventTypes.POINTERDOWN:
    if (motionController) {
        motionController.grabbedMesh = pickedMesh
        pickedMesh.setParent(motionController.rootMesh)
    }
    break;
case BABYLON.PointerEventTypes.POINTERUP:
    // release the object from the parent
    motionController.grabbedMesh.setParent(null)
    motionController.grabbedMesh = undefined
    break;
}
1 Like
11

The examples are using pointerInfo.event.pointerId but seems that it was (re)moved do you know what should be used instead?

12

It wasn’t removes, it should still be there. can you show me an example where pointerId is missing?

13

PointerInfo has a IMouseEvent and IMouseEvent has no pointerId. I might be wrong, Typescript is very new to me but my IDE is complaining about it.

14

pining @PolygonalSun - this should probably be a pointer event?

15

Since both WheelEvents and PointerEvents (which contains pointerId) inherit from MouseEvent and are passed through onPointerObservable, the event type was set to IMouseEvent to account either scenario. I was working on trying to get it to type it correctly as IPointerEvent or IWheelEvent as the time of notifying observers but wasn’t able to get it to work properly yet. Right now, for the time being, the easiest way around this issue would be to just cast as an IPointerEvent for access to the pointerId.

1 Like