Tracking webxr pointer during move events

Questions
1

I am trying implement an operation which moves (drags) an object in VR (XR) mode.
The trouble is that pointerInfo.pickInfo does not seem to be updated for move operations. Does anyone know a solution?
Does one need to trigger simulated click-events during move or is there a way to cast the 3D VR-pointer into a ray so that one can use the scene.pickWithRay method?

2

I tried this code

var get_controller_coords = function(scene, motionController) {
    if (motionController != null) {
        var ray = new BABYLON.Ray(new BABYLON.Vector3(), new BABYLON.Vector3());
        motionController.getWorldPointerRayToRef(ray, true);
        const hit = scene.pickWithRay(ray);
        return hit.pickedPoint;
    }
    else
        return scene.pick(scene.pointerX, scene.pointerY).pickedPoint;
}

But in a VR environment there seems to be a problem claiming getWorldPointerRayToRef is not a function. I looked through other examples with VR controllers, but could not find a single working one that is using this function successfully. Maybe this is an error in the current babylon system?

Anyway the work-around, which most seem to use is to attach the object to move to the controller mesh as a parent. This works, but if one wants to constrain the movement to certain dimensions, the only solution I found is to detach the object, enforce the constraints and reattach in to every move event.
For some reason the system seems to continuously fire movement events in VR mode. Is this a bug or a feature?

3

cc @RaananW

4

any pointer observer added to the onPointerObservable of the scene will get the correct pointerInfo together with the right (xr) pointer event. What you are doing is reproducing the already-existing pointer code. That is, if you are using the pointer simulation feature, that is turned on per default if you use the XR experience helper.
Have you tried using the onPointerObservable?

1 Like
5

Thanks a lot. Indeed I finally found the PointerObservables. They are great.
However, the problem is that I would like to have the look-and-feel of 6DoF manipulation.
See Babylon.js Playground
but at the same time also constrain the movement (e.g to the ground-plane or sometimes even a single axis). I tried the magnet constraints being active at the same time but this did not give useful results. I also tried constraining by my own code during the drag operation callback, but this yields yittering, since the drag-code somwhere maintains the pointer position and keeps resetting the position to it. The contraints in the pointer-based manipulators are great, but pointer-based movements are too sensitive, which is why I would like the 6DoF method, but contrained.
I thought about using a container-mesh or a little ball which is dragged without constraints and the object following with constraints, but the trouble is that this would need quite some messing around with the system to initialize the dragging, which is normally done conveniently by itself using the mesh to drag.
Any ideas?

6

I guess, preventing the mesh to follow the drag and then retrieving the pointer coordinates via the PointerID and then placing the mesh with the appropriate constraints may be the way forward.

7

I found a solution. See a minimal working code example here:

This is implemented by creating a clone(), dragging the invisible orginal but displaying the clone with constraints and replacing the coordinates of the original at the end of the drag with the clone coordinates. @RaananW and @carolhmj, thanks for the help!

1 Like
8

Great! glad you found a solution :slight_smile:

I improved the playground slightly - avoid GC by using copyFrom instead of assigning the vector, and use the WebXR experience helper, which avoids using the deprecated WebVR helper.

9

Thanks for your improvements, which are under:

Yet, it might be nice to eventually allow contrained dragging with a simpler interface? I now faced this problem multiple times and always have to go for a home-made solution.

10

AFAIK the gizmos (along with the drag gizmo) is supported in WebXR. is there a problem with the gizmos?

11

Not quite sure which gizmos you are referring to.
Maybe I missed something?

12

Gizmos :slight_smile:

Gizmos | Babylon.js Documentation (babylonjs.com)

1 Like