WebVR "grab" playground example doesn't work on Oculus Quest

Questions
#1

Hi there, I’m trying to figure out how to virtually “grab” and “throw” an object in VR.

There is this guide at webvr_helper#grab: Use the WebVR experience helper - Babylon.js Documentation

But that playground example (Babylon.js Playground) doesn’t work for me on Oculus Quest. The grabbing gesture doesn’t pick up the cubes.

I figured it was because the example is using the older webVR api and I need to update the playground example to WebXR api, however I’m having trouble finding an XR equivalent for the onNewMeshSelected observable, where can I find that?:

VRHelper.onNewMeshSelected.add(function(mesh) {
  selectedMesh = mesh;
});

VRHelper.onSelectedMeshUnselected.add(function() {
  selectedMesh = null;
});

Or is there a different suggested technique for getting an array of meshes (or mesh names) that my controller hand is near or intersecting?

#2

Might be a question for our WebXR master @RaananW

#3

Hi @owen,

WebXR in babylon works very similar to the way a mouse click would work if you use the default xr helper (and its pointer-selection feature) - pointer events are emulated when using the main component of your controller (the trigger button). scene.onPointerDown will work the same on desktop and in VR, and so will pointerUp and pointerMove.

#4

I didn’t know about onPointerDown, thanks. I’m trying to play with it now but I’m not sure what I’m doing wrong: I tried assigning scene.onPointerDown to a function like so, to see what it returns:

 scene.onPointerDown = (pointEvent, pickInfo, pointerEventTypes) => {
          //added some logging here
        }

I can get the callback to log some output when the trigger button is held down. However, after I use teleport once I no longer get log output despite squeezing the trigger. Am I setting this up wrong? Did the callback get removed?

Second question, I think the main trigger button is the wrong thing I want to observe for a “grab” gesture. Shouldn’t I want to observe the grip squeeze button (under middle finger) instead? And determine what eligible mesh is closest to the controller? For example to stack a small set of blocks on a table I just want to grip and let go of any block without first “picking” any block with a trigger button.

Are there any XR examples / playground that work for Oculus quest that have solved this stack blocks/throw blocks feature?

#5

You can read all about controller interaction here - WebXR Input and Controller support - Babylon.js Documentation

Try using our observers system instead of using direct function definitions - scene.onPointerObservable is your friend, as described here - Interact with a Scene - Babylon.js Documentation . This works only with the trigger. the other parts of the controller will need to be dealt with using webxr obsvables (the link i sent before). The physcis playground in the examples (https://playground.babylonjs.com/#B922X8#19) shows the way to deal with controller interaction (specifically squeeze) from around line 180.

I hope this helps! if anything is not clear, let us know

#6

Thank you! I’ll study your playground example. Great stuff btw!

1 Like