Using onPointerObservable XR Cardboard

(@RaananW This is sort of a follow-on from https://forum.babylonjs.com/t/enableinteractions-equivalent-in-xr/12722/18)

I’m developing an XR game for Cardboard and trying to use onPointerObservable to trigger a pick. However, onPointerObservable doesn’t seem to work in XR mode. Is there a way to enable it or is there some other way to detect a touch in XR?

TIA

Maybe use ray casts and then manually trigger the observable?

Hi Carl,

Cardboard is a strange bird in that it supports touch events to trigger certain actions, but was never expected to use touch for picking, especially since you don’t really know what (or where) exactly you are touching.
Are the events not triggered at all, or is picking not working?
If possible and fits the use case, you can force the gaze mode to allow user interactions with the scene

Probably not the issue, especially since ios doesn’t support WebXR at the moment :blush:

The issue here is the lack of controller support for cardboard, which in turn prevents the pointer selection feature to translate actions to pointer events. I’ll wait for Carl’s answer before I can help further, especially the only regarding whether or not the pointer events are being triggered and not picking or not triggered at all.

Ah, yes. I suspect that “strange bird” doesn’t begin to cover it. I’d forgotten that cardboard used to use a magnet to trigger. Does it still, or does it use touch?
Anyway, the “off-brand” cardboard I’m initially targeting uses touch.

I’ve tried using canvas.addEventListener("touchend", handleEnd, false); which doesn’t work in XR either. So, it looks like it’s a browser support issue and nothing to do with Babylon. Anyone know why?

FWIW touch is not being triggered at all. So, I may use gaze or move onto Gear. I have a Daydream, too. No idea if it’s still supported. The joys of developing on the cutting edge, I suppose. I just wish I was getting paid for all this soul-crushing effort!

I just checked browser support for WebVR and WebXR on caniuse. Oh. Oh my.

As always, thanks for support an feedback.

WebXR is the way to go, don’t trust webvr to work anywhere except firefox (which already has webxr support behind a flag).

I assume the touch events are not triggered because it is actually not being rendered in the browser but in some google VR bridge. There is sadly little we can do about it. Just to try - can you see if the “select” event is triggered?

Yes, WebVR is deprecated, hence the focus on WebXR. I just noticed the disappointing lack of support in most browsers.

I’m not too sure what you mean, but I added a select event listener,
canvas.addEventListener("select", handleSelect, false);
which is ignored.

I guess I’ll go and cry in a corner now…

This guy - https://www.github.com/DougReeder/aframe-button-controls/tree/master/aframe-button-controls.js seems to have added all possible events available - pointers, touch, select, squeeze, gamepad, webvr, WebXR, and somehow got it to work. Maybe find what works for you and integrate it?

I’ll be happy to know which event actually worked, I’ll be happy to integrate it in our pointer selection feature

A-Frame sits atop three.js, so… I’ll have to do some (more) research.

has nothing to do with the framework :slight_smile: he is using native events.

1 Like

As far as I can tell, he’s just using addEventListener to add touch events to the scene element.

Did you mean select as described here (https://developer.mozilla.org/en-US/docs/Web/API/XRSession/selectstart_event)?

After a little searching I found xrSession masquerading as session under xrHelper.baseExperience.sessionManager.session.

And onselect does seem to be working!

So, progress? I guess we’ll find out!

That’s exactly what I meant. The select event is the one I would expect to be triggered when you click.