Implementing custom pinch gesture in AR

Hi all, I’m working on an AR project and I would like to use a screen pinch gesture to scale a single model in the scene.

AFAIK pinch gestures are not built-in to the Web API, I’m using the onPointerObservable to get the pointerdown and pointerup events and reusing this code from the MDN docs:

[Pinch zoom gestures - Web APIs | MDN]

which relies on the comparing the locations of the two pointer events (from each finger) and checking if theyre moving closer of further apart.

The issue is, when logging these events in AR they don’t seem to have a location property (what would be clientX / clientY on a screen), this is always set to 0 no matter where on the screen I press.

Any idea what property of the PointerInfo object to use to make this gesture work?

Many thanks!

Adding @RaananW and @syntheticmagus for the XR part

1 Like

for more info, these pointer observables are being added on the scene. I’m triyng now to add them to the canvas but nothing is registering so far.

Hi there, after a lot of debugging ive found one problem thats stopping me.

To make pinch work, it needs to register 2 “POINTERDOWN” events and compare their locations (by storing in a cache), before a “POINTERUP” event with the same ID is registered.

The problem is having is that it doesnt seem possible to do this, as soon as the second touch event happens it acts as if the first finger has been removed from the screen (on my device at least, Chrome Android)

Heres a playground demonstrating what I mean. When 2 fingers are pressed, the sphere should turn green, but it never does (and console logs back this up):

https://playground.babylonjs.com/#F41V6N#509

I’d really appreciate if anyone could let me know whether:

  • this technique is flawed and theres another way to calculate a pinch gesture
  • web XR is somehow affecting the pointer events

Thanks again :slight_smile:

@PolygonalSun could you have a look ?

1 Like

We are internally detecting it for zooming on arc rotate camera so there is definitely a solution I guess :slight_smile:

for further info, turning AR off here means the double-touch works:

https://playground.babylonjs.com/#F41V6N#510

so its definitely something with the XR… ominous… :cloud_with_lightning:

no worries @RaananW will be back on Monday and can have a look

1 Like