How to capture multi-touch pointers using native js?

Questions
1

I need to process mouse pointer as well as multitouch mobile touches for custom camera movement.
I started with PC version, so I made a simple script to test it:

if ( "onpointermove" in window ) {
  canvasEl.onpointermove = ( ev ) => { 
    ev.preventDefault(); 
    pointerMove( ev ); 
  }
}
function pointerMove( ev ) {
  console.log( `XY = ${ ev.x }, ${ ev.y }` );
}

It captures coordinates fine. But when I do:

engine.enterPointerlock();

then my pointerMove() function is not called anymore. Why does it work like this? So I can’t continue with multi-touch pointers.

I didn’t found any multi-touch documentation using google or babylonjs website.
How to deal with multi-touch pointers to capture all coordinates for all pointers?
I need to capture all the screen coodinates for all pointers which are pressed and moving at that time.

UPD:
If multi-touch works out of the box, for instance like this:

scene.onPointerObservable.add( ( pointerInfo: BABYLON.PointerInfo ) => {
  if ( pointerInfo.type == BABYLON.PointerEventTypes.POINTERDOWN ) {
    // pointerInfo.event. ***pointerIndex*** ???
  }
} );

then how do I get the index of the pointer? I see this system allows to use multi-touch pointers on mobile and get different coordinates but how to get a pointer index for each pressed pointer?
I’m not talking about pointerInfo.event.inputIndex - it’s just smth like a button type (left, right, middle), but I need to get index of the pointer, or index of the finger if it’s more clear.

UPD2:
I’ve just seen some property called pointerId but for some reason it doesn’t visible for typescript types. Anyway that’s exactly what I’m looking for, just used as:
( pointerInfo.event as any ).pointerId
And it doesn’t work on mobile, I get large nevative value (< 0). On PC works fine, I get: 1.

  • iphone 12, safari, up to date: pointerId = -1680…
  • iphone 6+, safari/firefox: pointerId = 0, but it works only for single touch in Babylonjs and also it doesn’t work for Pointer MOVE event. It’s just never calling by Babylon on this phone. Anyway, I have full multitouch support in native JS on this phone.
  • android 10: pointerId = 0 and increases with every new touch down event, so now I have pointerId = 125.
    My code is:
	scene.onPointerObservable.add( ( pointerInfo: BABYLON.PointerInfo ) => {
		switch ( pointerInfo.type ) {
		  case BABYLON.PointerEventTypes.POINTERDOWN:
			console.log("POINTER DOWN", pointerInfo.event );
			break;
		  case BABYLON.PointerEventTypes.POINTERUP:
			break;
		  case BABYLON.PointerEventTypes.POINTERMOVE:
			console.log("POINTER MOVE");
			break;
} );

Move event never calls on iphone 6+ with this babylonjs system but this phone has full multi-touch support (with touch move events) with native js approach:

canvas.ontouchmove = ( ev ) => { ev.preventDefault(); pointerMove(); }
2

cc @PolygonalSun

3

So with Safari, Babylon.js generally deals with MouseEvent objects, which doesn’t explicitly have the pointerId property but they can be cast to PointerEvent objects (or IPointerEvent objects, like we use with BJS for JS/Native purposes). In general, Safari tends to treat its pointerId values differently than other browsers, notably as very large negative numbers. Futhermore, only Safari iOS 13+ supports PointerEvent and its properties so older Safari iOS instances have a value of 0 assigned for pointerId. I’m guessing that for your iPhone 6+, it’s not running a Safari version that’s greater than 13 so that might be why it’s not triggering for onpointermove.

If you’re trying to get it to work on the older phone, you might have some luck by using the TouchList object from a TouchEvent and keeping track of pointer: TouchList.item() - Web APIs | MDN (mozilla.org)

1 Like