Hitting `Max number of touches exceeded. Ignoring touches in excess of 5` when using a pen on a tablet (Samsung + S Pen)

Bugs
1

Hey there :wave:!

Recently, I used a Samsung tablet with its S Pen on the arc rotate camera playground example.

I noticed that when using the S Pen to for e.g. rotate the camera after the 4th interaction the camera always freezes (no interactions possible anymore). I.e. trying to rotate the camera the 5th time doesn’t work anymore.

After connecting the tablet to my laptop and using chrome://inspect/#devices to connect the Chrome DevTools I noticed that Babylon.js prints

logger.ts:107 BJS - [09:10:45]: Max number of touches exceeded. Ignoring touches in excess of 5

This message seems to originate from webDeviceInputSystem.ts either in this._pointerDownEvent or this._pointerMoveEvent.

This problem happens only when using the S Pen (not sure if it happens also on any other tablet with a “pen”). Furthermore, one can “clear” the activeTouchIds by e.g. clicking on a button (that’s why I’m clicking on the hamburger menu) and then the next 4 interactions work again until the 5th.

This problem doesn’t happen when using my finger:

When starting to set a breakpoint in this._pointerDownEvent I can observe that the array _activeTouchIds gets populated multiple times with the same pointerId (in the example 12).
image

:information_source: Interestingly, I was able to reproduce this behavior without a tablet and only using Chrome on my laptop by putting a breakpoint in this._pointerDownEvent. Of course, now there are only 2 instead of 5 slots, but the behavior seems to be the same/similar.
:exclamation: Without a breakpoint the issue doesn’t occur.

TL;DR:
Somehow it seems that the activeTouchId of a pen isn’t correctly “released” causing the available slots to be filled up until hitting the maximum.
The fact that this also happens without a tablet + pen but with a set breakpoint gives the impression that it could maybe be a timing issue which only gets evident when using a pen :person_shrugging: :thinking:. But this is just a guess.

Is anyone able to reproduce this issue and might have an idea what the problem/solution is?

Thanks a lot!

:information_source: Update

I just realized that a pointermove event gets triggered as soon as the pen is hovering close enough over the screen (but doesn’t touch it yet).

image

Could it be, that the same pointerId gets “retroactively connected” in this._pointerMoveEvent after the pen was lifted (i.e. pointerup was triggered).

I.e. maybe we can’t always “retroactively connect” a pointerId on pointermove event, since a pointermove gets triggered by the pen already when hovering over the screen. This is probably a different behavior then when using a finger, since a finger needs to be always touching the screen in order to trigger a pointermove event.
I.e. do we need to add an additional check in webDeviceInputSystem.ts#L428-L447? E.g. check if a pointerdown was first triggered for the given pointerId before we “retroactively connect” it :thinking: ?

2

This might be an issue for @amoebachant :slight_smile:

3

In my Angular app I seem to be able to work around it by keeping track whether the pen is touching the screen or not (hovering) and if not I stop the pointermove event from propagating.

// Host element is the canvas

private isPenUp = true;
@HostListener('pointerdown', ['$event'])
onPointerDown(event: PointerEvent) {
  if (event.pointerType === 'pen') {
    this.isPenUp = false;
  }
}
@HostListener('pointerup', ['$event'])
onPointerUp(event: PointerEvent) {
  if (event.pointerType === 'pen') {
    this.isPenUp = true;
  }
}
@HostListener('pointermove', ['$event'])
onPointerMove(event: PointerEvent) {
  if (event.pointerType === 'pen' && this.isPenUp) {
    event.stopImmediatePropagation(); // 👀❗
  }
}
4

I believe this is very much related, yes.
What (I believe without debugging too much) is happening, is that the pointer move populates the array correctly, and let’s you run the pointer move events. On the first pointer down everything is fine - but the pointId that is already in the array is not being reused. Then on pointerup the device slot is being cleared and then instantly repopulated in the pointermove (that happens while you hover after the pointer up).

onPointerDown is not checking if the pointerId is already present, which is the root of the issue. Also, the fact that we assume that a device is disconnected onPointerUp might be wrong, but your issue will be solved if we reused the deviceSlot already populated by the pointer move.

5

Thanks for looping me in, I’m going to look into your hypothesis @RaananW (pointerMove adds to _activeTouchIds, then pointerDown does, then pointerUp only cleans up one of them).

It didn’t repro on an iPad with an Apple Pencil (I always got pointerDown before pointerMove), but I’ll try on a Samsung tablet + pen shortly.

2 Likes
6

Hi @Philippe, thanks for reporting this bug! I was able to repro it on a Samsung tablet with a pen just like you reported, and confirmed it was indeed due to the _activeTouchIds slot not being reused by pointerDown if pointerMove was raised first, which happens with the pen on those devices. The fix was merged today.

Thanks!

5 Likes
7

This is awesome @amoebachant!

Thanks a lot!

1 Like