Image class has poor performance when transparent parts of image should be ignored (detectPointerOnOpaqueOnly)

This problem is in Babylon 4.1.0.

Let’s take any example with detectPointerOnOpaqueOnly: https://www.babylonjs-playground.com/#XCPP9Y#1510

The problem is in the Image.contains method. It calls every time when a mouse moves, and every time this method creates a copy of the image data:

This causes low fps when a mouse hovers over such images (GC/etc). I would suggest using a cache, something like the following code:

Seems good to me, let see what @Deltakosh thinks about this.

I think it may be a little more complicated, however, as the cache may have to be invalidated when the picture is updated / the width/height are not the same than when the cache was created.

Maybe you are right, it should be more sophisticated. I don’t change pictures at runtime, so it works for me.
Without that I get 20 FPS on Oculus Go :frowning:

I agree let’s improve that!

Issue created:

PR on its way:

1 Like

Merged!

1 Like

@Deltakosh @Evgeni_Popov I think there is a problem with this patch.

In some situations, there may be the following scenario: Control.contains method may be called BEFORE the image is drawn on canvas. In this way, context.getImageData(0, 0, width, height).data returns an array with zeros only. As a result, events like onPointerEnterObservable will not work anymore.

This situation may be reproduced if you are moving a cursor above the place where an image will be shown while the web page is loading. I can record a video with this bug if you need.

Thanks for the report, I have corrected that in this PR:

I simply reset the cache when the picture has been loaded.

Also, my previous fix was bugged, in effect the cache was not used…

@Evgeni_Popov The problem is in the following case:

  1. Image._onImageLoaded calls. But the image has not been drawn on the canvas yet.
  2. Control.contains calls. The cache fills with zeros from the canvas.
  3. Image draws on the canvas.
  4. Image class doesn’t update the cache any more because there are no reasons to do that. So, events don’t work.

Are you using nine patch images?

If not, I don’t see how it can fail because _workingCanvas is initialized in _prepareWorkingCanvasForOpaqueDetection which is called as part of the draw function.

So, as soon as _workingCanvas is initialized it is updated with the image data without context switches, so it’s not possible to process a contains in-between. And before draw is called, _workingCanvas is uninitialized, meaning that:

        if (!this._detectPointerOnOpaqueOnly || !this._workingCanvas) {
            return true;
        }

will return early in the contains function, meaning we won’t update the cache.

So, in your stream of events, it should normally be:

  1. Image._onImageLoaded calls. But the image has not been drawn on the canvas yet.
  2. Control.contains calls. The cache is not filled with zeros from the canvas because _workingCanvas is unitialized and so contains returns early
  3. Image draws on the canvas. _workingCanvas is now initialized as part of the drawing function
  4. contains is called : Image class does update the cache because _workingCanvas is initialized and the cache is currently empty
1 Like