Make GUI plane only pickable where Controls are placed (click through whitespace)

Questions
1

Hello!

To increase performance I use a single plane for multiple GUI controls.
On this GUI plane there is some whitespace where no controls are placed.
When I want to click a GUI element that is behind this bigger GUI, the events are blocked by the whitespace of the GUI.

This is the GUI I want to create:

whiteboard_gui
whiteboard_gui1953×1154 256 KB

The red box is the plane where I placed the GUI on:

whiteboard_gui_plane
whiteboard_gui_plane1953×1154 224 KB

This playground show an example. You cannot click the back buttons when you click between the front buttons:
https://playground.babylonjs.com/#SWB9QN#1

Is there a solution for that or should I create a plane for every single UI element?

Thank you!

2

Pinging @msDestiny14

3

Is it possible to archive a similar behavior to the FullscreenUI? When using the fullscreen UI only the Controls are pickable and not the whole screen.

1 Like
4

Let me take a look. :slight_smile:

5

Hi there! I agree with @marlon360 a fullscreen UI would help because then you can just layout controls onto of controls. However that may not get the depth you want.

I tried a bunch of different solutions but didn’t yield the result we wanted. I tried:
firstPlane.isPickable = false;
button.isHitTestVisible = true;

The problem is we want the mesh to not be active but the Gui to still be clickable.

6

This is a very naive approach, but you could play with mesh picking in that case - GUI blocking other GUI | Babylon.js Playground

Those events are anyhow being triggered, so there is no performance hit

2 Likes
7

You can sample from the ADT texture and discard hits on transparent pixels by adding this to the end of your intersects method on the related mesh (pi: PickingInfo, advancedTexture: AdvancedDynamicTexture)

      if (pi.hit && pi.getTextureCoordinates()) {
        const { x, y } = pi.getTextureCoordinates()!;
        const { width, height } = advancedTexture.getSize();
        const sx = Math.round(x * width);
        const sy = Math.round(height - y * height);
        const point = advancedTexture.getContext().getImageData(sx, sy, 1, 1);
        // if the texture pixel under the picked point is transparent, ignore the hit
        if (point.data[3] < 0.5) {
          pi.hit = false;
          pi.pickedPoint = null;
          pi.pickedMesh = null;
        }
      }
2 Likes
8

Hey, I am facing the same Problem of overlaping planes with ADTs. The solution from @FrameVRWill works well but we have multiple planes and the performance might suffer from the ray casts. Is there another more reliant way of making sure we hit a gui control and not an empty spot on the plane?

9

I wouldn’t worry too much about the perf impact of ray casts. Every time you pick in a UI it is basically doing this already.

1 Like
10

Yes, but then If I wanted to be a bit ‘picky’ (sorry), I still don’t completely understand why the ADT acts as a pointerBlocker, despite for having no controls in it and despite of setting any control ‘isPointerBlocker = false’ and ‘detectPointerOnOpaqueOnly = true’ or ‘isHitTest’ or for whatever you do. Obviously, nothing of all that works. Only the manual picking method works.

That’s what it is about, so…you know, it won’t be me coding it :dizzy_face: :sweat_smile: but admitting there would be a way to implement this (for lazy people like me :joy:), well, I suppose it would be sort of welcomed (at least by me :grinning:) cc @carolhmj What is your say?

11

This happens because in the end, the ADT is still a plane on the scene, and any regular plane would block rays from going through it, regardless of its material properties (because we test only on geometry and don’t really look at the material of whatever we’re picking).

1 Like
12

Thanks for your reply and… sorry… used the wrong words. My bad english again :sweat_smile: I totally get this part and, to be honest, never faced this problem because my GUIs for mesh always match the size of the plane. But, there might be use cases where it wouldn’t? Apparently, there are, according to this PG above… although, I’m not all too sure why it has to be like this!?

Anyways, I guess I just wanted to say: “In a perfect GUI world, wouldn’t it be nice that the ADT where not having any controls and having a property of like “detectPointerOnOpaqueOnly = true”, would actually allow to pick through its host mesh?”