Pick through world space GUI plane

Questions
1

Short version of the question is: how do I pick through the transparent parts of a GUI Plane that is in world space?

Refer to this PG: https://playground.babylonjs.com/#FXFTTV#3

Starting on line 32, I am creating a Plane and an Advanced Dynamic Texture so that I can render a UI that lives in world space. You will note that in my example there is a lot of transparent space between the buttons. What I need to do is pick up UI events (hover, click) on the buttons but allow pointer events to “pass through” the transparent spots on my UI plane.

In the PG example you will note that you cannot pick on the sphere (because it is behind the UI Plane).

If I set the ui plane to isPickable=false then I no longer pick up clicks on the buttons. Setting bg.isPointerBlocker = false does not seem to do what I need either.

Ideas?

2

Yeah this is a interesting use case:) what you can do is on scene.onPointerDown, if you detect that your plane was picked, you can locally turn isPickable false and run a second scene.pick to see if you are catching something else

3

Thanks for the response. I had considered this but I am not sure how to distinguish between clicks on a button (which should block the ray) and just clicks on the plane. Perhaps I need to be checking the alpha of the advanced texture?

4

Alright, I have a bit of a workaround. First, you have to take over picking in your scene. Then what I did was PrePick on the uiPlane by just trying to intersect the pick ray with the UI Plane.

const pick = ray.intersectsMesh(this._uiPlane as any);

If a hit was detected I then read the alpha value of the hit pixel. If the alpha value is less than 0.5 I set uiPlane.isPickable = false. If the ray did not intersect the plane or if the alpha value was greater than 0.5 then I set the plane to isPickable=true

Finally, I did a scene.PickWithRay(ray) and voila, it seems to be working.

Side note, getting the alpha was kind of trickiest part of this for me to figure out. If anyone is interested, here is a snip of code:

const uv = pick.getTextureCoordinates();
const px = Math.floor(this._uiTexture.getSize().width * uv.x);
const py = Math.floor(this._uiTexture.getSize().height * uv.y);
const alpha = this._uiTexture.getContext().getImageData(px, py, 1, 1).data[3];

2 Likes