How to interact with Babylon.GUI elements in immersive-ar?

Bugs
1

I have almost same problem as mentioned by this user , Cant interact with GUI.Button.CreateSimpleButton in WebXr immersive-ar mode. · Issue #8879 · BabylonJS/Babylon.js · GitHub

I am creating a simple 3d model viewer which should work in immersive-ar, but the problem is that the Babylon.GUI elements are not interactable during ar-session, I don’t know if its a bug or some mistake from my side, so here is my playground example = https://playground.babylonjs.com/#QA5WSJ#239

the button is working fine in browser,

2

I am sure @RaananW can help with this :slight_smile:

4

I didn’t dive into your implementation (as I am mobile right now), but this example has a working UI, you might be able to debug yours based on that - https://playground.babylonjs.com/#KMNNUS#27

5

can you please tell me if you are using it on android or iphone ?, I have tried the playground example you provided on my android phone, and its not working for me, whenever I click the button, the click event is not responding.

6

I will look into that. if it is not working it’s a regression that we need to solve. thanks for notifying us

7

I have found that its only not working when we are using fullscreenUI in WEBXR-immersive AR
when using it like
:-
BABYLON.GUI.AdvancedDynamicTexture.CreateFullscreenUI(
“myUI”
)

I tried with mesh panel like this

BABYLON.GUI.AdvancedDynamicTexture.CreateForMesh(
myPlane,
1024,
1024
);

and the buttons are responding normally , I think the problem is not with GUI , but only with AdvancedDynamicTexture.CreateFullscreenUI

8

The issue was the pointer up event not triggered correctly.

This PR is fixing it. Android AR and Oculus VR tested.

9

thank you for looking into this matter :slight_smile:

11

this problem is there once again

12

@PolygonalSun might be able to have a look, as @RaananW will only be back in a couple weeks after the break

14

i’ll look into that this week. Are you saing you can’t interact with the GUI in AR mode now?

15

Problem found and merged, will be up in the next nightly

17

thank you @RaananW , it has been solved, and its working fine