Pointer click detection is wrong after rotating the HTML (via CSS). Click on GUI elements not detected

Hi guys.

Ok, so I have a weird playground here (sorry for neck pain). Basically, I have a situation where I need to rotate the app, so it kinda goes in “landscape mode”. This is done by using CSS and rotating whole html. After rotation, clicks on the GUI elements are not detected. (uncomment the styles in the playground and you will see what I am talking about).

Any way to correct this?

1 Like

Let me open my grimoire and summon @PolygonalSun

1 Like

So, I took a look at this and from what I’m gathering, it’s not so much that the GUI elements aren’t being detected but that where the page thinks the GUI elements are, is vastly different from where they actually are. While I’m not sure on the best approach here, I’d imagine that it’d be a matter of also rotating the UI element to match new layout. If I can think of anything else, I’ll let you know. @DarraghBurke, any chance that you have any ideas for a good approach?

You might need to write your custom input which handles that.

This is a really interesting one, thanks for bringing it to our attention! I haven’t investigated carefully yet, but it looks like we are not applying the CSS transformation to our input system. It’s actually a bit of a philosophical question about whether or not we would want to do this. Effectively, we are assuming that the canvas unrotated with 0,0 being the top left of the screen. But when you rotate it using CSS, this assumption falls apart.

We need to ask ourselves if this is something we should take into account for input and/or GUI. The free camera does not respond to rotation either: if you rotate the canvas 90 degrees, the axes of pointer movement are now reversed from the axes of camera rotation.

If this is something that’s necessary for people’s projects, we may want to build a feature to account for CSS rotations. Whether that should exist for all input or just for GUI I’m not sure yet. One thing we can’t do is automatically apply it, as this may break backwards compatibility; it would need to be a flag, perhaps on the engine or maybe just on the ADT.

I’m going to think on it more and discuss with other Babylon developers. Thanks for bringing this up!

1 Like

I raised this issue with the team but ultimately we do not intend to add any logic relating to CSS transformations. We are going to leave that to the developer’s responsibility. I would say that if you need this behavior, you will probably want to extend Scene and AdvancedDynamicTexture with your own classes, where you overwrite the pointerX/pointerY calculations with your own function that accounts for the rotation.

But there is probably an easier way to achieve what you are looking for. Can you share a little bit about what your goals are?

You can always apply a rotation to the ADT’s rootContainer if you just want to rotate the UI: Simple GUI in fullscreen mode | Babylon.js Playground (babylonjs.com)

Hi. Well basically the goal is to have like “forced landscape mode”. As I understand, browsers are not allowing to force landscape orientation without user interaction (but I’ve been told that some engines, like Cocos, have ability to do this, but I don’t know how). So the idea was to detect portrait mode and via CSS to rotate the whole page to imitate landscape orientation. Rotating just the GUI will not work for me, I suppose.

Is there a way to somehow force engine to render everything rotated by 90 degrees…

One thing that may help is the ScreenOrientation API, particularly the lock() function. It allows you to force the screen orientation to be either landscape or portrait. Unfortunately, while it’s well supported on Android, it’s not supported at all on iOS.

Your best move might just be to show a message to the user recommending they rotate their phone to landscape mode. You can at least detect the current device orientation on iPhone.

I’m afraid this is just not something Babylon.js supports at the moment :frowning:

1 Like

Hello @nogalo just checking in, was your question answered?

Hey. Kinda yes. Didn’t quite get the solution, but I got the information that can help me plan further actions.

1 Like

Hello, I also meet the same problem now, may I ask how you solve it?