Interacting with GUI elements: with keyboard and gamepad controller (default navigation input UX suggestions)

Hello all,

I am working with the BABYLON.GUI
I have some questions on default User Experience improvement for navigating and working with the standard GUI controls.
Some questions are maybe already solvable out of the box, others may require a standard feature request so anyone using the BABYLON.GUI may be able to benefit.

Note that the user would have a Keyboard, or Gamepad (with buttons and axis/sticks), or Mouse (with scroll wheel) connected, or any combination of that, and would expect any of them to work with GUI ‘with default actions’ (the experience user has with other non-babylon GUIs e.g. their laptop or mobile Apps).

Control focus & navigation

  • How can I with coding set focus on Buttons, Checkboxes, Radiogroups, Scrollbars so that they would process user input? (it seems only InputText / InputPassword can have focus currently)
  • How can I have the user ‘Tab’ through the focussable controls (perhaps also Shift+Tab for backward navigation) and define the order for Tabbing?

Default input handlers

  • How can I set a control (preferably a Button) as the ‘default confirm’ button, even when not having focus but being visible? (e.g. generic responds to ‘Enter’ key or the ‘Select’/‘A’ button on gamepad)
  • How can I set a control (preferably a Button) as the ‘default cancel’ button, even when not having focus but being visible? (e.g. generic responds to ‘Esc’ key or the ‘Start’/‘B’ button on gamepad)
  • Same question but then for having any shortcut key / mnemonic (e.g. keyboard ‘S’ for Save button, keyboard ‘O’ for Open button, but perhaps also ‘X’ gamepad key)
  • Command sliders and scrollbars with input from Keyboard (Up/Down key) Mousewheel and Gamepad buttons/axis (e.g. First Thumbstick position)?

My aim would be not to create code hacks but to have the default GUI functions triggered by the input given. So the ‘default confirm’ button would initiate the same call as if the user Mouse clicked that Button etc.

Although I am now looking for 2D GUI, perhaps it would be of interest to have this UX available for 3D GUI as well.

I did walk through the docs and searched some of the API and GitHub project, but couldn’t find a clear answer thus far so that is why I am asking for your help :slight_smile:

Q

I do not think it is available out of the box but it might be a great addition to the framework :slight_smile: Would you be willing to contribute ?

Ok, I’ll start with making some Javascript template adjustments (extensions to BABYLON prototypes, e.g. adding a property to a prototype) within my game for these features. That is for me the most rapid way to iterate and test a solution.

When I got some parts working in my game, then these parts might be converted easily to Typescript code specific additions to the Git project.

If there are others also looking for these kind of features please show your interest by liking the post.

2 Likes

I noticed IFocusableControl in the new 5.0.0 alpha build, which is so far implemented for FocusableButton, InputPassword and InputText.

When the following GUI controls also get an IFocusableControl update, that would be great:

That would make it a lot easier to construct a generic way to navigate through controls with mouse, keyboard, or even gamepads (gamepad commands can be easily converted to simulated keyboard presses).

1 Like