How to let user use system keyboard in VR mode?

Questions
,
1

For example, in FrameVR’s VR mode, it provides a keyboard made with GUI.
Is there a possible way to use the keyboard from system or browser in my Babylon project’s VR mode?

1 Like
2

Hi @gunpowderfans , Gabe from Frame here! Do you mean being able to use your main computer keyboard with your hands while I’m VR mode?

1 Like
3

Yes actually. I’m wondering if it’s possible

4

AFAIK it shouldn’t be a problem using keyboard events while in an XR session. it should work the same, as this is a browser API.

Of course, the keyboard input should not be blocked by the VR environment (like windows MR does from time to time).

This works - WebXR basic example with teleportation | Babylon.js Playground (babylonjs.com) if you stay in the browser context (while vr is running).

5

… what about the system keyboard of the VR goggles, which (e.g. Pico 4) do not have a hardware keyboard. Can one somehow trigger to use such a system keyboard for some user input?

6

isSystemKeyboardSupported was added to the WebXR specs a few months ago. It should allow you to trigger the system keyboard in XR, if the browser supports it.

This is the tutorial on how to use it in the Quest browser (which does support it) - https://developer.oculus.com/documentation/web/webxr-keyboard/ . This is not (yet) integrated in Babylon

7

I’ve been using this a bit and I have a strange problem that…for whatever reason, occasionally my controllers will stop working. I’m still researching, but wondering:
“Is anyone else using this right now?” couple of strange things I noticed:

  1. the “babylon hands” freeze when you open the keyboard. It mentions in the article that it puts the xr system into a blurred state, but I’ve tried to detect this and can’t seem to figure it out.
  2. the “frozen controller” problem only happens once and a while…I can definitely make it happen if I click outside the keyboard, but even just using the keyboard normally occasionally makes it happen.
    I’ll post an example here after I have some time to build a simple example.
8

Here’s an example of what I mean. you can use controller x button to open the ‘native’ vr keyboard. It “mostly” works, but occasionally (looks like maybe if ray is intersecting something else when I hit it) it will freeze things up, with no apparent error/exception.

9

Let me add @RaananW to the issue so he can have a look after his vacations.

11

I will look into that during this or next week. I wanted to add the virtual keyboard as a feature (mainly because of all the hiccups you are experiencing). Will work on that.