Webxr and DOM overlays for VR

For our mobile and desktop scenes we show/hide DOM elements using CSS when interactivity is needed. This works real well. We want to do the same for HMDs.

However, I’m not sure if this is possible in WebXR. Looks like WebXR has DOMOverlay Module - WebXR DOM Overlays Module but according to the Babylon docs ( WebXR Augmented Reality Features | Babylon.js Documentation ) looks like this feature is AR only?

I also see some work done for webxr dialogs like such - WebXR-friendly pop-up dialog box . Does that mean there is no VR support for DOM overlays?

cc @RaananW
You could also use The Babylon GUI | Babylon.js Documentation since it works on both web and XR :slight_smile:

1 Like

when the webxr dom overlay feature was implemented, it was only supported in AR scenarios. There is now a comment in the specs, saying “Implementation choices include a fullscreen overlay on a handheld AR device, or a floating rectangle in space for a VR or AR headset.”. I am not sure, to be honest, which VR device supports DOM overlay. but we can enable it for VR sessions, of course.. We don’t imit WebXR DOM Overlay based on session mode. So - you can use it in VR, if the device supports it.

But I do like @carolhmj’s suggestion to use the GUI. no fullscreen support, but you can attach GUI to a mesh or use the 3D GUI elements

1 Like

For a historical reference, Feature 'dom-overlay' is not supported for mode: immersive-vr says com.oculus.browser (OculusBrowser/31.1.0.14.26.554554223 SamsungBrowser/4.0 Chrome/120.0.6099.144) as it fails to stat the session – if requested to start an immersive-vr session with requiredFeatures: ["dom-overlay"] an up-to-date Meta Quest 2.

If today or in future someone finds a device/browser combination out there that has implemented and enabled support for dom-overlay in immersive-vr session, please update this and all other threads on this so that we are first to find out :slight_smile: Stable or nightly, by default or behind a flag - any support goes.

I am not aware of any as of today, though the W3C spec doesn’t seem to be in a good enough shape for the vendors to implement just yet (in the dom-overlay & immersive-vr aspect specifically).

1 Like

Any Updates on this? I also get this error.

Feature ‘dom-overlay’ is not supported for mode: immersive-vr

Oculus Browser Version 31.3.0.11.57.563651913
Chromium Version 120.0.6099.268
On a Meta Quest 3

1 Like

That’s mostly a browser feature, so I sadly don’t have an update here. I wouldn’t expect DOM Overlay to be supported in VR scenes. But you can now use fullscreen GUI now! if the browser supports layers :-).

I am working on the docs for this new feature

3 Likes

I don’t think the intention is to have it VR compatible:
immersive-web/dom-overlays: A feature incubation repo for layering DOM content on/in WebXR content. Feature lead: Piotr Bialecki (github.com)

Also, in the docs it’s in listed in the AR features:
WebXR Augmented Reality Features | Babylon.js Documentation (babylonjs.com)

Things can change as the vendors see fit. I think GUI on a plane is best suggestion for VR.

1 Like

Looking forward to that, @RaananW , and thank you for the many webxr improments you have landed recently.

Where should we expect the docs, examples for GUI, Layers and other new things enabled by v 6.44 to show up as you write them?

And while I have your attention (with apologies for off-topic in this thread), is there other notable webxr stuff that you plan/hope to land soon-ish?

2 Likes

Where can we look for the documentation/examples of using the recent new GUI features in XR?

1 Like

https://doc.babylonjs.com/features/featuresDeepDive/webXR/WebXRSelectedFeatures/WebXRLayers#fullscreen-2d-gui-support

2 Likes

There are the open XR issues. Some are marked to be in 8.0, some are marked as future, which means they will probably take a little longer, unless they will be requested directly before:

Issues · BabylonJS/Babylon.js (github.com)

1 Like