What are the plans regarding WebXR Layers API?

I am curious as to plans, if any, regarding WebXR Layers API support in Babylon.js.

This issue covered what I would expect, but was closed:

However, only the XRProjectionLayer was implemented in a related issue:

I can see how XRProjectionLayer is important for being able to do fullscreen UI using layers. However, the other layers seem like they could be useful too. I believe layers would improve quality and performance for things like 2D text panels and video/photo textures. Maybe that’s somewhat outside of the domain of babylon.js, but perhaps at least something could be exposed for manually creating the other types of layers?

I am only curious as to plans and potential timing. I considered trying to hack things together just for testing, but I don’t see any easy way to do so with the current class structures and types.

Related links (fwiw):
BJS

Reference

Related Source

Antecdotal

cc @RaananW (please bear with us as Raanan is working on finalizing the build system for 5.0 :))

1 Like

I will answer rather broadly, but will update all related issues very soon.

We are planning to support all types of layers. Each (position-oriented) layer will have a corresponding babylon transform node and will be used just like any other mesh with its own render target texture as material. You will be able to display and image, a video, or an entire set of meshes on any of those layers. Integrating all types of layers will allow us to also have a fullscreen element to display fullscreen UI and all effects that require fullscreen layer (like so many types of post processes or the flare lens).
Open issues I have with this plan is the lack of cross-platform support for that. What I mean is that we were always trying to maintain desktop/XR/mobile support. One codebases for all. But there are no layers for the desktop. They only work in XR. This is something we might just have to accept. especially since other XR features don’t exist on th4 desktop as well, and there is no way around that as well.

I hope this is detailed enough and provides enough understanding to our future implementation!

2 Likes

Makes sense.

That is something I’m still having trouble fully understanding about this spec. Specifically, why it was made to be WebXR specific. Seems like it could have been written generically for any WebGL platform using a compositor. Maybe the only real world case for that is WebXR though.

Having to support more and more WebXR specific features like this in Babylon.js core feels …complicated.

I guess you have already seen the webxr-layers-polyfill. Based on that, I assume these layers/nodes could be made to work cross-platform. It’s just that the performance/quality differences would only be seen when in a real WebXR session. Implementing that polyfill in Babylon seems like a lot of extra work that should seemingly be outside scope of Babylon.js though.

Reiterating my thoughts, it feels …backwards. Seems to me like there could/should have been a “WebGLLayer” (with corresponding WebGLProjectionLayer, WebGLQuadLayer, etc). On desktop/mobile, those layers would have been minor conveniences with no appreciable benefits except ability to use a common API. But using same API , when in WebXR (or any compositor type environment), there would be real runtime benefits.

Definitely! Thank you. I’ll look forward to it.

1 Like

Good question! To which I have no answer… Maybe because, as you say, it might bring some convenience but no performance benefits for desktop apps (unless the browser implements them). Also, the XR layers are very WebGL specific, and will need some work to provide them in WebGPU, for example.

But, no answer will really provide a good explanation.

Oh, yes :slight_smile: The story of my XR life

The more I think about it, the more I think it will make sense to implement those to non-XR devices as well, and have them work for XR natively. BUT this will require some changes in our core library. And, in fact, force us to implement all of those layers ourselves :slight_smile: Seeing that the specs is still in alpha, this might not be the best decision for the time being. But this is still too early to talk about the specifics, as I have not yet started to implement that.

Beyond that WebXR is W3C & WebGL is Khronos, think there might not been a lot of support folding stuff into WebGL.

Not all browsers are implemented on a XR device & remember just how long it took to get WebGL 2.0 widely supported.