WebXR layers feature broken on Quest 2 with skybox and environment

Bugs
1

Hi,
Recently I tried to enable the new WebXr layers feature so that the Quest 2 renders both eyes in a single pass.
It is most definitely a performance improvement, however I did see 2 issues when enabling this feature with a skybox and environment:

  1. Controllers seem oversized
  2. pbr reflections are stretched to a single point

A playground to show these issues is here:

Thanks

2

cc @RaananW

3

thanks for letting us know! We’ll look into it

1 Like
4

Hello @RaananW,
Since pull request [XR] Fix updateTransformMatrix for multiview by rgerd · Pull Request #11934 · BabylonJS/Babylon.js · GitHub was merged, this immediate issue looks much better. Really appreciate the fix :smiley:

I have found some other related issues that occur when enabling multiview:

  1. Anti-aliasing is disabled on Quest 2. This can be seen by comparing the following playgrounds:
  2. When exiting XR on the Quest 2 using the playground with Multiview enabled (https://playground.babylonjs.com/#9K3MRA#741) the view rendered in the 2d Oculus browser has reflections that are broken in the same way as they used to be in XR (my original post)
  3. scene.clearColor is ignored and the scene is always cleared with a black colour, as seen in this playground: https://playground.babylonjs.com/#9K3MRA#789

Hope the above is clear, thanks again

2 Likes
5

I’m also experimenting with WebXR layers (and multiview). Regarding the anti-aliasing issue, I came across this information if it helps:

6

Thanks for reporting!
Apart from the antialiasing (which is a known issue with multiview in general, solved by the OCULUS muttiview extension), the other 2 are bugs. Do you mind opening a github issue to track them? ping me there, i’ll make sure to tag and assign it.

7

Hi @RaananW,
Thanks for getting back to me, I’ll create the 2 bugs first tomorrow :+1:

However I do believe there is an issue with the implementation of the OCULUS muttiview extension as I am testing on A Quest 2 and see aliasing.

After reading through the code I see that the method bindMultiviewFramebuffer of file engine.multiview.ts calls framebufferTextureMultisampleMultiviewOVR, once with the colour and once with the stencil. For both of these calls multiviewTexture.samples is passed in as the samples parameter, however this value is 0. My understanding is that it would need to be a value of 2, 3 or 4 for anti-aliasing to work.

As a quick test in my own project I set the Engine.prototype.bindMultiviewFramebuffer myself and hardcoded the samples to be 4 and anti-aliasing started working on My Quest 2. I did have some weird issues, that may be related to rendering layers (layer 0 wasn’t rendered until I added another item to my scene) but it was a quick test as it is quite late where I am right now.

I shall revisit it tomorrow, but I would appreciate your thoughts on this.

Thanks

1 Like
8

Hi @RaananW,
The 2 bugs have been created as requested:

I also looked further into the aliasing issue on the Quest 2. Here is a playground where I force the samples parameter to be 4 when calling into framebufferTextureMultisampleMultiviewOVR: https://playground.babylonjs.com/#9K3MRA#794

Note that nothing on renderingGroupId=0 actually renders, so all items except the skybox are moved to renderingGroupId=1, which ensures they are rendered.
All of the rendered items are correctly anti-aliased on the Quest 2. Am I missing something in the configuration which enables anti-aliasing correctly on the Quest 2 when multi-view is enabled? Or is this an example of a real issue, and that samples should in fact be set to the value held within the Engine.getCaps().maxSamples ?

If I am correct about the samples, then why would rendering group 0 be ignored? (maybe it is cleared after being rendered?)

Really interested to hear your thoughts.
Thanks.

2 Likes
9

That’s an interesting catch. The number of samples is set on the class MultiviewRenderTarget, which is the RTT used when enabling multiview. When initializing it, is uses this.samples = this._getEngine()!.getCaps().maxSamples || this.samples; to set the number of samples (defaults to 1). So it should work as expected! keyword is “should”. I’ll have to debug this and see what’s going on there. Let me assign myself and get back to you on this one. I am sadly a bit swamped at the moment, but I promise to take the time to check that.

3 Likes
11

Hi @RaananW,
I was just wondering if you have had chance to look into the samples issues when using the Oculus Multiview extension? It seems that the number of samples is still not being set correctly to me.

I can confirm that when I over-ride the Engine.bindMultiviewFramebuffer method to force the number of samples to 4,I no longer get issues with layers: https://playground.babylonjs.com/#9K3MRA#808

However, by default anti-aliasing is not working on the Quest2 as seen here: https://playground.babylonjs.com/#9K3MRA#741

I realise you are all crazy busy and we all appreciate the work you all do.
Thanks.

2 Likes
12

Sorry, no I didn’t… But I promise to find time this week. It should actually be a very straightforward fix.

Very much appreciated! :slight_smile:

13

Took way longer than it should have, sorry again

This PR solves it - [XR] assign the number of textures correctly by RaananW · Pull Request #12340 · BabylonJS/Babylon.js (github.com) . Working wonderfully in the oculus browser.

After it is merged it will be in the next version released (either 5.0.4 or 5.1.0, depending on the release schedule).

3 Likes
14

Thanks @RaananW :+1:

1 Like