NativeEngine render target texture not working with MSAA

Anupam_Das · July 25, 2023, 12:13am

working on it. stay tuned and thanks for the support.

Anupam_Das · July 25, 2023, 3:29pm

@bghgary pretty sure this is somewhat unrelated but we are getting an Unsupported texture format or type: format 6, type 0 . Im not sure what format 6 and type 0 is. Could this be related to:

which is that 16 bit textures are not supported by Native? (we will try converting to 8 bit)

bghgary · July 25, 2023, 3:47pm

Right, this is what I mentioned earlier. If this is what you are seeing, I can try to fix it, but it has nothing to do with MSAA. The post process is trying to create a texture with a format that isn’t handled yet.

Anupam_Das · July 25, 2023, 4:26pm

well… we actually found some textures were using in assets that are 16bpp so we are converting those to 8 (those textures should be 8bpp anyway so that was our bad) and we will run our test again. I have a feeling that will get rid of that error. I dont think we have any PP stuff using 16bpp textures but regardless it would be good to support 16bit images long term. I dont want you to do wasted work so let me run this test with our converted assets and hopefully we wont get that error and can get back to debugging AA

bghgary · July 25, 2023, 5:44pm

Okay, sounds good. Just to add more information, format 6 and type 0 is TEXTUREFORMAT_RED and TEXTURETYPE_UNSIGNED_BYTE which I believe is coming from the depth of field in the post process.

bghgary · July 25, 2023, 5:46pm

These are the formats we currently support:

github.com

BabylonJS/Babylon.js/blob/master/packages/dev/core/src/Engines/nativeEngine.ts#L2638-L2650


      
          /* eslint-disable @typescript-eslint/naming-convention */

          import type { Nullable, IndicesArray, DataArray } from "../types";

          import { Engine } from "../Engines/engine";

          import type { VertexBuffer } from "../Buffers/buffer";

          import { InternalTexture, InternalTextureSource } from "../Materials/Textures/internalTexture";

          import type { IInternalTextureLoader } from "../Materials/Textures/internalTextureLoader";

          import { Texture } from "../Materials/Textures/texture";

          import type { BaseTexture } from "../Materials/Textures/baseTexture";

          import type { VideoTexture } from "../Materials/Textures/videoTexture";

          import type { RenderTargetTexture } from "../Materials/Textures/renderTargetTexture";

          import type { Effect } from "../Materials/effect";

          import { DataBuffer } from "../Buffers/dataBuffer";

          import { Tools } from "../Misc/tools";

          import type { Observer } from "../Misc/observable";

          import { Observable } from "../Misc/observable";

          import type { EnvironmentTextureSpecularInfoV1 } from "../Misc/environmentTextureTools";

          import { CreateImageDataArrayBufferViews, GetEnvInfo, UploadEnvSpherical } from "../Misc/environmentTextureTools";

          import type { Scene } from "../scene";

          import type { RenderTargetCreationOptions, TextureSize, DepthTextureCreationOptions, InternalTextureCreationOptions } from "../Materials/Textures/textureCreationOptions";

          import type { IPipelineContext } from "./IPipelineContext";

This file has been truncated. show original

If bgfx supports more, we can easily add more to this list if needed.

Anupam_Das · July 25, 2023, 7:28pm

yep we are using DOF and we have the defaultRenderingPipeline enabled and enable DOF from there. We will try to turn off DOF and try that. We will also try removing the defaultRenderingPipeline entirely from the scene and see if the error will go away.

Anupam_Das · July 27, 2023, 10:01pm

@bghgary looks like we were able to overcome our AA hurdles. Do you know when your AA fixes will be on main branch of babylon native? One unrelated question is regarding tonemapping operators. We’re using ACES and this doesn’t seem to be supported in Native. Am I correct on that and if so do you know what the status is on that? Docs say “some” postprocess are supported but does not go into specifics .

bghgary · July 27, 2023, 11:03pm

I’m actually waiting for you to confirm it works before I merge. It sounds like it’s working, so I will merge ASAP.

I am investigating the post process from the PG you linked above. There are definitely some gaps and possibly bugs in the implementation. I haven’t looked into ACES. Can you send a PG of what you are doing?

Anupam_Das · July 28, 2023, 5:57pm

@bghgary here is how we do the ACES tone mapping currently. there is no difference on how we handle it on the front end vs. back end… all goes through DefaultRenderingPipeline. Let me know if you have any questions or need anything else,

bghgary · July 28, 2023, 6:01pm

Thanks, I will investigate this today and get back to you.

bghgary · July 28, 2023, 6:05pm

BTW, is DOF important for your scenario?

bghgary · July 28, 2023, 7:56pm

This PR will make ACES work: Enable texture float rendering by bghgary · Pull Request #14096 · BabylonJS/Babylon.js · GitHub

bghgary · August 2, 2023, 12:16am

Sorry for the delay. I ran into some issues with the Babylon.js PR. I think everything is working now. You should use the following:

Babylon.js npm package 6.14.2 or later
BabylonNative at this commit or later
BabylonNatvieExamples still in PR here

One thing I will note for ACES tone mapping is that you might want to use the following code instead of a separate post process pipeline. Using a separate post process pipeline will affect the background which is probably not what you want. Doing it using the following code will only affect the model.

    scene.imageProcessingConfiguration.toneMappingEnabled = true;
    scene.imageProcessingConfiguration.toneMappingType = BABYLON.ImageProcessingConfiguration.TONEMAPPING_ACES;

bghgary · August 2, 2023, 8:03pm

BabylonNativeExamples PR is merged now. Both MSAA and ACES should be working now. Let me know if you run into issues.

Anupam_Das · August 10, 2023, 7:53pm

@bghgary thank you guys for the excellent support on this, all seems to be working great so far. we are venturing into actually rendering mp4 video using native so if we run into snags there i’ll keep you updated. exciting times.

Anupam_Das · August 23, 2023, 4:15pm

@bghgary Hello again, it appears that MSAA is broken again in Native. Not sure if this was a regression. we tried rolling back to 6.14.2 but still broken. I dont think anything on our end has changed in terms of the setup. Pretty much follows the example. We’re using samples:4 in the output texture. We are still using the default rendering pipeline but ive tested this with and without that. I’ll try to just run the example again and see if its happening in the example.

sebavan · August 23, 2023, 7:31pm

Is it broken even when you reverted ??? this sounds weeeeeirrrrdddd

Let us know how it goes with the example and if it is related to recent change in babylon.

Anupam_Das · August 24, 2023, 12:14am

Looks like this is not to do with Native or BJS. Examples work and it also works on some of our scenes but some scenes dont work. So its something within our scene setup. Some of the scenes use defaultRenderingPipeline but we’ve removed that and still it breaks. So something else with the scene is breaking msaa. I’ll update this post when I find the issue.

Topic		Replies	Views
MSAA not working in WebGL2 Questions	2	818	February 27, 2019
Using MSAA causes OpenGL errors with certain color-renderable formats Bugs	5	258	December 6, 2023
MSAA without DefaultRenderingPipeline? Questions	3	737	March 24, 2021
Understanding how BJS deal with antialiasing with and without pipeline Questions	9	1134	February 21, 2023
Using WebGL API directly to draw into a babylon RenderTargetTexture Questions	17	635	September 2, 2021

NativeEngine render target texture not working with MSAA

Related Topics