This might be one for you @Evgeni_Popov - we’re finding that for larger scenes, using the transmission helper that’s set up after importing GLTFs with the KHR transmission extension causes pretty strong perf degradation.
I have a couple of theories as to why this might be, perhaps you can offer some advice:
Mip-Map Generation
This could be a fairly expensive operation for a full-screen rendertarget. Last we spoke you suggested a PG that modifies the transmissionhelper rendertarget to be a half-float format. Unfortunately it doesn’t seem to be possible to disable mip generation in the same way? If you’re not planning on rendering frosted glass, this ought to be an unnecessary expense.
Rendering the Scene Twice
Given that the rendertarget seems to be set up similarly to how rendertargets for something like reflections are set up in Babylon, and the transmission helper actually passes a lit of opaque meshes through to the RT, I’m a little suspicious that it’s actually forcing the entire scene to render twice, in order to achieve the glass refraction effect? It would explain the perf drop on larger scenes. If that is the case, can this be avoided? In principle you should only need to copy the frame buffer to a new rendertarget after you’ve finished rendering opaque meshes, I would not expect there to be any need to render an entirely new version of the scene?
Do these concerns make sense? Is there any other reason you can think of that this would be an unreasonably expensive operation?
As a sidenote, it would be great to see some of this documented. Though I’ve been able to find some of the typescript that creates the transmissionhelper, I feel like it would be worthwhile to update the Reflection/Refraction docs with information on how the transmission helper works, and how you could recreate its functionality in code, if you wanted to set up more physically correct glass materials without importing a GLTF for it specifically - as it stands, you get good results with it, but it feels like something of a black box.
We are also not seeing the Transmission Helper getting created when loading the appropriate meshes into a secondary Babylon engine/scene/canvas. Is it possible that the original is getting picked up only ?
The mipmap generation is necessary to simulate roughness. If you don’t have roughness on your materials it could indeed by disabled. We are going to add a new option to the update function to let you disable mipmap generation.
Note that scene._transmissionHelper._opaqueRenderTarget.gammaSpace = false is not necessary anymore as it is the default setting now (the helper won’t work correctly with gammaSpace=true).
Regarding the rendering, yes, technically we could make a copy of the framebuffer after the opaque objects are rendered instead of rendering those objects two times (a first time to generate the transmission helper render target and a second time for the regular rendering). However:
we have no built-in mechanism to inject some code after the different passes of the rendering (opaque, sprites, particle systems, transparents). I guess that would not be too difficult to add some new observers for that
we would depend on the framebuffer type currently in effect and of the gammaSpace setting of it. Regarding the gammaSpace setting, we could make a conversion when copying the current render buffer to the transmission texture if gammaSpace is not false in the first place, but if the type of the current render buffer is not the one we want, for eg if it is unsigned byte (as it’s generally is) and we would like half float (which is generally what we want - or full float) then we are screwed, even if creating the transmission helper with half float type the copy won’t give us back the precision we lost when outputting to unsigned byte. Also, if some image processing is applied at the material level, tone mapping or other effects are already applied in the current render buffer, which is something we don’t want in the transmission texture.
Because of the 2nd point I don’t think we can come up with a scheme that would work in all cases, so what we could do is adding the observers as explained above (if it is ok for the Babylon’s team) and then you could try to disable the transmission helper and replace it by a copy of the framebuffer (after the opaque pass) and set this texture as the refraction texture of the transparent objects (with the caveats explained above)?
Also, as explained here the transmission helper is normally not meant to stay, we should replace it by something else, that’s why we are not really documenting it (should it stay, we will add proper documentation and better reusability).
Thanks, that makes sense! Yeah, I can see how there are some rendering pipeline set-ups that would make this difficult. I think it would make sense if this pipe isn’t compatible with an LDR pipeline, or image processing that’s run in the materials, rather than on the scene.
An extension was getting triggered that we did not realize and making a PBR material linked to the main scene.