Cube RenderTargetTexture cameras, invertY, and rendering

Hi all, I have a few questions about RTTs.

First, for a Cube RTT (actually, same for a normal RTT as well), I noticed the texture is created with invertY as true and there doesn’t appear to be a way to specify otherwise. In the below PG, I’ve addressed that by adding a parent xform node to the camera with y inversion and rotations as needed for the +y and -y faces. It works, but if I can create the texture without inversion, that would be simpler.

Second, regarding the way I’m adjusting the cameras for each face of the cube RTT render, it seems a bit hacky/inefficient. I think by changing the camera target each time, the matrices need to be recalculated many times and can’t be cached. Is there a better way to do this? I noticed a camera rig concept that looks like it’s used for VR which might be similar to this case but didn’t find much documenting how it works.

Last question, I’d like to be in full control over when the RTT renders. refreshRate = 0 works well to render once after which I can choose when to render the RTT again, but not exactly what I want. I think using a separate scene might work, but I will have large meshes that I don’t want to duplicate in both scenes. Any thoughts here?

In the PG, I have a skybox in the main scene that samples from the RTT. The RTT renders another skybox with static texture and a rotating camera.

This only impacts texture “upload” mainly when data are unpacked but not for rendering per se.

It looks highly similar to what we do in https://github.com/babylonjs/Babylon.js/blob/master/packages/dev/core/src/Probes/reflectionProbe.ts#L139 and yes I guess some info could be cached per index as long as the center does not move.

Not sure what you want here ? (not rendering at all before using it will result in potentially broken data in the texture which is why the API enforces it I believe). If you are adventurous :slight_smile: you can set the private _currentRefreshId to -2 which will never render automatically. If that works we could add smthg for this.

1 Like

Thanks for your response!

I thought this was the case, but I couldn’t come up with another explanation why all the faces of the texture were inverted in Y in this PG https://playground.babylonjs.com/#V6U09N#5
I suppose it must have something to do with how cubemaps work internally in OpenGL. It does say the v coordinate goes “down” for X and Z.
Cubemap Texture - OpenGL Wiki

Thanks for the reference! Looks like another option for getting the camera xforms set up correctly.

I was finding it tricky to get the framework to actually only render once when setting refreshRate = 0. I think it might be detecting that something changed in the scene and needed a rerender? I’m not sure. Here’s a PG showing the behavior https://playground.babylonjs.com/#V6U09N#6 - face 0 is rendered between 5-7 times each reload.

I tried your suggestion, _currentRefreshId = -2, but that didn’t work I think because of the above issue, resetRefreshCounter() is getting called overwriting it. So I tried creating a subclass where I just turn off _shouldRender and that works… mostly.

Strangely, if I only render once, everything looks good except for the +X face, which is upside-down. If I render a second time (uncomment line 120) the +X face is corrected.

Yes, it will render in the RTT until all ressources are ready. You can use scene.executeWhenReady to make sure everything is ready before returning from the create scene function:

Using the executeWhenReady method as stated above + passing true as the second parameter will fix this problem.

1 Like

I’ve been digging more into this today, and I was still finding it surprisingly difficult to get my RTT to render correctly with a single render call.

My understanding of the PG you linked, is that it is preventing the renders that happen while the materials are still loading by preventing the render loop from starting until the entire scene is ready. Makes sense, but won’t work when I need to RTTs on the fly while having a main interactive scene.

I tried to adapt your PG to use an explicit render call, but still running into the same strange +X upside-down issue. https://playground.babylonjs.com/#V6U09N#13

I even tried a totally different approach for waiting for the specific material, effect, and texture to finish loading and got the same issue. https://playground.babylonjs.com/#V6U09N#12

In both cases, uncommenting the second render line results in a correct render. I still trying to determine why the single face is upside-down when only rendering once.

Immediately after posting the above, it occurred to me to try fiddling with the transform node for the first face, and I found that no changes were being reflected in the render. So that meant that the matrix was not being calculated correctly for that face. Turns out this line was preventing the transform node’s world matrix from being calculated, only for the first face.

Not sure if the scene’s render ID is missed from being incremented somewhere or if it’s just a quirk of they way I’m using the camera, but here’s a now working PG, rendering only a single time to the RTT.

1 Like

Good find!

I would say it behaves as expected because the world matrices are automatically calculated once for the frame, and you are modifying a node that has already had its world matrix calculated. Because you need the world matrix to be up to date for the current frame, you have to explicitly regenerate it.

Also, it works for faces > 0 because scene.renderId is incremented after each face is rendered. But this is a bit of a “lucky” side effect because it turns out that the Node.getWorldMatrix uses scene.renderId as part of its dirty mechanism…

Thanks to your findings, I noticed that when checking for RTT availability, we don’t increment the scene.frameId counter when we should. This PR will fix that:

With this PR, your PG will also be fixed and you will no longer need to call invertYFixNode.computeWorldMatrix(true).

However, as mentioned above, this works more because of the way things are done internally and I would still call computeWorldMatrix even with the PR merged in (for example, if the texture was a regular 2D texture and not a cube, this would not work even with the PR applied because in this case we don’t increment scene.frameId after the RTT texture has been generated).

2 Likes