RenderTargetTexture camera behavior

Hello guys,

I am working on a scene using the render target texture and I noticed a bug (or I am missing something?). If I set the refresh rate of my rtt to render once an call rtt.render() the rtt renders the active camera of the scene and not the camera attached to the rtt.

I created a pg. If you move the camera to a different perspective and press “r” the rtt generates an image of the scene. But as I said it uses the wrong camera.

https://playground.babylonjs.com/#N2FNND#2

Any ideas?

Best

Hi!
You can achieve the effect you want by setting layerMasks for the objects and the cameras. Add both cameras as active.

Look at this PG. The RTT will be refreshed every second. It will always render the box from above and the camera will get closer to the box each refresh. The forceRender variable can be left out and you can render the RTT directly in the setInterval method or anywhere else. I’ve just introduced it to better control when to render the RTT.

https://playground.babylonjs.com/#N2FNND#9

    module.layerMask = 1
    camera.layerMask = 1
    camera2.layerMask = 2

    scene.activeCameras.push(camera)
    scene.activeCameras.push(camera2)

And there is a little trick for RTT debugging:

    await scene.debugLayer.show()
    scene.debugLayer.select(rtt)

:vulcan_salute:

1 Like

Thanks, @roland!

That works fine :slight_smile:

Does it have any negative impact on performance to add both cameras as active?

1 Like

You are welcome!
I’m sorry, I don’t know the correct answer to your question, but I believe this is the correct approach for multiple camera rendering. Maybe someone more experienced will enlighten us🤗

1 Like

as @roland noticed, masking is the correct approach and it won t impact performances.

2 Likes

Thanks, good to know that this is best practice here!

1 Like

Weird! I adjusted my code exactly following your pg but it always renders the object with the perspective of the second camera above the normal scene with the first camera. Event though setting the layer mask of camera2 to 0 it is shown. I assumed that .layerMask = 0 doesn’t render it at all. Any ideas what the problem could be? Setting layerMask of meshes to 0 makes them invisible as expected. But anyhow it won’t work for cameras.

EDIT:

I found something. It is related to: scene.freezeActiveMeshes(): https://playground.babylonjs.com/#N2FNND#11

To improve performance I need to freeze active meshes. So is it still possible to handle this?

you could force your object to be always selected with module.alwaysSelectAsActiveMesh = true

https://playground.babylonjs.com/#N2FNND#12

I am using this approach all the time and I never had this problem. I suppose, something must be tweaked in your code to make it working. The best way to help you would be to see all your code, but I know it’s quite a hard decision to share private code. However there is still a chance that you have discovered another bug. :hugs:

As far as I know freezing meshes makes sense at the startup, when you already crated your static meshes you are not going to manipulate anymore. So you freeze them. The engine will not update the frozen ones so you gain some performance boost on the cpu side.

If you haven’t read this yet, I highly encourage you to do so:

Have a nice day!

EDIT: just realized that you have issues rendering a frozen mesh into the RTT. I bet @sebavan 's trick will solve this issue.

@sebavan
Unfortunately that doesn’t solve this issue. Even in your pg :frowning: The render target texture is still visible on top of the normal scene.

@roland
I’ve read this doc a lot so I am pretty sure I use all techniques mentioned in it :smiley: Thanks for your help anyway!

So the challenge here is to manually trigger the render of the rtt with a frozen mesh? Do I get it correctly?

Exactly. I am fine with unfreezing meshes temporarily but it seems that freezing meshes leads to this overlap of the layer masks in general. The layer mask of camera2 should just exist to create a snapshot of the scene from this perspective but the output should be invisible.

Why is life so complicated? :smiley:

As you have mentioned, you tried to freeze/unfreeze and it didn’t work for you but this seems to be OK:
https://playground.babylonjs.com/#N2FNND#13

    scene.onBeforeRenderObservable.add(e => {
        scene.unfreezeActiveMeshes() // added
        if (forceRender) {
            rtt.render()
            forceRender = false
        }
        scene.freezeActiveMeshes() // added
    })

    scene.freezeActiveMeshes() // added

Or one thing that has popped out of my mind just right now: Can’t you use a second scene to render the RTT?

1 Like

It would be too good to be true if it just worked straight away. :smiley:

Thanks again for your approach, @roland. Is it okay to freeze and unfreeze on every frame? I am afraid that the performance gains get decreased due to this method.

A second scene should be possible. I hoped I can render the RTT without that, but it doesn’t seem so obvious to achieve it.

But I got an idea to render the RTT without a second scene and even without layer masks. Am I not able to change the camera perspective right before rendering the RTT and resetting it afterwards? https://playground.babylonjs.com/#N2FNND#17

1 Like

Do you need to freeze and unfreeze every frame? I thought you need to render the RTT on demand. In that case you don’t need this:
rtt.refreshRate = BABYLON.RenderTargetTexture.REFRESHRATE_RENDER_ONCE

Anyway, cool trick how you solved the problem! :slight_smile: The most basic solutions are always the best! LOL!

Yes, I need to render on demand. But your pg used:

scene.onBeforeRenderObservable.add(e => {
scene.unfreezeActiveMeshes()
if (forceRender) {
rtt.render()
forceRender = false
}
scene.freezeActiveMeshes()
})

Therefore I was conscious if this is the most performant way if you call freeze/unfreeze on every render.

Since my whole scene only renders on demand the short manipulation of the camera perspective is not visible to the user. So changing the camera perspective right before rendering the RTT works for my case :slight_smile:

2 Likes

Oh yes, obviously the freezing and unfreezing must be put in the condition body. My bad :hugs::roll_eyes:

3 Likes