Projecting world space coordinates from one scene to screen space of another

Questions
1

That’s a long question so here is bit more detail what I am trying to achieve.

I am setting up couple of alternatively rendered game scenes, one for 3d world, another for 3d tech tree and additional always rendered UI scene, required by either case.

Here is basic concept

------[ui scene]--------  (always renders), throttled to ~20/30FPS
---[tech-tree scene]----  (on/off) renders at optimal 60FPS
------[game scene]------  (on/off) renders at optimal 60FPS

Main reason of doing it is GUI lacking performance. Following split would allow throttling frame rate to avoid dropping frames while rendering others.

Challenge comes with rendering GUI labels over correct elements rendered in another scene. Hence the question:

What is the best way to go about projecting world space coordinates from one scene to screen space coordinates of another scene?

2

Took me few hours but I think I got it. Started with little proof of concept for projecting from world to screen space.

https://www.babylonjs-playground.com/#MTN1PJ

Going from there I have added second UI scene that renders over main scene. Figuring out scene transparency proved to be the biggest challenge.

BabylonJS doesn’t seem to respect uiScene.clearColor set as Color4 with alpha value, regardless of autoClear value.

uiScene.clearColor = new BABYLON.Color4(1,0,0,0.5);   // potential bug
uiScene.autoClear = false

Lastly updated projection calculation to project onto ui viewport. I used uiCamera for viewport calculation although either ui or main camera can be used since their values of their viewports are the same.

Here is final result.
https://www.babylonjs-playground.com/#MTN1PJ#1

Screenshot 2021-01-22 at 07.16.33
Screenshot 2021-01-22 at 07.16.331704×864 59.4 KB

Final step would be throttling renders for each camera but that is outside of the scope of this question.
Example of how to do it can be found on stackoverflow/a/19772220/6096446

1 Like
3

The clearing is not a blending with the already existing content of the framebuffer (coming from the main scene rendering), it will write the color value you provide to the framebuffer. If alpha < 1, there will be a blending between the framebuffer and what is behind the canvas (if any) but not between two scene rendering. You can use a post process if you want to apply a color blending after the first scene is rendered.