To all, sorry I kind of went off on a tangent discussion with @shimmy, but it is a very relevant topic to the final implementation, plus it’s a suggestion that I think will benefit everyone in the long run. Anyway, here’s the next installment!
Part 4 - Implementation (cont.)
So when we last left off, we had a plan for how to create a ping pong buffer for progressive rendering. If you would like to see the actual code for setting up the render targets inside Babylon, please check out the ongoing port by @PichouPichou (main port) and @JohnK (BVH port) in the original Babylon path tracing thread
I ask that you refer to their port code instead of me directly copying and pasting all my old code here and discussing each line because firstly, it would take too much room and time, secondly it is changing on a daily basis as they work through the port, and lastly, I am not as familiar with Babylon as I am with three.js. So I would like to defer to the above coders’ exact implementation if you would like to see a line by line implementation.
Now we can start talking about the various cameras necessary for path tracing in this system. As previously mentioned, since we have 3 separate scenes - pathTracingScene, screenTextureScene, and screenOutputScene, we must now have 3 cameras, 1 for each scene.
I’ll talk about the screenTextureScene and screenOutputScene cameras first, because they are identical in terms of camera needs, and in fact with three.js, I just created a single camera that was linked up to both scenes. Then you just call render on each sequentially. Since the rendering occurs in 3 separate scenes and these 3 separate stages, we can reuse the same camera for 2 of them.
This camera I’m referring to is an orthographic camera. The view frustum looks like a regular box, so there is no perspective. We don’t need perspective on these 2 scenes because their total geometry is 2 large triangles that cover the screen. These orthographic cameras/or camera (if you reuse) will not move at all, nor change any camera settings during the entire app. The 2 large triangles sit right up front in the view of these orthographic cameras and the 2 triangles in their scenes do not move either. Think of the whole set up as a big movie theater movie screen. The big screen doesn’t move, but the path traced image it is displaying will appear to have 3d depth and will definitely move and constantly update hopefully at around 30-60 fps.
I saved the trickier camera for last - the pathTracingScene camera. It requires multiple features for everything to work right. Similar to the other 2 scenes we just covered, it needs the 2 large triangles so that we have access to every single pixel on the screen and so we can send an individual personalized ray through each of those pixels all at the same instant, in hopes of doing the ray tracing in parallel on the GPU. However, the pathTracingScene also needs a dynamic camera that can move, rotate, change field of view (zoom in and out), and a camera that has some sense of depth with objects that are farther away getting smaller. A free perspective camera will fit the bill perfectly!
If you look at the list of uniforms sent to the pathTracingFragmentShader, you’ll see a 4x4 matrix uniform with the name uCameraMatrix. This is how we’ll get the updated free camera info in Babylon’s js engine sent over to the path tracer. This matrix holds the camera position and rotation and will update every animation frame. Also, in setting up the free look camera, we will provide the usual camera attributes, like its field of view and aperture, both of which will get sent via float uniforms to the ray creation stage inside the shader’s path tracer.
If the field of view is low or narrow, the rays cast out from the camera will be more parallel, and if the field of view is high or wide, the camera rays will diverge away from the line of sight more, giving a more distorted view. If the aperture is 0.0 (a theoretical pinhole camera with an infitesmal aperture opening), the rays will all start exactly at the camera origin point, and travel along their assigned ray directions, producing a perfectly sharp image, no matter what the distance from the camera. As the aperture increases in diameter (more like real world cameras), the rays will first start at a randomized aperture location (based on the requested diameter) and their directions will be toward their own focal point out in the scene, but their directions will be offset slightly by the random aperture position. The scene objects that happen to be close to the focusDistance point will be rendered perfectly sharp. As objects get farther and farther away from this point out in the scene, they get more blurry. As the aperture size increases, this effect becomes more pronounced.
Now here is the tricky part: the path tracing camera must be a perspective camera so we can talk about field of view, but it also has to render to the 2 large triangles that cover its viewing area, and if you recall, when we were dealing with the 2 triangles in the other scenes, we used orthographic cameras. This has the unwanted consequence that if the perspective camera turns or moves, the movie-screen 2 triangle plane gets moved around and eventually gets clipped when it falls out of view! You’re left with a black background and your beautiful path traced scene is being rendered on a quad that has disappeared, like a flat-screen TV floating away in the darkness of space, ha ha.
After much trial and error, I finally found the solution: make the 2 triangle quad a child of the perspective camera. Or another way to look at it, have the camera parent the quad. This way, the quad gets ‘stuck’ on the screen and no matter what crazy motions you make with your camera, the image quad (2 triangles) sticks right to it, kind of like in a funny movie where a large newspaper flies onto a windshield of a speeding car and sticks there no matter what the driver does, ha ha! 
A couple of extra camera parameters are needed in path tracing: Focus distance as a uniform float, and a boolean uniform of whether the camera is moving or not. I will go into more detail down the line when we get into more of the actual path tracing and sampling theory aspect of rendering.
I won’t cover user input like mouse and keyboard, because it is handled with the underlying js engine (Babylon in this case) in the usual manner. These inputs are typically linked to the camera variables and their uniforms that we just discussed. That way the path tracer can immediately respond to user input, such as increasing the camera’s field of view with a mouse scroll action, for instance.
In the next post, I’ll cover the scene objects and geometry that we want the path tracer to actually render and interact with.
Till next time!