applyVerticalCorrection issue resulting in sliced model

[Pieter_van_Stee]

Hi there,

I’m working on an architectural project, and it’s going great.
However I have an issue with the camera it seems.

To be able to match the renders we made in 3ds max I have to apply vertical correction

scene.activeCamera.applyVerticalCorrection()

The model comes out like this in the viewport.

This is without the camera modifier applied.

Is there something I’m overlooking?
This is why we need the camera correction, to be able to match the renders we made.
It’s matching our renders (almost) perfectly

Thanks!
Pieter

[Pieter_van_Stee]

Anyone an idea about this?
Is there a way to ease out this VerticalCorrection? If I transition to an aerial position the VerticalCorrection is unwanted, and causes this clipping issue.

Please let me know if there’s anything that can be done about this.
Also a bit more control over this kind of tilt/shift would be great.
The tilt you guys implemented a while back is actually a bit more complex than shift.
For our renders we’re using mainly the shift (vertical & horizontal) which just shifts the viewport in “view-space” vertical:top-bottom or horizontal:left-right.
This would be awesome if that’s possible

This is/was the topic about the requested feature for controlling the vertical shift (/tilt).

Thanks!
Pieter

[Pieter_van_Stee]

Also… To illustrate a bit this vertical shift/tilt I made the following screenshots:

Initial camera with camera and target on same height.
This often causes the issue that we see too much ground and not enough sky/building/… to correct this you would look up…

Just looking up will look like this:
As you can see this causes the vertical lines to converge, it’s natural but not always what you want in your composition, therefore there are (in reality as well) vertical tilt and shift (not sure this exists in reality, but tilt lenses do) lenses.

Using such a vertical tilt lens will correct this by tilting the projection plane of the camera lens (if I fully understand it, more in detail on this here)
You can control this vertical tilt with a value, but usually it’s just wanted to get those vertical lines straight again, this can easily be done by using the “auto vertical tilt correction” (or the camera.applyVerticalCorrection() in BJS).
This will skew your view a little bit, but when not exaggerated it’s not a problem.

This will result in this view:

In 3ds Max:

When unchecking the “auto vertical tilt correction” you get to see the number with which the image is tilted. And this number is controllable. That would also be a necessary option in BJS, so that when we animate a camera transition we can let this number transition to match from the first camera to the next

At the moment I did this in the following way:

            const updateTilt = setInterval(() => {
              cam.applyVerticalCorrection()
            }, 5)

An alternative to this however is the vertical shift
Here we would start again with the same view, looking straight forward:

But instead of looking up and correcting with vertical tilt we would be using the vertical shift
What this does is like taking a virtual snapshot of the view but this extends beyond the viewport-boundaries and shifts, or pans this view up-down or left-right, like so:

What’s interesting about this method is that we can also use the horizontal shift, to choose our perspective center-point, but still shift the view a bit left or right if there’s something important that needs to be in view:
Here you can see the center of the perspective (circle) vs the center of the composition/image (cross)

So we would really appreciate to be able to control both the vertical shift (horizontal & vertical) and the vertical tilt (horizontal & vertical). And this with an automatic vertical tilt, but still controllable with a number so we can tween this tilt between camera positions.
Seconday it would be awesome if we could also control the lens shift (horizontal & vertical), just with a number (auto is not needed/possible (?) here).

Thanks a lot!
Pieter

[Pieter_van_Stee]

Great,
Already one thing to cross of my whish list: camera.projectionPlaneTilt

This can be used to animate the tilt back to 0 for aerial views where this is not really desired.

Pieter

[CraigFeldspar]

Hi @Pieter_van_Stee, thanks for the detailed explanation

So from what I understood, in BJS for now we only have vertical tilt correction, and auto vertical tilt correction - and you would need us to add horizontal/vertical shift, that’s correct ?

If I get this right in terms of maths, shifting the projection plane would mean that we offset the sides of the frustum by some % of the width/height of the frustum. It may seem simple on the paper, but we have many assumptions on the frustum (especially the mesh culling pass) that would be broken if we just updated the projection matrix. @sebavan do you think that’s something we can consider implementing ?

[Pieter_van_Stee]

Hi Craig,

Indeed, that’s what we would need to be able to match our rendered visuals.
I understand that the technical underlying system might be more complex than the esthetic surface.
But if this would be possible in any way this would be great.

We have been working with threejs in the past. There the V/H shift was possible, so maybe you could hav e a look how they did it?

PS, any idea on why the model is being clipped when the vertical tilt is applied?

Thanks so much
Pieter

[sebavan]

I guess we could at least support the rendering and disable culling in such which won t harm in many situations as we are mainly CPU bound not really GPU. Do you want to give it a stab ?

Also cc @PolygonalSun

[CraigFeldspar]

If only I had more time … I’ll see if this is an easy fix

[Pieter_van_Stee]

Thanks @CraigFeldspar

For now we can manage with the tilt correction, but would be awesome to have the Shift as well

[carolhmj]

Hey @sebavan would you like me to create an issue for this?

[sebavan]

Let s wait and see if @CraigFeldspar has some time first

[dennemark]

Hi,
I am running into the same issue with the clipping. It seems to be related to nearest and farthest clipping plane. But the camera viewport helper looks bit weird. It seems to cut off at a certain distance. Any ideas?

Normal Perspective

Normal Perspective View

Tilted Perspective

Tilted Perspective View

here is a playground showing the camera: https://playground.babylonjs.com/#L20FJ4#21
not sure though, why the tilt is not resettet on click of button.

[sebavan]

cc @CraigFeldspar

[CraigFeldspar]

Hi @dennemark, I’m not quite sure of what I should see on the playground you shared, could you walk me through it ?

[dennemark]

@CraigFeldspar sure

The playground shows a camera with projectPlaneTilt applied. The white square in the blue marked part is the minZ value. However, I am not sure why there is another angle in the red part. It cuts of the upper part of the view.

In the images of my prior post I was showing this in an example where the camera should show some buildings with vertical lines. The last image shows how the tilted perspective cuts of the upper part of the building because of this angle.

Actually I just changed the maxZ value to a value like 2. Please try it out in the playground. You will see, that the maxZ clipping plane is not parallel to the minZ value. I think this might be the issue, why this red angle exists - it is the maxZ clipping plane, but it tilts the further it is away.

[dennemark]

@CraigFeldspar can I up this? Do I need to add more information?

[CraigFeldspar]

Hey sorry @dennemark as always I’m swamped and this issue fell through the cracks, gonna have a look before the end of the week

[CraigFeldspar]

Hey @dennemark, I could have a look briefly, and I think I know what this is, but I would need your help.

I see you shared a screenshot with your models being clipped. Could you by any chance reproduce that on a playground ? The gizmo is actually inaccurate on that, I would need a simple example where a model is actually clipped in the view when applying vertical correction

[dennemark]

Is this ok? https://playground.babylonjs.com/#L20FJ4#22
You can see how the boxes are cut off.

[Pieter_van_Stee]

Hi there, as we’re working on another project involving this vertical tilt I would like to ask for your help again.

Via this link you can view the current state of the project

For this project we are working for a client where we want to present a new construction project. The model is presented in a babylonjs scene, where we have an animation throughout controlled through swipe/scroll. When you near a chapter point the viewer will get snapped to the nearest chapter point.

On most of the chapter points, there is an overlay image. As this is one of our selling points this is very important to get right.
However at this point using the scene.activeCamera.applyVerticalCorrection() gives a few issues which I would love to get resolved, as we will use this feature a lot in the future.

clipping issue
First we have the clipping issue. So when applying the applyVerticalCorrection()* this will introduce a sliced view, where the top of the scene will be removed from view. Don’t know why, or at which value this will be cut off.
This effect can be deminished by setting the minZ of the camera to a higher value. Changing this in my case to for example 1.5 or higher will resolve the issue of clipping the top of the scene, but this will clip everything near to the camera when you’re moving close to certain objects.
This is also not desired, and I’d love to see this resolved.

Overlay mismatch
A second issue we are having a different 3D view from the same camera from which the render was made. As you can see the both don’t match nicely. There appears to be a vertical mismatch between the image and the 3D model.
Please let me know if you need additional information about this issue.

*VerticalCorrection()
In architecture is’t often desired and important to have a nice composition for your image.
Therefore photographers & architects often make use straigt/parallel verticals without the vantage point above (or below). When looking straight on it’s no issue as this is naturally the outcome, however usually we want to show a bit more sky, or the top of the building. This will introduce a vantage point high up in the sky, and will lead to converging vertical lines

An example below:

(instructables)

We’d like to get the middle sketch.
This can be achieved in 2 ways: vertical shift & vertical tilt.
Vertical Tilt will look up, and tilt the “render” back so that it looks straight.
Vertical Shift will look straight forward, and pretends the “render” is bigger than the viewport, to slide/“shift” the “render” into place

Both will deliver a similar result, however they can be slightly different, as (I presume) they are calculated in a different way.
In the past we have been working in ThreeJS, there we had the option to work with vertical shift, which worked 1:1 with our CAD software (3dsMax). This allowed to deliver us an exact overlay image, without clipping.

Please let me know if anyone can help me to resolve the overlay & clipping issues here.
Also let me know if you need any additional information to be able to debug this.

Thanks in advance
Pieter