I have tried all settings in the node blocks that I can think of (via Inspector) but can’t figure out why it isn’t working. It looks like the refraction vector isn’t applied at all.
Am I missing something or is this a bug in NodeMaterial?
Another related problem that I’m also not sure if this is a bug or just some hidden settings somewhere: Adding thickness based tint to the transmission.
If I can get these working, then its just matter of extracting thickness from a RenderTargetTexture depth buffer (that renders opaque materials and backfaces of refractive materials) and I think this is gonna be amazing.
There is a bug when calculating the refraction coordinates in the right-handed system (the PG works in the lhs system), depending on the type of the texture (2D or cube). This PR will correct the problem:
Also, you should not connect the alpha input to the FragmentOutput block, the material has no alpha (the refraction is handled without needing the alpha component) and you will break the rendering if you do.
With the correction, tinting the refraction will work, as long as you don’t connect the alpha component and don’t use the “Link refraction to transparency” switch for the Refraction block:
@Evgeni_Popov Hmm. I already flagged this “solved”, but just realized that the alpha solution doesn’t sound right.
What if there is no skybox or other geometry in the background. In my use case such (default) mode should render the transparent model with alpha depending on the active opacity value: https://www.babylonjs-playground.com/#FIWZP6#8
With tint active I would at the very least expect the refracted content that isn’t transparent to become tinted. A fully correct behavior would also increase alpha depending on how thick the volume is, but that I can likely hack with node blocks and isn’t really relevant to the issue of whole tint disappearing.
Could you provide a PG using the PBR material that would show how it should behave?
If you set tintAtDistance = 1, remove the connection to the FragmentOutput.a input and uncheck “Link refraction to transparency” from the Refraction block, you will get the same output than when using a PBR material (when the PR is merged!):
You can provide a thickness texture to modulate the refraction effect.
Sorry, I didn’t explain it well. I already saw that the same currently happens with PBRMaterial as well when link refraction to transparency is active.
I was just saying that why isn’t tint applied if alpha is connected? I don’t see any reason for it and the current behavior seems like a flaw / bug.
Therefore a “correct” behavior would apply tint, and increase alpha to account amount of light absorbed by the volume, no? Just like the existing useRadianceOverAlpha increases alpha to allow specular highlights even if alpha is zero.
I think linkRefractionWithTransparency is a legacy parameter that is kept for backwards compatibility, but really shouldn’t be used. cc @sebavan to confirm (or deny!).
In the implementation, the tint color/thickness is not used when linkRefractionWithTransparency=true, which is why the refraction is not colored (I think these parameters were added after the implementation of linkRefractionWithTransparency).
Note that linkRefractionWithTransparency is a property of material.subSurface, not of material.
Hmm. Hopefully that isn’t the case. In web use case it is very common to render 3D to a transparent canvas so IMHO that should be supported where possible. (And in case of tint it definitely is possible)
I suppose worst case scenario I can just implement my own tint behavior on top of the PBR node “refraction” output. But hopefully that isn’t the case so waiting for @sebavan input.
As @Evgeni_Popov mentioned, we can not really change the behavior now for back compat reasons
The linkRefraction... flags does 2 things:
Disable blending
Read refraction data from albeado and alpha (for respectively tint and refraction intensity)
The fact that it blends with the canvas is mostly the issue here as it means we are basically leaving some alpha values despite not blending and so several layers will all overwrite each other
Not really. With refraction that is indeed the only correct way to do it. Whether you are refracting environment texture, or a transparent render target texture like in this case. You need to overwrite everything behind it (or the un-refracted content would show through)
Maybe I’ll have to come up with a PR with a new toggle to enable tinting over transparency at some point, if maintaining custom tinting code proves too difficult. But first need to come up with a working proof of concept with depth buffer based thickness and all that.
I managed to come up with a pretty robust solution for calculating depth:
Accumulated rendering of depth values to a float texture where backfaces add depth, frontfaces subtract depth. As a result you get good thickness value for volume meshes as long as the mesh is closed.
Unfortunately accounting items inside volume in the thickness (such as the mosquito in the earlier model) requires bit more hacking in NodeMaterial as that texture needs to be sampled at the refracted coordinates, and there is no way to access those from NodeMaterial… Therefore I’ll just have to replicate refraction manually to make that work.
Seems to also work pretty well for Translucency to approximate subsurface scattering effect. At least as long as using very low resolution thickness texture. (Could be handled with mipmap lod bias if NodeMaterial only allowed to sample with one…)
// Need to set alpha to enable alpha blending even with transparencyMode??
shaderMaterial.alpha = 0.0;
You don’t need to do this, you need to set needAlphaBlending: true in the options when creating the ShaderMaterial instead (the shader material does not use the transparencyMode property to enable/disable blending).
Phew, took a while to replicate refraction + tint entirely in NodeMaterial so could “proof-of-concept” test the opaque volume factor component that I calculated in the render target green channel.
@sebavan Would be amazing to get this implemented in Babylon.js PBR workflow itself. Looks like I’m going to have huge headache adding this much modifications via NodeMaterial to do this. Even if I use custom node blocks to do most of the work…
I think this could be added as additional PBRSubSurfaceConfiguration / SubSurfaceBlock property. As in:
useVolumeRenderTarget = true or something like that. Which when enabled would:
Automatically generate the render target setup and apply the textures to materials that have the flag enabled
Override custom thickness values with the generated ones
Those would be mainly matter of just adding a new #define conditions in the subsurface shaders to use calculated thickness / tint values if the option is enabled
Use the refraction texture from the render target (unless user has specifies manual refraction texture)
And you would also have opaqueMeshesList property or something like that where people can add meshes that should be refracted & included in the transmission thickness calculation (both come at the same cost after all)
That being said, that is way too deep in Babylon.js internal feature for me to add personally. But can of course join in helping + testing if someone wants to add this.
As we are planning our next release including rendering improvements, I wonder what @Evgeni_Popov think about this one ? sounds like some of the refraction updates we discussed ?
To add to my thoughts above. Probably a multi option would be better than just boolean flag.
For example: computeVolume = Constants.VOLUME_ACCUMULATED; or so. But in the future it could be easily extended with for example VOLUME_RAYTRACED that would work as parallel option. And users could use the older less precise volume calculation methods as fallbacks when new and more advanced ones arrive.
