Babylonjs equivalent of Threejs's modelViewMatrix?

I’m trying to create equivalent GLSL for a simple fresnel effect shader in Babylon, working from a Three.js shader. Here’s how it’s done in Three.js: Glow Effect - ShaderFrog.com (click on the “vertex” tab to see the source)

  vec4 pos = modelViewMatrix * vec4(position, 1.0);
  fPosition = pos.xyz;

I’m trying to port this code to Three.js. There’s probably other ways to achieve fresnel in Babylon, I’m interested in how to convert this GLSL specifically though.

Based on looking at the matricies in Babylon, I’ve come up with this:

    vec4 pos = (world * viewProjection) * vec4(position, 1.0);
    fPosition = pos.xyz;

This is close. However, when I rotate my object, the fresnel effect doesn’t stick to the rim properly. So it’s not taking into account the view matrix?

Screen Shot 2022-03-27 at 6.04.49 PM

I see babylon’s shaders define

uniform Scene {
  mat4 viewProjection;
  mat4 view;
};

But I don’t see anything else related to the view. Am I missing something? Or do I need to do something specific to force Babylon to update the view matrix in my shader?

I’m generating this shader by creating a PBRMaterial, then modifying the source GLSL via processFinalCode, and injecting new code in.

Hello! Providing us with a Playground repro would be helpful for us to take a better look at your code :smiley: But looking at these snippets you provided, one possible issue is the order the matrices are being multiplied: it should be (viewProjection * world), so that the world is the first thing to be multiplied by the position. :slight_smile:

1 Like

@carolhmj I was able to build a reproducing playground - interestingly in my playground the rim does move as the camera moves, but it’s in the wrong place on the object, so I think I’m doing the math wrong still

Oh, here we go, the ArcRotateCamera doesn’t update the view matricies properly, note how the fresnel rim doesn’t move in this scene, using an ArcRotateCamera, but it does (even though wrong) with the FreeCamera

I’m not sure what’s going on yet, if I add a breakpoint here in my project:

The breakpoint gets hit, so I think the viewProjection and view matricies should be supported regardless of the camera type?

I would think your computation of the fresnel is having an issue as world and viewprojection updates correctly. the simplest proof is that when you move the camera you can see the object from a different viewProjection.

You should double check your fresnel computation.

@sebavan To disambiguate this as an X/Y problem:

I’m trying to automatically convert a Three.js shader to a Babylon shader, by mapping uniforms, varyings, and other data, from the GLSL source code of a Three.js shader to a Babylon shader. It’s working in trivial cases. Here’s the same noise shader running Three.js, plugged into a Three.js PbrMaterial’s normal map (some other magic goes on to be able to use shader as a normal map):

And, after my tool performs a source code conversion, that same shader being plugged into a Babylon PbrMaterial’s bumpMap (same magic needs to go on to make the shader usable in a Babylon uniform)

Mapping the fragment shaders isn’t too bad in the trivial case. Now I’m trying to go a little deeper and map vertex shaders, which is why I’m asking specifically about modelViewMatrix. I have a Three.js shader which references this uniform, and I’m trying to find if there’s a 1:1 mapping I can do between this uniform and Babylon uniforms, to perform the conversion automatically.

My tool uses a compiler, so it can manipulate source code to do anything, including replace any references to modelViewMatrix with any other expression, and it can also inject or remove uniforms in the translated Babylon shader if needed.

That’s why I’m avoiding different logic like using vEyePosition and I’m looking for a 1:1 translation in this case.

1 Like

modelViewMatrix is mesh.worldMatrix() * camera.getViewMatrix()

or at shader level it would be: worldView

1 Like

@Deltakosh if I add the worldView to the shader in place here it doesn’t update as I rotate the ArcRotateCamera - do I need to add a per-frame calculation in javascript to update this uniform if I wanted it to update for this camera type?

Sorry I was unclear: it is world * view

test | Babylon.js Playground (babylonjs.com)

@Deltakosh Thank you for the example link. It also appears your link has the same issue mine does, where rotating the camera doesn’t keep the fresnel rim “stuck” to the edges of the object. My assumption is that the view matrix is not properly getting updated in the shader, so the rim is only accurate at a specific angle. Do you know why the view matrix isn’t getting updated (or is my assumption incorrect?)

view matrix is updated per frame so I believe the problem should lie somewhere else

(I double checked with Spector)

Hi @andyray just checking in if you’re still having issues and if we can help with anything :smiley:

@carolhmj what lovely community support, thanks for following up!

I think I’m trying to understand why this line in a Three.js shader:

  vec4 pos = modelViewMatrix * vec4(position, 1.0);

Is significantly different from this line in a babylon shader:

  vec4 pos = view * world * vec4(position, 1.0);

I’m guessing there’s some difference in the way three vs babylon handle the view matrix? or maybe the handedness of the matrix?

2 Likes

When you say “significantly different”, is it the visual results that are different? Or just the line itself?

1 Like

In this shader Glow Effect - ShaderFrog.com (you can use the tabs on the top left to view source, or click “edit source” to play with the source code). You can see the outline effect is properly “sticking” to the object when the object is rotated. The calculation of the fresnel rim is based on the model, view, and position, and the normal of the surface.

I’m trying to define a mapping between threejs matricies and babylon matricies. You can see in any of the babylon linked playground examples above, the effect does not properly stick to the rim of the object in babylon.

Ah - i just found the problem! The original shader linked above multiplies the normal by the normalMatrix, which I thought was a no-op, but the normalMatrix in three.js is the “inverse transpose of the modelviewmatrix” three.js docs

as a hack, if I do this

            mat4 x = inverse(transpose(view * world));
            fNormal = (x * vec4(normal, 1.0)).xyz;

Then the fresnel rim works properly.

3 Likes

Woah, nice job!