Sculpting Texture, possibly matcap

[MrDon]

I’ve searched around, vibe coded out, but for can not come up with a good texture for sculpting using Babylon 8, soon to be 9, and the GPU engine.

Here’s the effect I am looking for, Blender, zBrush etc.

Here is what I am getting

I’ve tried looking for matcap textures across this forum but nothing seems to work good. They often come off too shiny. example 1, example 2 or example 3

I’ve tried adding ambient occlusion to the scene and env light, or emissiveFresnelParameters, to no avail. My lighting setup is one spot light to the upper left and one to the upper right of the camera looking at the mesh being edited.

When I tried to add the post processing from this playground, with a refactor to GPU shaders, but my scene straight up glitches.

.

Any insight or direction would be greatly appreicated.

[fuyutami]

Hi, to use a matcap first you need to create a matcap texture: create a smooth sphere, add material, lights and create a render like this:

Then pass it as texture input to a shader like this: Babylon.js Node Material Editor

[sebavan]

Let me add @PatrickRyan our amazing in house artist to the thread

[MrDon]

Unfortunately, I can still not see the edits where the materials normals are facing the camera with this material.

[fuyutami]

@MrDon Do you compute the normals after displacing the vertices?
When you update vertices data with new positions you should do something like this:

const indices = mesh.getIndices()
const normals = []
BABYLON.VertexData.ComputeNormals(positions, indices, normals)
mesh.updateVerticesData(BABYLON.VertexBuffer.NormalKind, normals)

[PatrickRyan]

@fuyutami is on the right track with the matcap shader, but there are a couple of edits you need to make to dial it in.

This is your graph as you designed it, but you can see that there are some artifacts on the shader ball. This is because you are using local mesh normals instead of world normals. If you just place a world normal block before you transform to view space, you will correct for that. But that is where the second edit reveals itself:

You can see that now the texture used for your matcap is upside down on the mesh. This has to do with a long list of conventions that all affect one another, in this case what the Babylon engine expects for texture orientation and what glTF expects. So we just need to simply invert the Y axis in view space using a negate block:

Now depending on if you are using a Babylon mesh or a glTF, you may need to flip Y or not, so you may want to set up a parameter on a lerp to do that flip if needed for a mesh. But with both of those edits made, you can see the output is stable with complex normals.

Here’s the updated shader for you: Babylon.js Node Material Editor.

For @MrDon, in terms of what kind of texture you need, if you are looking to replicate what you see in some of the sculpting tools, you just need to experiment with the gradients on your matcap texture. This is one I quickly put together to mimic some of the qualities in your example. Keeping your highlight above the centerline on your sphere will help differentiate your surfaces as if lit from above and behind you. The broader you make your highlight, the more matte the surface feels and the sharper you make your highlight the wetter/glossier the material will feel. Placement of your highlight and how quickly the gradients interpolate between light and dark areas will have a big effect on how your material feels.

To get you started, I am attaching the png for the matcap I showed and the psd I used to make it so you can experiment with it. Don’t hesitate to ping with other questions.

sculpt_matcap.zip (267.4 KB)