Show projected 'cursor' on mesh surface

I’m looking for a sanity check on my implementation of how to show a projected brush cursor on a mesh for a “drawing on mesh” operation. I’ve tried searching forums and playgrounds for something similar but didn’t find anything close enough. I want to be able to draw multiple brush sizes and shapes onto the mesh.

Currently, I’m using a Dynamic Texture, and manually drawing the cursor onto it with Canvas operations, and mixing that texture with the diffuse via a custom node material. I might move it to it’s own material as we’re already using MultiMaterial on the target.

I considered animating a mesh above the target mesh, but was concerned with Z-fighting on that approach. Decals also seemed more suited to a permanent modification of the mesh texture and not ideal for something that follows the cursor.

We’re still on Babylon 4.2 if that rules anything out from the 5.0-alpha builds. Any suggestions?

1 Like

Could you use a spotlight / projection texture for this?

You can consider using CustomMaterial or PBRCustomMaterial if your mesh is already using StandardMaterial or PBRMaterial.

Please find the documentation for custom material.

The part you might need is Fragment_Before_FragColor. You can pass where you want the highlight as a 3D coordinate as uniform input. And then you can modify the color you want to highlight for the area close to the input 3D coordinate.

Great suggestion, and then I could draw the cursor shape on a DynamicTexture to use as the projection texture (since it’s resizable / reshapable).

I’m using only part of the StandardMaterial node material setup, only enough to get shadows, so I’ll have to see if I need to bring more of the standard material in or not.

Seems like this must be much more performant than redrawing a texture every mouse movement?

Not even sure you need to use the dynamic texture at all per se. You could just have the textures pre-built and swap out the projection texture on the spotlight. But if you have some specific need to generate the texture dynamically I believe that would still work.

I think on Node Materials the spotlight would just come in as part of the lighting information. I’ve done it once before, but forget off hand. You can definitely get the spotlight to show on a node material though.

And I would think this approach much more performant than the dynamic texture. It would end up being mostly matrix math and the GPU, vs notoriously slow canvas drawing and the CPU.

I brought it in and the light does display on the material I’m using, but it reintroduces an issue I had previously with lighting where everything gets a blue tint. I fixed it previously by reordering some assets and awaiting them before setting up lights, but adding this selection light in where I add the other lights causes the same issue. If I run the inspector, and disable/renable the directionalLight the scene renders normally, but I’m not sure what the root cause is to begin with.

Is this something anyone has seen before? I’m not working in playground so would take some time to recreate an example. I suspect it’s some kind of race condition.

It seems like this weird blue lighting issue happens if I create any lights after the DirectionalLight is added to the scene (this one has a ShadowGenerator attached). If I create the SpotLight before this, everything is rendering correctly. I’m glad I can work around it, but I still don’t understand why this happens.

I found an example PG based on Fragment_Before_FragColor.

1 Like

Thanks. I’m on a node material so I’d have to implement that manually, but that’s another option - render the cursor entirely in the shader after passing the point in as a uniform.

Another issue I’m battling using the spotlight is it is illuminating meshes loaded after the spotlight is created even though I’ve set selectionLight.includedOnlyMeshes = [ this.ground ]; when I create it. Do I need to start using layer masks to have it only even affect the ground and ignore everything else regardless of when it is loaded?

ohhhh Could you create a quick repro in the playground ? it sounds like a bug to me

I’m thinking this is a race condition. I’m able to replicate it on this playground if I throttle my network Babylon.js Playground but that’s obviously because it’s waiting on the texture to load and parse before setting the includedOnMeshes. I can’t get a reproducible test case in the playground otherwise, and so I’m assuming that some of the promises for asset loading in my case aren’t waiting as expected. At any rate, I was able to restructure some code and remove the issue, and will be looking into those asset loading methods as well.

1 Like