Custom Attributes for Instanced Mesh to be used with Custom Shaders

Questions
1

Hello~

I’m currently trying to figure out the best way to have custom attributes for instanced meshes to be used in a shader. Unfortuantely I don’t have much to share in the form of code yet, since I’m still not sure about the right way to go here. And there are so many possibilities.

I stumbled across this thread on the forums and it seems similar enough to what I want to do:

Looking at the PBRCustomMaterial it seems like a great way to use the custom attributes once in place.
But I’m unsure on how to push it to the mesh. I was looking at the _renderWithInstances function of mesh.ts to see how the matrix world0-world3 is handled and that is where I’m currently stuck.

Just like aWeirdo did it in his overlay example, it would be awesome to have the attributes added to the mesh somehow. Like when the matrix gets added to the Buffer.

mesh.setFloatAttribute(instancesBuffer.createFloatBuffer(“customfloat”));

Would something like this even be possible without creating a new mesh type?

I have been looking at this problem since Babylon 3.x and always dreaded the copying of a lot of Shaderparts to make a potential pbr variant. But with the new PBRCustomMaterial it seems closer than ever. So I wanted to give it another shot.

Maybe someone can nudge me in the right direction of the right way to go, I would really love to get this working.

There might also be a couple of people coming from Unity that could be looking for this too.
Over there it is called per instance data: Unity - Manual: GPU instancing

Thank you!

2

Hello and welcome!

This has to be done in multiple steps:

1 Like
3

Hi Deltakosh,

thanks for taking the time!

I think I’m missing something important still, I played around with setVerticesBuffer before and ran into the same problem, I can’t seem to get the value to actually be transferred to the shader. I figured I’m not supposed to use it that way and moved on. :smiley:

Since you reassured me, I tried using your steps to make a new Playground example using the PBRCustomMaterial using the attribute to change vert position because that was the easiest to test:
https://www.babylonjs-playground.com/#FWCCWW#2

To “test” the shader part I just filled it with a static value of 2.0 (Line 32):
https://www.babylonjs-playground.com/#FWCCWW#3

Does it have to get set every frame or at a certain time maybe?

Thanks again!~

4

This could not work with PBR material as you need to hijack the list of known attributes:

The good material is probably the ShaderMaterial here

5

You were faster :slight_smile: I was just looking at all the objects going down the rabbit hole, when I noticed the attribute name was missing on the effect. I tried to hack the attribute to the effect in hopes it would work :blush

I was kinda hoping to leverage the features of the PBR shader, but I’ll definitely check out ShaderMaterial and report back. Still trying to grasp the whole picture :sweat_smile:

Edit:

Yup got it to work, with ShaderMaterial.
https://www.babylonjs-playground.com/#G3P9LH#2

Reducing one draw call at a time. :smiley:

Time for a nap, I’ll tinker around a bit more later. Would still love a nice way to achieve this with standard materials though. An attributehelper feature for the future maybe? :slight_smile:
I find it quiet handy in a lot of my Unity projects for various reasons.

Would love to make it happen. If you see potential for this too, maybe I could start working on something to get the ball rolling.

Thanks!

Edit:
Just noticed that I’m not done with the ShaderMaterial yet, and that I’ll have to look at the instancesbatch too, in order to fill the array appropriately depending on which instance is visible, like renderWithIntances does:

Thinking out loud, I guess a potential “extension” would need to tell shaders (geometrybufferrenderer, depthrenderer, etc. too I guess?) which custom attributes to use on compile and the mesh would need to know as well in order to handle everything in renderWithInstances depending on visibility. At least we are already evaluating it so that might be a good place to handle that anyway.

Ok, really time for some sleep now, thanks again for the help, learned a lot about babylon today. :yawning_face:

6

If you want you can try to submit a PR that will let stdmaterial and pbrMaterial have a new userAttributes property (like a string[]) that you can use to extend the attribs list

This way you could just use regular materials

7

Will definitely try :slight_smile: Was I right in the assumption that for a clean solution the attribs are also needed for the depthrenderer, buffergeometryrenderer etc.?

I also noticed that in some of the cases like with the shadowGen the MaterialHelper wasn’t used to apply instance attributes. If there wasn’t a particular reason for it, maybe that could be a nice first PR for me as well, so the ‘world0-3’ strings are all in one place (MaterialHelper) before messing with the attributes further.
For instance: Babylon.js/shadowGenerator.ts at 33471939d8b696799fa57796b47b08c5bf834702 · BabylonJS/Babylon.js · GitHub

The thing I’m still uncertain about is the batch handling going on in _renderWithInstances. Is there any nice / preferred / elegant way to “hook” into that process? Or would using scene.beforeCameraRender be the way to go here outside of the meshclass.

8

Well depthRenderer and all other guys only needs geometry info and if you want to upadte them it’s gonna be tricky

I will recommend to start with regular materials first.

Also you are right we can start by unifying the way we deal with world0-3

9

Thanks for the input, I’ll start with that when I get home again.

In regard to the renderers, agreed, I’ll leave it for now, just wanted to get a feel of what else might be involved in the long run. :slight_smile:

Edit: Maybe somewhen in the future, it would be nice to be able to leverage such an attribute system to also enable per instance mesh morphing with individual weights where needed, which would be pretty sweet imo. :smiling_face_with_three_hearts:

1 Like
10

Can’t agree more