I am really happy that I finally gave it a try to build a shader. My goal is to manipulate the color of specific vertices. To achieve that my plan is to pass multiple points to the fragment shader and use specific color intensities for each of them. All vertices of the object that are not defined by my stated points should use a default color. The array of points could be huge with thousands of points. I assume this is too much if it will be computed on every frame. But it is fine to render the shader only once because the points won’t change.
Unfortunately I encountered some problems:
Is it possible to pass an array of vector3 to the fragment shader?
How to check if the position is included in the array to get its properties?
You can’t have variable length arrays passed to shaders, you must use a fixed size. In your case, you can set the biggest size you will ever need, something like:
I assumed that my int pointsLength will be interpreted as static value. Your method would be fine for me but increasing the limit doesn’t work: “FRAGMENT SHADER ERROR: too many uniforms”. As mentioned the array could contain hundreds or thousands of points.
Unfortunately I can’t use the values of the vectors in my array. Is it even possible to check if the possition exists in the array to receive its properties? What is the best approach to color specific vertices?
If you want to assign a color per vertex and that those colors be interpolated by the GPU, then you can simply add a color attribute to your mesh and set a color for each vertex that way.
Regarding your PG, you must use setArray3 / setArray2 to pass an array of vec3 / vec2 to the shader. Note that Babylon is expecting you pass an array of numbers to these methods, not an array of Vector2/3:
How am I able to add a attribute with values for each vertex to my mesh?
I have a dataset of hundreds/thousands of vector3 coordinates which are points on my object I picked before. These vector3 coordinates I store in my points array.
In addition each point has two specific values (intensity, radius) which influence the color of this specific point. I want to store this information in my properties array.
My idea is that the fragment shader loops through all vertices of the object. Then it should check if the current position of the vertex is in my points array. If yes it should get the index to be able to look for the properties of the same index. If no the vertex should use a default color.
My goal is to create a heatmap on top of the model based on my data.
This doesn’t fit my needs since a picked point could be anything on the surface of the model (like x: 0.08883678461393885, _y: 0.9999999999999996, _z: 0.5396330116068166) and not just the vertices. So my initial approach can’t work in general?
But this leads to problems with the uv map of the model. In my case I can’t adjust the uvs so it should handle it independent of the uv. Is it possible to a custom texture that covers the whole model like an overlay?
Custom UV
I found a function that should unwrap the model based on the normal of each face. Unfortunately it didn’t work and leads to the same results as the image in my previous post (https://www.babylonjs-playground.com/#9MPPSY#16). How is it possible to create a new uv data set for the whole surface. I can imagine that it will get complicated with more complex objects than just a box.
Decals
Decals solve the problem with the faces and uv map but is it just possible to use textures instead of dynamic forms? https://www.babylonjs-playground.com/#9MPPSY#15. Is it possible the create a heatmap with decals?
Here’s an example on how to add a color attribute to your sample:
In the same way, you can add an intensityRadius attribute:
Note that you should combine intensity/radius in a single attribute (as in the PG) because the number of attributes you can use in a vertex shader is limited (and depends on your GPU).
[…] Just saw your edit:
Then you would need a texture instead I guess. Try to look for “heat map” in this forum, I think there has been some discussions in the past.
Yes, I think I have to go with a texture. My idea would be to merge all meshes and unwrap it. As mentioned here I tried to do that but the function doesn’t work. I assume that the algorithm is especially for complex models much more complicated. Is there an algorithm that I can use to get new uv data?
For testing I created a cube with a smart uv with blender. In blender texture painting works fine but in bjs it doesn’t handle it correctly: https://www.babylonjs-playground.com/#9MPPSY#17
I already looked for heatmaps here on this forum but also stackoverflow, threejs, google, etc. I wasn’t able to find anything that could help me.
Too bad. I would love to do that on my own. The result would be anything but a nice uv map though.
Alternatively I could merge and unwrap it with blender on my server? I would prefer to stay in babylon for this process but if that is the only way I have to do it. Or is there another algorithm for unwrapping which is public?
Besides that I have still no clue how to fix the issue with painting on the object (here). If I hit a face the point isn’t drawn on the surrounding faces.
I created this pg to visualize the problem a little bit better: https://www.babylonjs-playground.com/#9MPPSY#19. If you paint on a face it places the circle as expected on the texture. But this leads wrong positions on the 3d model if the circle touches an edge. I found this repository which solves this problem but I don’t know how to adapt this: Chameleon.js
I was able to link heatmap.js with my dynamicTexture since both are working with canvas2d. Passing the uv coordinates to it works pretty good and it behaves the same way as the pg above. Obviously this leads to the same problems. Since all values influence each other in terms of intensity I am not sure how it is possible to adapt the heatmap canvas to fit to the uv.
)