Hi everyone,
I have created an application with a mesh viewer, that looks roughly like this playground: Babylon.js Playground
My problem is, that using the current visualization, the user has a hard time seeing the tiny peaks on the mesh when the view angle of the camera is not perfect.
Unfortunately, for my use case, it is crucial that those specular highlights (? i think it is called like this) can be seen, in the best case from all directions.
Is there some way to improve the lighting in my scene, so that those highlights are visible no matter how the camera is oriented?
I already tried to add a spot light that always shines in the direction of my camera, but it doesn’t always create good results.
For example in this screenshot, there are almost none of the peaks visible:
Edit: Do I maybe need multiple spotlights from different angles?
I think something like a light box with multiple lights from all sides could improve visibility, but i’m not sure if that’s the correct way to solve this problem.
Edit 2: I’ve added more lights to my scene, switched from spot to point lights, and increased the max number of lights for this material.
I think the result is better than the first version, but if you have suggestions how to improve it, I would be happy to hear.
@major-mayer, the first thing I would try is to change from StandardMaterial to PBRMaterial so that you can use an environment texture to give your mesh illumination from all angles. This will prevent the need for multiple punctual lights and you are able to have light coming from more angles that won’t affect the limits of lights per mesh. You can find a bunch of IBL textures at polyhaven. If you download a .hdr file and then open a mesh in the sandbox and drag/drop the .hdr file on the sandbox window, the tool will convert the file to a usable environment. Then in the inspector under the tools tab, you can download an .env file which is the converted IBL file. You then use this as the environment texture in your scene.
Hope this helps but long back with additional questions.
Hi @PatrickRyan
Thanks a lot for the information. That’s so many new things for me at once i don’t even know where to start
I think if I want to make use of this sandbox, I first need to find out how i can convert my Draco mesh to a supported mesh format.
But I think the bigger problem is that I don’t know which of the hundreds of HDRIs available on Polyhaven would be suitable in my use case.
Intuitively I would guess a studio scene like this could fit, but since I have no clue how they actually work, I would like to ask you what you think?
@major-mayer, let me run through an example here of how I would approach it. First, we need to get an asset that uses draco compression in glTF. I chose this asset that I have from a previous demo:
I used a gltf version of this file and ran it through the gltf-pipeline command line npm package to convert this to a draco-compressed glTF, which will load into the sandbox.
As you can from the first screenshot above, the draco-compressed glTF can be opened in the sandbox without any issue. Now that we have the asset loaded, it’s a simple task to test out several HDRIs from polyhaven. I quickly chose an outdoor HDRI, then we want to download as an .hdr format. The size of 2K should be more than enough for the conversion so there’s no need to download a larger file. You could also try a smaller resolution if you want to avoid larger downloads. The smaller source file could be fine for some applications and may be worth testing.
scene.environmentTexture = new BABYLON.CubeTexture("environment.env", scene);
In terms of what environments are best, interior lighting studios like the one you linked are usually good for showing areas of contrast. They are also good for metallic materials as they show good contrast in specular reflections which is why you see a lot of jewelry/watches photographed with this type of environment. You could also look for an environment with a strong directional light like the sun to help highlight surface detail. This will work best if the surface detail is at a raking angle to the lightsource.
Hope this helps get you a step closer. Let me know if you have more questions.
Hi @PatrickRyan,
first and foremost, thank you for your great and detailed answer!
I think it will be a great help for other people out there struggling with similar problems.
I have two following questions:
In my real application the user can load around 400 different meshes on demand, that are slightly different, but those differences are important.
When using those environment textures, is it necessary to perform this process for every individual mesh (I guess this would take forever using the sandbox)?
Or can I just do it once and use the same environment texture for all my meshes?
Second:
Unfortunately, I have my colored meshes only in the .drc format (and also in the non-standardized colored STL format, but I would like to avoid using that).
I couldn’t find out how to convert my Draco meshes to a GLTF scene (?).
I tried searching through the Draco C++ API (which I use to generate the drc. meshes), but wasn’t successful.
If you know any way to export a draco::EncoderBuffer to a GLTF file, feel free to tell me.
Edit: I think I found a way, but it requires changing my CMake configuration. I think I will only proceed with this, when I know that the first question is no obstacle.
@major-mayer, you shouldn’t need a different environment for each mesh. One should be fine. What I would suggest is to test your lighting on the trickiest of the meshes. If you find one that works for that mesh, it should work for most. And if converting your mesh to a gltf isn’t easy, you can grab any mesh that has similar material properties in terms of surface perturbation. A search on something like sketchfab could help as they have assets that can be downloaded in gltf format. Finding an asset with similar surface detail will give you an idea of what the lighting will look like.
I will say that lighting a scene is a skill that needs to be practiced. It comes down to understanding how the surface will reflect light, where to position your lights, and how the light type will describe the scene. In studio photography we think of this as “painting with light”. You start with darkness and add one light at a time until your subject is lit in a way that is both pleasing and describes the silhouette and detail well while creating a path for the eye to follow. You may not dive into your lighting that deeply, but think of it as how your surface changes as your light position changes. Even rotating your environment while looking at your mesh can give you a good idea if your surface detail is supported by the light or if the light fights against it.
Yeah I am definitely impressed now how much work goes into the correct lighting of a scene until it looks good.
Anyways, I converted my mesh to glTF, loaded it in the sandbox, added the HDR file that i linked before and then exported the environment file.
I tried to adjust my scene that it now uses PBR Material and no lights anymore (I think they are not needed, right?).
It now looks like this: https://playground.babylonjs.com/#F2CUSQ#4
A bit better I would say, but I am sure you have some ideas how to improve it.
Especially, I don’t know much about the PBR material settings, I just experimented a bit with the values of metallic and roughness.
@major-mayer, you can certainly add punctual lights like point, spot, or directional to your scene with IBL. For example, if you need to cast shadows, you will need a punctual light to cast them as we can’t cast from IBL.
For your purposes, I would probably use a metallic of 0.0, as you don’t want environment reflections interfering with understanding the model. For roughness, I would try values between 0.4 and 0.6. The lower values will have sharper specular highlights which can help describe the surface as they roll over it.
Broader specular reflections can make subtle surface perturbation a little hard to discern. So I would err toward a little sharper specularity.
