hi all
Im wondering , so I plan to have some fully baked unlit assets mixed with assets that need a color change but I still wanted baked highlights and shadows.
Instead of lightmaps , because of the mix of fully unlit , I was thinking to perhaps make those materials have a 50% gray diffuse and still do a full bake.
then create a version of unlit shader that uses a color and a blend mode of overlay using the baked texture.
This is my first brain dump and it might not be the best solution , has anyone done something like this before?
even with lit shaders lightmaps have to either be “highlight” or “shadow” maps , they are not both. Overlay blend mode is a mixture of two blend modes using the 50% gray although in my mind somethnig is telling me this will have unwanted darkening in the center values
perhaps a modified concept of overlay? where 50% gray contributes nothing , like the zero point.
Anyway , i need to figure this out soon and will report back here with findings myself , unless someone else provides a knowledgeable solution before i tinker with it.
cheers
here is a quick concept in NME :
Im not a shader expert , i did dabble with them in unity many years back , that is about it.
So here , i used add and multiply then lerped that .
To try make a center zero point ( 50% ) I just scaled that result and scaled the original color and added them back together. I know not correct, I need some maths in there somewhere haha , I will keep at it.
I need something to push the high and lows back up , so i guess the thing to figure out is making 50% gray the zero point…
hi all
ok here is a better solution , firstly making the blends use the image for both imputs , scaling it by the scalar image ( 50% gray with baked lighting ) then using add blend.
This actually works very well , im not clued up enough to know if such a shading operation is expensive?
a small cavaet i noticed is that black makes the texture completely black , i didnt like this so used the remap node to map the range 0-1 to 0.1-1 ( mapping to a low of ~0.16 looks a little better … but im experimenting , i tried that because the jump from 0 to 10% in the scalar image seems much more drastic than the other 10% differences )
so now i will generate the code and when I get to it in my project I will have to see how to use it. I need this logic for more than one material in the scene , for example the walls and the floors.
So then I can change the floor texture and have the baked lighting aplied to it but still a shadeless material to best match the fully baked shadless assets on the floor which dont require configuration
Same for walls , except with walls its just a color change , ( there are over 2000 wall colors )
v3 :
I added separate scale nodes to the add and multiply to test something , I noticed as soon as you change the source image to something brighter … it blows out very badly in the highlights.
I will still compare it to actual bake results a little later in the dev but for now I tried several and It has me thinking the current add node solution is too primitive in that it doesnt factor in the actual pixel value being added. For example in the latest test I took a very bright texture ,
and saw I had to scale the add result down to almost 15%.
With 100% this is the blow out :
with 15% …
with a darker source , using 100% scale on the add node has no blow out :
maybe the same is true with actual bake results but im almost sure the add node as is needs some scaling based on the pixel value ( luminosity ) ??
to be continued
@shaderbytes, if I were to approach a shader like this, I would do it with just a simple lerp. Us your lightmap as the mask between your base color texture and your base color texture multiplied by your shadow color.
This is a super simple example, but you can see that we have full control over the color of the material as well as the color of the shadow. If you think of a lit surface (that is correctly exposed in the camera) the brightest point of the light would show you the exact material surface color. If you shift exposure on the camera, you could blow out the surface color, but for the most part, you could assume that the brightest part of the light on the surface would show the base color. Specular highlights would be different, but the base color would be as if it were unlit.
This is the reason for lerping between the base color and base color multiplied with the shadow color. This technique could be greatly expanded upon, but the basic approach would be the same. I hope this unblocks you, but let me know if you have more questions.
1 Like
Thanks patrick
I did mention I want both highlights and shadows. Hence the whole 50% gray thing and mentioning the overlay blend mode as example , which is a combination of screen blend mode for above 50% gray and multiply for under.
Your suggestion is only doing multiply now and avoiding highlights but I do want to handle highlights because as mentioned im setting up a scene with fully baked unlit textures.
So those baked assets would have shadows and highlights. The brightest parts are not base color.
Then I need some asset materials to be configurable but I still want them to best match the fully baked unlit assets.
So the plan was to make those materials 50% gray when baking ( full bake as well ) so you will have shadows and highlights. 50% gray is now the zero point.
My second and third link to NME actually have this concept working pretty well. I was just pondering now about getting even more refined acuracy in that kind of set up and that I need to compare it to actual baked results at this point.
here is a render of the scene as example , so things like the vanity and basket etc would be fully baked and not require configuration.
Things like the floor and walls , need diffuse configuration but I still want the baked lighting, notice the floor has shadow and highlights from the light coming through the window. The wall also has shadows and highlights from the ceiling lights.
( Im fully aware baked highlights are not dynamic, so these are not traditional lightmaps, im using unlit - so highlights are baked and I dont mind the highlight angles not changing. )
here I baked the maps so I have the toys to play with properly now ,
this is a sample of the floor fully baked :
this is the floor with diffuse base :
this is the floor 50% gray color , fully baked :
So the goal is to take diffuse base and 50% gray baked textures and get them looking like full texture bake 
( small side note , the baked images do have post processing on them , the base diffuse does not. )
here is a new NME link with these being tested , I stripped it done to just blends and crude add :
Ideally it is a working concept but what I need to refine is :
-
the highlight is too strong
-
probably related , im using a single channel from the 50% image as the scalar which I imagine is incorrect , I tried using “level” but it gave bizarre results … this should be using the “value” of the combined pixels ( luminosity ) ?
-
There is no color tinting. The baked image has some tinting from the lights , the 50% gray image is not truely grayscale , it has color values , but the simple blending im doing doesnt seem to bring that mix into the result.
lastly i will do a few other tests with lighter and darker base diffuse textures and compare to bakes again so Im not just solving the problem in one corner.
@shaderbytes, I apologize for missing the part about wanting to control the highlights. In that case I would make the shader similar to something like this version.
For some reason when saving this version to the snippet server, the embedded textures were removed, so you will need to upload or link them again. But what this one is doing is doing an interpolation between the base color and the base color plus light color based on the greyscale map you provide. Having the light as a separate color allows you to dial in the power of the light added to the base color texture as well as tint the light if needed.
You will see that in both the path of highlights and the path of shadows, there are SmoothStep nodes modifying the lightmap. What this is doing is manipulating the gradient for controlling both shadow and highlight shapes. Moving this range around will change the look of both the highlights and shadows. There is also a shadow power node which will lighten or darken the shadows overall so you can make them more transparent to the base color if needed. This should give you enough control to make the final render look exactly how you want.
Additionally, this approach gives you a little more control over the lightmap that you bake. Right now the lightmap is based around your 50% mark of add/subtract which may be modifying your lightmap. If you bake a full range map of black to white, you will be able to pack in more gradient to your lightmap. Since this is basically a mask, having the gamut of the mask contain both black and white gives you more data to use when dialing in which parts of the lightmap to assign to highlight and shadow. Right now, your gradient is compressed, which is showing a little more banding due to compression.
Additionally, note that I am converting your base color image to linear as well as the light color to ensure they are in the correct space, but since your mask is just a greyscale image, baking it out in linear color space will work. Otherwise make sure that the mask texture is converted to linear as well.
nice , let me have a look at what you created here thanks
I flipped the UV on the X now since it was driving me mad comparing the preview to my bake image and they are flipped. Anyway … why is the X flipped if I may ask and you perhaps know? I mean even the thumb of the texture node in the NME viewport displays the image correctly left to right. Just the preview is flipped.
Is it something with how babylon handles UV and it will always be a case of having to flip the X for NME shaders or are the UV’s simply flipped in the model used in the preview.
I guess I will find out the answer to this down the line but thought to ask anyway.
This has to do with NME not handling Texture.invertY in the same way as the engine. You can correct it with a negative scale on V for the texture and rotating the camera to the correct angle. The preview window is just another Babylon scene with an arcRotate camera set to a specific alpha and beta value. We have been talking about ways to make this more seamless, so improvements are coming.
Why the V when its the horizontal that is flipped? , I fixed it flipping the U ( x ) , eg :
is this wrong?
@shaderbytes, this isn’t wrong, but it is the opposite of what Babylon is doing to a texture. Consider this image:
This is how a texture is loaded into NME by default. What Babylon is doing is setting a flag to invertY on the Texture class when the texture is loaded based on the value of the parameter. In the case of this image, it does appear that the letters are inverted on X because they read backwards. However, if you put a negative scale on Y instead of X and rotate the camera:
You can see I added a negative on the Scale V parameter before rotating the camera 180 degrees. This then works in conjunction with the invertY parameter on the Texture class. If you are inverting X in the shader and Y in Texture, you will need to manage both parameters. However if you have the Texture class and shader both handling the inversion on Y, then setting in either place will generate the correct orientation for the texture. I hope this makes sense, but feel free to ping back with more questions.
many thanks for the explanation. That is something that could easily snag a person without such knowledge ( like it did to me )
Anyway not to worry more about that , was just curious. still testing my baked stuff.
For interest sake I tested exporting unlit materials from blender ( using background shader in blender ) for the baked assets, I noticed they still get a PBR material in Babylon , i believe its due to it being the default for GLTF/GLB, this means you can actually still use the built in lightmap (or shadow map ) feature of the Babylon PBRMaterial on unlit , I tested it and it works but obviously now its only doing either highlights or shadows again, it still pretty cool. I do however wonder if there is any unnecessary overhead with having a PBRMaterial for a unlit material? Would you personally change this to a custom shader with less overhead? Or are the gains negligible?
I also tried to export unit with alpha ( still background shader with texture alpha plugged into “strength” input) but it didnt work as expected off the bat on just importing, without having to then still manually change the material properties like transparency mode or having tell the material to use alpha from diffuse etc. but now im going all over the place in this thread.
thanks for your time.
@shaderbytes, the reason that glTF files come into Babylon with PBRMaterials assigned is due to the glTF loader creating PBRMaterials due to the glTF specification for material settings. When you export your materials from Blender as unlit using the background node in your material, the PBRMaterial that is created has the unlit parameter enabled. So even though the material uses the PBRMaterial class, the unlit parameter will bypass any lighting calculations. So, most of the expense of the material will be removed from the material.
I think, for you question about using a node material versus an unlit PBRMaterial, I would err on the side of if you need anything custom of not. The unlit PBRMaterial should be pretty light, and so may not be much heavier than a simple unlit node material. The biggest difference would be loading and parsing the node material json which will have a small cost.
I’m not sure of the process of setting up a Blender material to use unlit with alpha blend. Setting the value of strength won’t give you alpha as it’s just a multiplier on the unlit color value. I did look quickly to see if there was a quick way to do it, but I didn’t find an obvious path. If you need several things like unlit, lightmaps, and alpha, it may be worth just making a node material to meet all your needs.
1 Like
