HSV nodes in the NodeMaterial

Right now there are ColorSplitter and ColorMerger nodes, however they work only with RGB.

Sometimes it is useful to work with HSV (for example to change the saturation while keeping the hue and value the same). So it would be great to have an HSVColorSplitter and HSVColorMerger.

Adding @PatrickRyan to the thread :slight_smile:

@Pauan I understand the ask, but I will have to think about implementation. We can’t pass anything but RGB to the frag out and most of the nodes expect RGB values in 0-1. HSV is a different scale so a split and merge node would imply nodes to manipulate those HSV values in between. If we imply an HSV channel with a 0-1 range, we could get some of our nodes to work with it, but then a miswire could lead to wildly unexpected results. It might be that we need an HSV node that has sliders for HSV that are set to alter the RGB values but that single node converts back to RGB at the end. Let me think through the UX and implications of all this. Thanks for the suggestion.

1 Like

@PatrickRyan I’m not asking for an HSV type, nor am I asking for HSV support in the fragment shader. I am only asking for an HSV splitter/merger.

The ColorSplitter splits an RGB into 3 separate floats (R, G, B). The HSVColorSplitter would behave the same way: it accepts an RGB and splits it into 3 separate floats (H, S, V). And the HSVColorMerger would accept 3 float inputs and return an RGB.

It’s identical to the current ColorSplitter/ColorMerger, except instead of splitting into R, G, B floats, it instead converts the RGB to HSV and then splits into H, S, V floats. The range of all 3 floats is 0 to 1.

Blender does have a combined “Hue Saturation Value” node, however it is implemented by converting the RGB to HSV, multiplying the H, S, V, and then combining back to RGB:

https://developer.blender.org/diffusion/B/browse/master/source/blender/nodes/shader/nodes/node_shader_hueSatVal.c$47

This would be nice to have, however it can be trivially implemented in user-land with an HSVColorSplitter and HSVColorMerger, so it’s not needed. And having HSVColorSplitter/HSVColorMerger is strictly more powerful.

Also note that in addition to the “Hue Saturation Value” node, Blender also has an HSV Splitter and HSV Combiner nodes.

@Pauan, I understand the ask, but the UX isn’t straight forward and there are some things we need to define. Since you are asking for a splitter and a merger, the assumption is that you want to be able to place nodes between the split and the merge. If we are mapping hsv as a normalized Vector3, then our other nodes will work, but we are switching color space in the middle. If you used your hsv splitter and wired one output into the frag out (since the frag out can take a float) you are in essence representing an hsv value as a gamma space RGB color.

We also would likely assume gamma space RGB coming into the hsv node, but we have the ability to change a color or texture to linear space right on the node. So if you put your hsv nodes before the base color input of our PBRMetallicRoughness node, the hsv merger would also need the ability to convert back to linear space as well.

Basically, we already have some issues understanding if values are gamma space or linear space just by looking at the graph and now we would be adding another color space to the mix. This may mean we need to add some iconography to the graph to start being explicit about what color space you are in at any point in the graph so that users don’t fall into traps. And all of this would need to be designed with accessible controls as well.

That said, I am not saying that we won’t add the nodes because there is value in being able to procedurally change a color through hsv, it’s just that I have to look at the holistic design of the tool and account for any edge cases or pitfalls before we add them in.

2 Likes

There isn’t anything special about HSV in this regard… you would have the same exact problem with the regular ColorSplitter.

And the problem in that case is not caused by the HSVColorSplitter, since it just returns a float. So any solution would have to deal with floats (in general), not HSVColorSplitter specifically.

Ideally Babylon would consistently use linear space colors everywhere. The only time it needs gamma space is when importing a texture (which will convert from gamma to linear) and when outputting to the fragment shader (which will convert from linear to gamma).

But that’s a separate topic, since that also applies to the regular ColorSplitter, it’s not specific to the HSVColorSplitter. HSVColorSplitter should just behave the same as ColorSplitter.

I agree that the current color space situation in Babylon is very confusing, however HSV is not really a different color space, it’s just a different way of viewing RGB.

From the user’s perspective, they’re just dealing with 3 floats. It doesn’t matter whether those floats are (R, G, B) or (H, S, V), all of the color space issues are exactly the same.

And the user never deals with HSV, they deal exclusively with RGB and floats. There is no separate HSV type, so it’s not adding a new color space.

That’s a great idea, but that’s completely separate to this feature request.

All of the issues you brought up already exist with the regular ColorSplitter, HSVColorSplitter doesn’t make things any worse. All of these questions are solved simply by making HSVColorSplitter behave the same as ColorSplitter.

So it seems very strange to me that you’re essentially saying “we can’t implement HSVColorSplitter until we fix these other unrelated problems which already exist with the regular ColorSplitter”.

I fully support improving the color space management in Babylon, but that can (and should) be done separately from HSVColorSplitter, because it affects everything (including ColorSplitter).

This feature request is extremely small in scope, it’s just asking for something which is identical to ColorSplitter except it outputs (H, S, V) floats instead of (R, G, B) floats.

@Pauan, to say that doing a conversion of a color from one space to another, and in this case we are talking about non-absolute color spaces which are a different way to map RGB but retain the same color, is the same as exposing one channel of a color3 as a float is an oversimplification of the problem. Add to this that the conversion to hsv includes if statements in the conversion and we have a different problem all together as we need to optimize the glsl derived from the node material for the user. Below is the actual conversion from rgb to hsv.

We have other nodes that require if statements in the shader, but we are very hesitant to just add more without a full exploration into the implications on the engine, it’s ability to render to low-end devices, and what it means for backwards compatibility. Because once we add something to the engine, we are promising to support it from then on.

So even when a feature seems relatively small, we have to do our due diligence to understand the impact of the feature. And acknowledging that we already have some UX challenges around understanding color space in NME so we should just add more complexity which could ultimately make the problem tougher to solve is not the way we approach developing the engine. We have these types of discussion every day and no feature is put in without fully vetting what it means.

However, NME is a powerful tool and we have features that will allow you to get what you need right away by creating custom nodes. You can set up an rgb to hsv conversion or any other repetitive task in NME and save it out as a custom node. I even did the first one for you which converts rgb to hsv linked below. Many of our nodes are just shortcuts for adding multiple nodes, the one minus node being the most obvious, and this conversion node would be the same thing.

To use this node, you simply click on the add node button at the top of the node list:

Then point to the custom node json and it will be added to the custom nodes list. More importantly, your custom nodes will remain in your custom nodes list so you don’t need to reload them every time you want to use them. The node will come in as a collapsed frame but you can always expand the frame to make customizations if you need. And the frame allows for custom labels so you can understand what is needed for input and what you have in the output.

This will help you get started for now while we run the feature through our process.

rgb2hsv.zip (2.0 KB)

2 Likes

It should be possible to improve the implementation later, since it won’t change the behavior.

Also note that there are GPU-friendly algorithms for HSV:

That’s fair, however this feature is very common and has well defined behavior (in Unreal, Blender, Unity, and even BabylonJS).

I’m not suggesting that you shouldn’t explore the implications, just that the problems you brought up are not specific at all to HSV, and are not made worse by HSV.

Thanks, that saves me the time of creating my own nodes.

Just some more notes: HSV is quite popular, fast, and easy to understand, but it has major problems.

In order to fix those problems, various other color spaces have been created, such as CIELAB, HSLuv, etc.

But those color spaces have their own issues, and they’re rather slow and complicated.

So the best color space is a recently created Oklab. It has better perceptual uniformity than CIELAB, it has better blending than CAM16-UCS, and the code is very simple and fast.

Here is some GLSL code for doing linear RGB → Oklab (and vice versa):

// PI * 2
#define TAU 6.28318530717958647692528676655900577

float radian_to_percent(float r)
{
    // GPUs don't branch on simple ternary expressions
    return (r < 0.0 ? r + TAU : r) / TAU;
}

vec3 rgb_to_oklab(vec3 c)
{
    float m0 = 0.4122214708f * c.r + 0.5363325363f * c.g + 0.0514459929f * c.b;
    float m1 = 0.2119034982f * c.r + 0.6806995451f * c.g + 0.1073969566f * c.b;
    float m2 = 0.0883024619f * c.r + 0.2817188376f * c.g + 0.6299787005f * c.b;

    float c0 = pow(m0, 1.0 / 3.0);
    float c1 = pow(m1, 1.0 / 3.0);
    float c2 = pow(m2, 1.0 / 3.0);

    float a = 1.9779984951f * c0 - 2.4285922050f * c1 + 0.4505937099f * c2;
    float b = 0.0259040371f * c0 + 0.7827717662f * c1 - 0.8086757660f * c2;

    vec3 labResult;
    labResult.x = radian_to_percent(atan(b, a));
    labResult.y = sqrt(a * a + b * b);
    labResult.z = 0.2104542553f * c0 + 0.7936177850f * c1 - 0.0040720468f * c2;
    return labResult;
}

vec3 oklab_to_rgb(vec3 v)
{
    float h = v.x * TAU;
    float c = v.y;
    float l = clamp(v.z, 0.0, 1.0);
    float a = c * cos(h);
    float b = c * sin(h);

    float m0 = l + 0.3963377774f * a + 0.2158037573f * b;
    float m1 = l - 0.1055613458f * a - 0.0638541728f * b;
    float m2 = l - 0.0894841775f * a - 1.2914855480f * b;

    float c0 = m0 * m0 * m0;
    float c1 = m1 * m1 * m1;
    float c2 = m2 * m2 * m2;

    vec3 rgbResult;
    rgbResult.r = +4.0767416621f * c0 - 3.3077115913f * c1 + 0.2309699292f * c2;
    rgbResult.g = -1.2684380046f * c0 + 2.6097574011f * c1 - 0.3413193965f * c2;
    rgbResult.b = -0.0041960863f * c0 - 0.7034186147f * c1 + 1.7076147010f * c2;
    return rgbResult;
}

It returns a (hue, chroma, lightness) vector, with all three floats being 0 to 1. The chroma is essentially equivalent to saturation. This means Oklab can be used as a superior replacement for HSV.

The only downside is that the Oklab color space is quite far outside of RGB (the maximum saturation RGB can support is ~0.323). That means it has to clip the chroma when converting to RGB.

This isn’t a problem per se (the same thing happens with pretty much every color space), it’s just something to keep in mind. The clipping can be done with a standard clamp node, though there are much better clipping algorithms which can be used instead.

It would be very useful to have both HSVSplitter and OklabSplitter. HSV is useful for converting existing shader algorithms to Babylon, and Oklab is useful as a superior perceptually-uniform alternative to HSV.

This looks like a lot of extra instructions for the shader and I worry about perf here but on the other hand I also understand where you are coming from.

It’s not that many, it’s mostly just multiplications and additions. The lack of branching means that it should parallelize quite well. And of course you only pay the cost if you actually use it.

1 Like

To be clear, these are new nodes, they don’t change anything about the existing nodes or shaders (which still use RGB). So if you don’t use the new nodes, then there is no cost at all (either in performance or file size).

This simply gives an extra optional feature for people who want to do color manipulation at runtime (changing the hue, saturation, or value of a color or texture, which is a common thing to do).