Put monochromatic image into another image channel (r/g/b)

Questions
1

Hi guys

What is the proper way (if there is one) to merge 3 monochromatic images into one RGB image?

Basically, let’s say user uploads the image, I take that image and put it in the green channel.
I am thinking to use some canvas API manipulation, but I still don’t know if that’s possible. I could use DynamicTexture for this approach maybe.

Also, I played with node material editor. And basically I need something like this.

https://nme.babylonjs.com/#Y9TZ3C

But I am not sure how to use this properly. User will upload image on run-time, and I am not sure how to provide the texture block with the uploaded texture from user.

Once the images are merged, I need to set that image as an input to metallicTexture (basically, I am compositing ORM texture within the app on run-time). And it probably will have to be saved as a png or jpg image somewhere on storage (I should be able to draw this newly created texture on canvas, from where I could use toDataURL and store it somewhere)

I would appreciate any help and ideas on this.

Thank you guys.

2

nme is definitely the way to go :slight_smile:
you can then get any block and affect them new values

Example:
https://www.babylonjs-playground.com/#63DUHB

4

Indeed, using the NME is a great way to do.

Even greater would be to be able to save the output of the NME preview by providing a width/height for the generated file. That way, you could use the post process module of the NME as a 2D image tool to generate pictures :slight_smile: Maybe something for @msDestiny14?

6

Hello again. This thing is back on my task list again. And I want to figure it out. But to be honest, I am not sure how to achieve it.

So I will explain it a bit further.

I created a platform where you can upload your own model, and you can choose if you want to work with StandardMaterials or PBR Materials. So basically each material on the model can be one of two.

So in the case of PBR Materials you can work with the inputs (something similar as on Sketchfab). So you can add Albedo texture, Bump…etc. That part is simple enough as I only need to add the texture path to the proper inputs. But in the case of Roughness that’s not possible. So as mentioned above, I want to somehow create ORM texture.

There are several issues (except the one where I don’t know how to do it xD).

image
image1197×626 54.4 KB

You can see on image that I have inputs for all of those textures, which should affect the model in real-time (some delay is okay, but effectively when user uploads a textures, the model should be affected).

You touched on the NME here, and that sounds like good idea, but I am not sure how to merge these textures into one image, and then connect it to the metallicTexture channel of the PBR Material.

Even if I do that, how to make it somewhat real-time? For example, if I input only AO map, what should I do with the roughness and metallic (just leave them as black?). On any change on the maps composition from the user, I need to update the final ORM map. If the user hits save after he is satisfied, then the map is saved as jpg/png on the storage. So basically, I am thinking that I need to somehow render and update the map on the canvas for example, and use the url data from the canvas as preview on the model, and on save button I can download and store that data into some format.

So basically, I cannot just use NME and create everything and use that shader on the model. I need to get image and use it as input into one map channel (metallicTexture) of the existing shader (PBR).

Any ideas and explanations would be helpful, as I am kinda stuck on this.

Thanks

7

Here is how you could use a NME based texture: Node Material | Babylon.js Documentation

and Node-Based Procedural Textures - YouTube

8

You could try something like that:

For the metallic input, you can either use a fixed float value or a texture, depending on the metallicSwitch value: 0 for texture and 1 for fixed value. You could do the same thing for roughness, AO, etc.

You can update a texture by providing base64 encoded data to the Texture.updateURL method if it’s easier than working with png/jpg files created by the user.

1 Like
9

Hey guys. I combined your answers, and managed to figure out approach that works for me. Thank you for your responses.

Now I have another issue, and I am not sure how to solve it.

This is the shader I am using

https://nme.babylonjs.com/#WXA9A9#2

This is the example object with the albedoTexture on image

Now, if I change metallicTexture (blue channel to assign metallicTexture) to the same texture as albedoTexture (and I removed albedo here and assigned color to see it better), you can see that texture is tiling
image

I am not sure why, and how to fix this. I get it that it has something to do with the UVs, and I tried several things to try to change the effect, but nothing helped so far.

This is part of the code that creates the nodeMaterial
image

and this is how I assign the changes image

Any help is appreciated. Thanks

10

Hard to say without a repro.

You can try to update the uScale and vScale properties of the texture and see if that changed anything.

12

https://www.babylonjs-playground.com/#UXSNJI

I tried to create playground for this. I think this replicates the issue.

If you go into ground material and see how albedo affects the material you see the result

image
image843×824 119 KB

If you check metallic or roughness you can see different result

image
image891×818 121 KB

13

The ScreenPosition block gives values between -1 and +1 for x/y, so you should convert this range to 0…1 for proper uv lookup:

https://www.babylonjs-playground.com/#UXSNJI#1

1 Like
14

Oh, that makes sense. I completely missed that. Thank you @Evgeni_Popov. I am still working out on complete solution for my needs, so there might be some more issues along the way. But so far, so good. Thank you man.

15

I have another issue, and I am not sure what is going on. I cannot replicate it in the playground, so I will just ask as a long shot that you might have some idea what is going on.

I’ve implemented shader properly, and for metallic and occlusion texture it works perfect. But for roughness texture I have issue

So basically, I have option to add roughness texture and remove it (by selecting none of course)

image
image1090×910 81 KB

This principle is used for each texture. Same thing.
So when you choose the texture, that textures is set to the Green channel of ORM map I created in shader from above. And that works okay.

But if you select “none”, I need to remove this texture from the Green channel. I’ve done this by calling

mat.getBlockByName(“G”).texture = null; (not sure if this is proper approach, I tried dispose, didn’t remove the texture from the block)

Now, this approach actually works perfectly for metallic and AO textures. Textures are removed properly and everything is okay.

But when I remove the roughnessTexture with the above approach, the cube becomes completely black as on image above, and I get following error

image

image

Hopefully you guys have some ideas on what is going on. Thanks again

16

Even if that works for metallic/AO textures, it is by some side effect, you should not handle a missing texture like this.

Indeed, the shader code is still performing some lookup in textures, but with null/empty texture objects: I don’t know how a GPU should handle this, and I’m not sure this case is really specified.

You should instead ask yourself how your shader code should handle a missing metallic/AO/roughness texture?

I guess that for AO it means a full white texture, for metallic and roughness a full black texture? If that is so, you could create a 1x1 white texture and a 1x1 black texture and connect them to the right inputs depending on the choice of the user.

Or you can update your material to use some switches to handle a no texture case (shown here only for AO):

image
image710×447 47.4 KB

https://nme.babylonjs.com/#WXA9A9#6

UseAO=1 to use the AO texture, UseAO=0 to use the AODefault value instead.

2 Likes
17

Implemented working solution. Thank you guys for your help as always <3