Grayscale height image to (normal) bumpTexture

Demos and projects
1

I’ve been frustrated by the lack of an automatic grayscale heightmap to (normal) bumpTexture.

This TypeScript class fills the void:

https://playground.babylonjs.com/#U6UXVB

Use:

	new Filter().getNormalTextureFromHeightMap('h',hUrl,scene,Filter.BOUNDARYMODE_WRAP,Filter.BOUNDARYMODE_CLAMP)
	.then((t)=>{
	    const tex = (t as unknown as BABYLON.Texture);
	    tex.level=8;
	    tex.vScale=-1;
	    tex.vOffset=1;
	    mat.bumpTexture=tex;
	});

The “new Filter().” comes from I thought this would be useful as a generic class implementing filters for canvas.

What I really wanted was to extend RawTexture, but unsure how exactly to do that and cover any edge cases.

There is a unique algoritm for that saves about half the multiply operations. There are also options for handling edges with WRAP, CLAMP, or MIRROR and selecting the preferred gradient operator (Sobel, Scharr, or Prewitt).

Suggestions welcome.

4 Likes
2

Coooool !!! is there a GPU version ?

3

No, but it could easily be adapted to GPU. I need help getting started on GPU.

Should it be a WebGPU process to create a texture? I can write the internals of a WebGPU shader, but not sure how to adapt it to WebGL2 texture. It makes more sense as a “write texture once” instead of “store heightmap as a texture and process every frame.” The core mathematics takes 8 surrounding pixels to calculate the normal for the central single “pixel”.

The core calculation is in the heightToNormal() function.

1 Like
4

Love it, such a useful idea!

5

I found a similar, but simplified, ProceduralTexture called NormalMapProceduralTexture that implements a central difference calculation to calculate x and y gradients and the Normal from those gradients.

The document says one option is for a ProceduralTexture to run “once to create the texture which is put into cache.”

It also says “There are 2 types of procedural textures: code-only, and code that references some classic 2D images, sometimes called ‘refMaps’ or ‘sampler’ images.”

Does anyone have any examples that

  1. implement a “run once” CustomProceduralTexture,
  2. Specify an input refMap or sampler image to a CustomProceduralTexture,
  3. create a temporary and intermediate GPU memory area (e.g. image/texture) for use within the ProceduralTexture that is further processed into the final output texture, and

Can anyone explain or point to some examples, especially number 3 above.

Edit: there is also HeightToNormal block for use in Node Material. Is that usable for a one-time conversion of an input heightMap into a bumpTexture?

6

https://playground.babylonjs.com/#OYC4R8

Edit: improved version reducing texture calls when GRAYSCALE defined. Takes advantage of bilinear sampling. Also found out textureGatherOffsets (last paragraph below) requires integer offsets, so won’t work with this method of texture sampling reduction.

https://playground.babylonjs.com/#OYC4R8#1

Edit 2:
Implementation that assumes sIngle channel (GRAYSCALE) and takes advantage of bilinear sampling, reducing the texture sampling from 12 to 8. Attempt at textureGatherOffsets for use with Central Difference, but I couldn’t find the correct way to call it (“no overloaded method that matches…”), so CENTRAL still uses 4 texture samples. This one cycles quickly through PREWITT, SOBEL, SCHARR, and CENTRAL.

https://playground.babylonjs.com/#U6UXVB#3

The ProceduralTexture implements HeightMap (aka displacement) to NormalMap (aka bumpTexture) in a GLSL fragment shader using one of four methods, selected by setting defines.

HeightMap (aka displacement) to NormalMap for use as bumpTexture. Optionally select one of the four major x/y gradient operators: Central Difference, Sobel Operator, Scharr Operator, and Prewitt Operator.

The central difference algorithm is implemented as in the NormalMapProceduralTexture and also in a similar algorithm as the rest, for testing.

  • No defines: original central difference algorithm (fastest)
  • #define SOBEL: use the Sobel operator for dx/dy calculation
  • #define SCHARR: use the Scharr Operator
  • #define PREWITT: use the Prewit Operator
  • #define CENTRAL: use the central difference algorithm, in the same (slower) fashion as the Sobel, Scharr, and Prewitt.

As in NormalMapProceduralTexture, the input is assumed to be linear sRGB.

The algorithm could be made more efficient, but would require additional storage for intermediate results. Currently, every pixel from each texture access is converted from linear sRGB to Luminance. For the “fast” (original Central Difference) algorithm, there are 4 texture accesses per pixel. For the Sobel, Scharr, Prewitt, and (new) Central Difference, there are 12 access per pixel.

A faster method might be to first calculate Luminence on every pixel, reducing three-channel RGB to single-channel grayscale, storing in an intermediate texture. Then for each grayscale pixel, calculate the three-pixel kernel convolution (for Central Difference, the Luminance value is the “kernel convolution”), storing into another intermediate texture (or channel). Finally, for every pixel obtain the four values (from above, below, left, and right) and calculate dx and dy to obtain the slope and convert that to a normal.

The final four values from the intermediate texture can be obtained with (OpenGL ES 3.2 only) textureGatherOffsets:

    // untested
    // using offset order: +x,-x,+y,-y
    // vec3 offsets[4] = [vec2(texelSize,0.0),vec2(-texelSize,0.0), \
    //                    vec2(0.0,texelSize),vec2(0.0,-texelSize)];
    // vec4 values = textureGatherOffsets(baseSampler,vUV,offsets);
    // vec2 slope = vec2(values[1]-values[0],values[3]-values[2])*divisor+0.5;
    // gl_FragColor = vec4(slope, 1.0, 1.0);
1 Like
7

Nice