GPU Accelerated Creation of HDRCubeTexture

[kzhsw]

Playground: Babylon.js Playground
Version: 7.24.0
Engine: WebGL2

1. Problem

Loading panoramic cube textures is dominated by CPU work on the main thread. Empirical Chrome profiling on the playground above:

Scenario	Dominant CPU cost	Share of load time
HDR → 1K cubemap (WebGL2)	ConvertCubeMapToSphericalPolynomial	~54%
HDR → 1K cubemap (WebGL2)	CreateCubemapTexture (panorama reprojection)	~30%
HDR → 1K cubemap (WebGPU)	ConvertCubeMapToSphericalPolynomial	~45%
HDR → 1K cubemap (WebGPU)	ConvertPanoramaToCubemap	~27%

Observed wall-clock: ~5 s for 1K HDR, multiple minutes for 2K HDR. Both dominant functions are pure arithmetic over image pixels — ideal for GPU. Everything runs synchronously on the main thread and stalls the UI.

The same three hot paths affect every cube texture class that ingests a panoramic source, not just HDR. All of them inherit EnvCubeTexture._loadTexture() (packages/dev/core/src/Materials/Textures/envCubeTexture.ts:207-349) and share the pipeline.

WebGL2

WebGPU

The same thing happen to CreateCubemapTexture, which takes ~30% of the cpu time, seems also could be gpu accelerated.
In case of webgpu, ConvertPanoramaToCubemap took ~27%, and ConvertCubeMapToSphericalPolynomial took 45%.

2. Scope: affected cube texture types

Anything inheriting EnvCubeTexture or lazily computing BaseTexture.sphericalPolynomial from a cube texture benefits:

Class	Source	Reprojection cost	Harmonics cost	File
HDRCubeTexture	RGBE .hdr	Yes	Yes	Materials/Textures/hdrCubeTexture.ts
EquiRectangularCubeTexture	PNG/JPG panorama	Yes	Yes	Materials/Textures/equiRectangularCubeTexture.ts
EXRCubeTexture	.exr panorama	Yes	Yes	Materials/Textures/exrCubeTexture.ts
CubeTexture (6-face or .env/.dds with lazy SH)	Existing cubemap	N/A	Yes (on first sphericalPolynomial access)	Materials/Textures/cubeTexture.ts
Any BaseTexture.sphericalPolynomial lazy getter on a cube	Existing cubemap	N/A	Yes	Materials/Textures/baseTexture.polynomial.ts

3. Current CPU pipeline (bottlenecks)

fetch(url) ─► decode (HDR/EXR/PNG) ─► Float32Array panorama
   │
   ▼ (CPU)  PanoramaToCubeMapTools.ConvertPanoramaToCubemap()
   │         panoramaToCubemap.ts:95-228 — 6 × size² trig + sample loops
   ▼
   CubeMapInfo  (six Float32Array faces on CPU)
   │
   ▼ (CPU)  CubeMapToSphericalPolynomialTools.ConvertCubeMapToSphericalPolynomial()
   │         cubemapToSphericalPolynomial.ts:203-328 — 6 × size² SH integration
   ▼
   engine.createRawCubeTextureFromUrl(..., faces)  — upload 6 faces

Key functions and line ranges already confirmed:

• PanoramaToCubeMapTools.ConvertPanoramaToCubemap — Misc/HighDynamicRange/panoramaToCubemap.ts:95-228. Inner loop: CalcProjectionSpherical does atan2+acos+two clamps per sample. Supersampling multiplies work by samples².
• CubeMapToSphericalPolynomialTools.ConvertCubeMapToSphericalPolynomial — Misc/HighDynamicRange/cubemapToSphericalPolynomial.ts:203-328. 6 × N² pixels × ~20 float ops accumulating 9 SH coefficients.
• CubeMapToSphericalPolynomialTools.ConvertCubeMapTextureToSphericalPolynomial — same file, lines 64-132. Path used when the texture is already on GPU; issues 6 × texture.readPixels() then runs the same CPU integration.
• EnvCubeTexture._loadTexture gamma/half-float/byte fallback loop — envCubeTexture.ts:247-295. Per-face Math.pow loop when gammaSpace=true or no float support.

4. Proposed GPU pipeline

Mirror the two prototypes already in the repo root:

• Single-Draw Equirectangular to Cubemap Converter — WebGL2 panorama → cube reprojection. Pre-bakes face direction vectors into vertex attributes, renders all 6 faces into a single 3×2 atlas FBO in one draw call, then copyTexSubImage2Ds each atlas tile into a cubemap face and generateMipmaps on-GPU. No branching in the fragment shader (just atan/asin + one panorama sample).
• GPU SphericalHarmonics Creation — WebGL2 SH integration from a cubemap. Renders 9 MRT integration textures (4 + 4 + 1) sized N × 6N, face-reduces into N × N, then pyramid-reduces to 1 × 1. Packs 28 floats as 7×4 RGBA8 via floatBitsToUint and reads back with readPixels(..., UNSIGNED_BYTE, ...) — a single ~112-byte sync readback instead of 6 float face reads.

Target pipeline:

fetch(url) ─► decode (HDR/EXR/PNG) ─► Float32Array panorama
   │
   ▼ upload as TEXTURE_2D (RGBA32F / RGBA16F / RGBA8 depending on source)
   │
   ▼ GPU pass 1: PanoramaToCubeGPU
   │    Mirrors the v3 HTML: 6 faces → 3×2 atlas in one draw,
   │    copyTexSubImage2D into TEXTURE_CUBE_MAP, generateMipmap.
   │
   ▼ internal cube texture (float or half-float) ready for the material
   │
   ▼ GPU pass 2 (optional, when generateHarmonics): GPUSphericalHarmonics
   │    Integrates against the cubemap, packs 28 floats as RGBA8,
   │    reads back 112 bytes → SphericalHarmonics → SphericalPolynomial.
   │
   ▼ Assign to InternalTexture._sphericalPolynomial; fire onLoad.

Benefits directly addressed by the profiling data:

• Removes the ~30% panorama-reprojection cost (moves to GPU, runs in milliseconds for 1K/2K sources).
• Removes the ~54% SH integration cost (GPU MRT + tree reduction).
• Removes the 6× readPixels(FLOAT) stall in ConvertCubeMapTextureToSphericalPolynomial — replaced with one readPixels(UNSIGNED_BYTE, 7×4) call. This is the biggest win on drivers where float readback is slow (notably many Windows/Chromium combinations; the prototype comment flags this).
• Removes the six CPU-side Float32Array allocations (one per face at size²×3 floats) in CreateCubemapTexture. For size=1024 that’s 6 × 12 MB = 72 MB of main-thread allocation per load today.

5. Risks and open questions

• MRT capability on mobile. Some WebGL2 implementations cap MAX_DRAW_BUFFERS at 4 (which is exactly what we need). WebGPU always supports 4 color attachments. Fallback to CPU path is safe.
• Float vs half-float render targets on iOS WebGL2. SH accumulation precision demands RGBA32F. If EXT_color_buffer_float is unavailable, run a two-pass split (two RGBA16F accumulations + CPU combine) or fall back to CPU. Needs measurement.
• Determinism. GPU fp math is not bit-identical across vendors. Acceptable for SH (reconstruction is low-frequency anyway); visualization tests already have a tolerance. Document explicitly.
• Panorama orientation convention. The v3 HTML uses a different face basis than Babylon’s FACE_FRONT/BACK/… in panoramaToCubemap.ts. Must verify the mapping end-to-end against the reference images before flipping the default.
• EnvCubeTexture’s gamma/half-float/byte fallback loop (envCubeTexture.ts:247-295) is separate from the two hot paths — it still runs on CPU for legacy types. Moving it to GPU is a follow-up; it’s not in the profiling top-3.
• Cache / reuse. A single GPUSphericalHarmonics instance is expensive to compile. Add a per-engine singleton cached on engine._gpuSphericalHarmonics so repeated loads don’t re-compile shaders.

[Tricotou]

I don’t know if the solution is going GPU accelerated, but still I upvote this topic, and I’ll follow it.
On my apps, the only thing which can really freeze the main loop if this CubeMap loading.

For example on my Hovercraft game, I manage to load everything smoothly expect the sky texture which freezes the loading for a few seconds each time…

Also, I remember a conversation on the forum about the fact that JS is single threaded, which is the main problem for loading textures without blocking the main event loop… I hope one day we will have a fix for this

[kzhsw]

For the ConvertPanoramaToCubemap part, seems three.js have a gpu accelerated impl here. It should take at most 6 draw calls and 6 readPixels (or 1 draw call with multirendertarget). Also, since the algorithm of ConvertCubeMapToSphericalPolynomial does not use too much branching, looping, or read-after-write, it could be also rewritten in shaders.
To speedup loading cubemaps, babylon.js’s internal env format could be an option.
It’s very hard to use multiple thread in js, using it would suffer from limitations of browser, or copy the data to and from threads every time data needs to be passed cross thread.

[Deltakosh]

This is something that @sebavan may found interesting!!

[peterimg]

Hi @sebavan

Did you ever found a solution to this? This is super slow. Those 2 to be precise, but one contains callback to other:

[sebavan]

I tend to never do it at runtime or never on large texture.

Unfortunately in your case, the conversion to cubemap would not be enough of a gain. Can you share the usecase you have to generatre them at run time to see if there would be another solution ?

[peterimg]

Hi @sebavan

My texture is 1k. 1024x512px so bare minimum from polyheaven. That’s why I’m surprised by the impact. And the sky looks pixelated, so it’s not that great either.

I use it like that:

hdri = new HDRITexture(…)
scene.createDefaultEnviroment({enviromentTexture: hdri})
scene.createDefaultSkybox(scene.enviromentTexture)

So nothing fancy.

[sebavan]

Why not converting to env file for the pbr part ?

[kzhsw]

Updated the main post referencing the 2 GPU accelerated prototypes