GPU Picker: optional picked point and normal reconstruction

Feature requests
1

Hi everyone,

I opened a PR that adds optional depth-based picked point and normal reconstruction to GPUPicker:

This is based on the idea discussed in the GPU picking point/normal example thread:

The feature adds an enableDepthPicking switch to GPUPicker. When disabled, the existing color-only picking path remains unchanged. When enabled, the default picking shader writes both the picking id and screen-space depth through a MultiRenderTarget, allowing the picker to reconstruct:

  • pickedPoint in world space
  • normal reconstructed from neighboring depth samples

The new data is available from:

  • pickAsync
  • multiPickAsync
  • boxPickAsync

For multiPickAsync, I also added optional readback tuning:

gpuPicker.multiPickAsync(points, false, {
    readbackStrategy: 0,
    maxIndividualReadbackCount: 32,
    individualReadbackAreaRatio: 16,
});

This lets callers choose between a full bounding-rectangle readback and per-point readbacks. The default auto mode uses a conservative heuristic, while rectangle and individual can be forced for profiling or platform-specific tuning.

A few notes:

  • The depth path is macro guarded, so depth rendering is not compiled into the default path unless enabled.
  • Depth picking uses WebGL2 / WebGPU / Native-capable MRT paths.
  • Custom picking materials and special material plugins can return missing or inaccurate point/normal data unless they also write the depth attachment.
  • Normals are reconstructed from depth, so they represent the visible depth surface rather than material-authored vertex normals.

I would love feedback on API shape, naming, readback heuristics, and whether the behavior fits Babylon.js expectations.

3 Likes
The pickedPoint returned by my GPUPicker is not correct
2

Looking at it!!