[LITE] No supported path for GPU-generated geometry, proposal + prototype

Feature requests
,
1

Hi all,

We have had a lot of success building Manywhere on Babylon.js, and watching how fast Lite is moving we wanted to look at migrating early rather than late. We ran into one gap, and since it looked general rather than specific to us, we prototyped a fix before proposing anything.

The gap: there is no supported way to render geometry produced on the GPU. Every geometry entry point takes CPU-side typed arrays (createMeshFromData, updateMeshGeometry, resizeMeshGeometry, updateMeshPositions, and the rest), so “procedural” in Lite today means CPU-procedural. If a compute pass generates vertices, the only way to draw them is to read them back and re-upload.

There is also no user-facing compute API: 0 of 702 exported functions dispatch anything, and GPUComputePipeline and beginComputePass never appear in the public .d.ts. Compute exists internally (mipmaps, BRDF decode, thin-instance culling) but is not reachable. We checked the feature comparison, the lab scenes, and the typings before concluding this, so please correct us if we missed a route.

Core Babylon.js went through this already, in this thread: thin instances predated WebGPU and could not consume storage buffers, and support was added afterwards. We are asking for the Lite equivalent.

Why it matters

Manywhere generates terrain entirely on the GPU. A compute pass fills one bounded storage slab, chunks take slots in it, and they are drawn straight from it with no readback. A readback would stall the frame, and the design depends on terrain staying analytically queryable without waiting on the GPU. The same constraint applies to marching cubes, GPU particles with custom geometry, mesh deformation, and erosion simulation, so this is not really about our game.

What we prototyped

We implemented it on a fork rather than guessing: about 620 lines across 9 files, two commits, all 1,571 upstream unit tests still passing.

  • Writable, vertex-capable storage buffers. createStorageBuffer(engine, bytes, { writable: true, vertex: true }). These keep no CPU shadow, since there is nothing meaningful to mirror when the GPU owns the contents, so after device loss they reallocate empty and the owner refills.
  • Per-attribute vertex layout overrides. position and normal are currently hardcoded to float32x3 at stride 12, one buffer per attribute. An override lets one interleaved allocation back several attributes and lets position be float32x4, so .w can carry packed per-vertex data. Lite already renders strided buffers in gltf-interleave.ts; that machinery is just wired only to the glTF loader.
  • createMeshFromStorageBuffer, with baseVertex so several meshes share one slab. We used the draw call rather than a non-zero setVertexBuffer bind offset, because gltf-interleave.ts notes those corrupt vertex fetch on some AMD and Dawn paths.
  • A compute seam, shaped after createShaderMaterial:
const filler = createComputeShader(engine, {
  computeSource,
  uniforms: [...],
  storageBuffers: [{ name: "slab", type: "array<vec4<f32>>", writable: true }],
});
setComputeStorageBuffer(filler, "slab", slab);
await prepareComputeShader(filler);        // createComputePipelineAsync

beginComputeBatch(engine);
for (const chunk of chunks) { setComputeUniform(...); dispatchCompute(engine, filler, groups); }
endComputeBatch(engine);                    // one encoder, one submit

Storage binds as an opaque StorageBuffer, so no raw WebGPU handle crosses the public API and pillar 4d holds. Nothing in the core render path imports the module, so it tree-shakes away when unused.

Result

Four terrain chunks generated by compute into one shared slab, drawn from distinct baseVertex slots, with the colour gradient driven purely by position.w. No readback, no copy, no engine._device. 30 frames, zero validation errors, clean disposal.

gpu-terrain-native
gpu-terrain-native1280Ă—720 14.2 KB

One thing worth flagging: our first batching attempt shared a single uniform buffer, and all four chunks silently rendered as one, because queue.writeBuffer is ordered against submission rather than recording. Each dispatch now takes its own slot in a dynamic-offset ring. Any batched compute API has to solve this.

What we would like to agree before opening a PR

  1. Is user-facing compute in scope for Lite at all? It is a new public surface, so that is your call rather than ours.
  2. Per-dispatch parameters, API or convention? Our dynamic-offset ring keeps dispatchCompute ergonomic. The alternative is a single-dispatch API where callers put per-item params in a storage buffer indexed by workgroup ID, which is what our production code does today. That is less API surface and scales better.
  3. Where should the vertex layout live? We put it on the material. It could equally hang off the mesh via the existing MeshGPU._vbLayout that the glTF interleave path already uses, which would reuse machinery instead of adding a parallel mechanism.

Happy to reshape any of it. If there is appetite we will open a PR with the lab scene, golden reference, and bundle-size ceiling that CONTRIBUTING.md requires, and we can split it so the storage-buffer change lands independently of the compute seam.

Yilmaz

1 Like
2

I like it a lot. My only ask and this is non-negotiable is to not move the current scene bundles. This is the main point of Lite (hence the name ;))

So to your first question: yes, user-facing compute is in scope. It’s worth noting the full Babylon.js engine already ships this concept (ComputeShader + StorageBuffer created with a vertex flag, so the same buffer is both a compute write target and a vertex source), so this isn’t new ground for the ecosystem: it’s about bringing that capability into Lite in a way that fits its functional, tree-shakeable style. Aligning the naming and semantics with core where it makes sense will help users move between the two.

For 2: I’d go with API calls: setComputeUniform / setComputeStorageBuffer between dispatches, mirroring ShaderMaterial (which already has setShaderStorageBuffer). It matches what our internal compute does today: one bind group per dispatch, replayed in a single pass. Dynamic-offset rings are a fine internal optimization later, but let’s keep them out of the public v1 surface.

For 3: Vertex layout → on the mesh/geometry, not the material. The material shouldn’t care where the vertices came from, and keeping it there means one material stays reusable across CPU- and GPU-sourced meshes.

One thing to keep in mind: Lite already runs compute internally (e.g. thin-instance GPU culling) with its own dispatch-batching. Ideally the public API generalizes that existing machinery rather than introducing a parallel encoder/dispatch path: that keeps the surface small and the “Lite” promise intact.

Thanks a ton for your willingness to help!!!

3

Thanks, that’s the steer we needed. All three answers are implemented. A few things came out of building it that are yours to decide, so I’d rather raise them here than bury them in a PR.

Bundle sizes first, since that was the non-negotiable. Nothing moves. We measured it as a same-machine A/B against upstream/master, both sides built from source, across 18 scenes spanning PBR, Standard, ShaderMaterial, sprites, billboards, thin-instance, VAT and glTF. Nine scenes are byte-identical, and the largest movement in either direction is 6 bytes of minifier noise, skewed negative. Every ShaderMaterial scene is exactly 0.

2, per-dispatch parameters. Done as you asked: setComputeUniform / setComputeStorageBuffer between dispatches, no ring in the public surface.

It works because each dispatchCompute is its own submission. The moment dispatches are batched into one command buffer they all observe the last uniform value written (queue.writeBuffer is ordered against submission, not recording), and a set of differently-parameterised dispatches silently collapses into N copies of the final one. We hit that while prototyping, so batching is deliberately absent from this first surface instead of shipped with the trap in it.

Our own terrain code does want the batched form eventually: one program, many parameter buffers, dispatched together on the frame’s critical path. We’d like to propose it as a follow-up, with that ordering problem solved.

3, vertex layout on the mesh. Agreed, and it landed as a split, because the layout turned out to be two things with different owners.

The mesh owns the packing: byte stride and per-attribute offset. That’s MeshGPU._vbLayout, the record the glTF interleave path already produces, reused as-is.

The material owns the format. It decides the WGSL type of input.<attribute>, for example a float32x4 position packing data in .w. That’s the shader’s own signature, and it can’t come from whichever mesh happens to be drawn.

That split is what makes your “one material reusable across CPU- and GPU-sourced meshes” hold.

One shape decision we’d like your read on. Formats are applied with a setShaderAttributeFormats(material, …) call instead of a ShaderMaterialOptions field. Carrying the field on the material cost 36 bytes in every ShaderMaterial scene, declared or not, and the tightest of those scenes have double-digit byte headroom. As a function it costs nothing when unused, and it matches setShaderTexture / setShaderStorageBuffer. We can move it back to an option if you prefer.

On generalizing the existing machinery instead of adding a parallel path. Half done, and the missing half is a byte question.

The dispatch recording is now shared. One compute-pass helper records dispatches into a pass and dedups consecutive setPipeline calls, and both the thin-instance culling batch and the public seam go through it, replacing the copy of that loop each had.

The frame encoder is not shared. dispatchCompute opens its own encoder and submits, the way the loaders, the IBL/BRDF preprocessors and the GPU picker already do. Recording into engine._currentEncoder instead would fold N dispatches into one submission, and we implemented it, but it needs renderFrame to publish whether a frame is recording. Clearing _currentEncoder after finish() costs exactly 10 bytes in every scene, including scenes that never dispatch (uniform across the five scenes we measured). Every existing scene has at least 13 bytes of headroom, so it fits today, but it permanently spends most of the margin on the four tightest ones. We left it out on that basis. It goes back in if you want it.

One thing we found that isn’t ours

While checking that the layout split didn’t regress anything, we hit what looks like a pre-existing bug, unrelated to this proposal.

The ShaderMaterial path doesn’t consult _vbLayout at all. PBR, Standard and picking all do; material/shader/ never mentions it. So an interleaved glTF mesh drawn with a ShaderMaterial is fetched at the canonical tight stride.

Repro on a clean upstream/master, using an asset already in the repo: Buffer_Interleaved_03.gltf, whose POSITION / COLOR_0 / TEXCOORD_0 share one bufferView at byteStride 28. Same mesh, same camera, same frame, only the material differs.

  • with its loaded PBR material, it renders correctly
  • with a trivial position-only ShaderMaterial, it renders nothing at all

The mesh does carry _vbLayout._p = { _stride: 28, _offset: 0 } at runtime. The shader path just doesn’t read it, so the four vertices come from the wrong byte offsets and the quad collapses.

On scope: our PR does not fix this. Our layout resolution sits behind a hook that only installs when the storage-mesh factory is in the bundle, so a glTF-plus-ShaderMaterial scene is still broken on our branch. We checked that; we didn’t assume it.

We did build the unconditional fix and confirm it renders correctly. The measured cost is about 700 bytes, confined entirely to ShaderMaterial scenes. Of the 18 scenes measured, 11 moved 0 bytes, since PBR, Standard, sprites, thin-instance and VAT never enter that path. The 7 that use ShaderMaterial moved +695 to +708 B, which puts six scenes over their ceilings by 542 to 693 B. So it can’t simply be dropped in.

A cheaper shape probably exists, and we haven’t measured it. gltf-interleave.ts already precomputes _vbKey, so it could precompute the GPUVertexBufferLayout[] alongside it and hand the shader path a ready-made array. That would keep most of the cost inside the interleave module, which only ships when a glTF has interleaved data.

Your call whether you want that fixed as part of this work, split into its own PR, or left alone, and whether the ceilings move for it.

What the PR will contain

Two commits, split as offered, so the storage-buffer half can land independently of the compute seam.

  1. GPU-resident geometry: createStorageBuffer({ writable, vertex, index }), createMeshFromStorageBuffer with baseVertex slots into a shared slab, borrowed buffers so retiring one slot doesn’t destroy the slab, and the mesh/material layout split above.
  2. The compute seam: createComputeShader / prepareComputeShader / setComputeUniform / setComputeStorageBuffer / dispatchCompute / disposeComputeShader, plus the shared compute-pass recorder.

Plus lab scene 285, with a Babylon.js reference built on core’s ComputeShader + StorageBuffer(VERTEX) instead of a CPU-generated mesh, so the parity claim is about compute output on both engines. Full-image MAD is 0.000.

We dropped a per-draw-index feature from the branch as scope creep. We can propose it separately if it’s of interest.

Yilmaz

4

Ok great! thanks again!

5

and please if that’s ok send a second PR for the ShaderMaterial support of interleaved data :slight_smile:

6

Both are up:

584 needs 583 first, since the vertex-packing support it installs is added there. It was cut from 583 and carries those commits too, so once 583 is squashed into master I will restack it rather than leave you with a branch that tries to replay them.

One correction to what I wrote above, in your favour. I quoted ~700 bytes on every ShaderMaterial scene for the interleave fix, which would have put six of them over their ceilings. That was the cost of resolving packing unconditionally in the ShaderMaterial path, and it is not what 584 does. Installing the support from the interleave module instead measures 0 bytes across all 79 scenes tagged gltf, shader or interleaved, which is every scene that can reach either half of the change: 79 of 79 byte-identical, scene258 included. The hooks’ only consumers live in the ShaderMaterial path, so a bundle without that path proves them unreachable and folds them away.

The cost appears only where both halves are present: +1678 bytes in a scene carrying interleaved glTF and a ShaderMaterial together. Today that is the new scene 286 and nothing else, which I checked rather than assumed.

About 440 of those bytes are the declared-format machinery, which interleaved geometry does not need. Splitting shader-vb into packing and formats would recover them. I left it whole rather than adding a module for a case one scene reaches, but it is a small change if you want the bytes.

Both PRs are green. 583 is rebased onto 46cc8631, and scenes 285 and 286 both sit at full-image MAD 0.000.

Yilmaz

1 Like
7

I will review them today!