Hi again,
Still hunting memory leaks in our app, after the fix from my previous post ( PBRCustomMaterial: ShadersStore memory leak on dispose() ) has been merged. They both come from the way the entry key in caches is computed for PBRCustomMaterial.
The problem
When you mutate a property that changes shader #defines on a PBRCustomMaterial at runtime (e.g. transparencyMode, albedoTexture, backFaceCulling…), BabylonJS recompiles the shader and adds a new entry to engine._compiledEffects. The previous entry (compiled with the old defines) is never removed. It becomes a permanently orphaned Effect.
This happens because PBRCustomMaterial builds its cache key as custompbr_<uniqueId>+<defines>. When defines change, both the old and the new key are unique by construction and will never be reused. dispose(true, true) only cleans up whichever key is current at the time of disposal => the orphaned one stays forever.
Interestingly, this also occurs with PBRMaterial, but is bounded at most one orphaned entry per define combination, since the key doesn’t include uniqueId and can be reused across instances. With PBRCustomMaterial, the uniqueId in the key guarantees a new orphan on every cycle, making the leak unbounded.
Reproduction
Playground: Babylon.js Playground
Click the button repeatedly and watch _compiledEffects grow by 1 on every click.
_compiledEffects is logged:
The pattern that leaks on each cycle:
// 1. Material is created and rendered — Effect A compiled
const mat = new BABYLON.PBRCustomMaterial("pbr", scene);
mesh.material = mat;
// 2. Define-changing property mutated — Effect B compiled, Effect A orphaned
mat.transparencyMode = BABYLON.PBRMaterial.PBRMATERIAL_ALPHABLEND;
// 3. Material disposed — Effect B removed, Effect A stays forever
mat.dispose(true, true);
Thanks for looking into this!