GPU memory leak on iOS when using HDR textures?

Bugs
, ,
1

Hello there :slight_smile:

Working on a project for a client, I struggled like hell with an issue with iOS devices. (I remember by the way I already saw @RaananW writting : “Apple…” here on the forum ^^)

Here is the bug : my app would run smooth, but on iOS after a reload (or 2 or 3, depends on the device), it would crash with an WebGL context lost error, and then nothing to do but relaunching the browser, either Safari or Chrome does this.

I was going crazy trying to find the source of the error

After hours of debugging, I came accross the solution here :

@Matteo_FF had the exact same issue, and “solved” it by getting rid of HDR texture. (Thanks for the solution by the way :heart_eyes: ).

That said, I think avoiding the use of the broken feature is not a fix, and that’s why I’m opening this issue.

A few details about this report :

  • I could not do tons of tests since I have no Apple hardware myself, I only have limited access to browserstack where I test my code remotely on real hardware. (not emulators)
  • The bug is (at least) triggering on all iOS devices I have tested so far, I.E. iPhone 13, iPhone15, iPad 5th gen, iPad 9th gen.
  • The title of this topic is pointing out GPU memory leak, also I’m not 100% sure it’s the issue, I just know by experience that this WebGL context loss, might be VRAM issue, because considering it’s after a reload, it should not be browser related, or driver related… But I might be wrong on that.
  • Here is a PG to reproduce. Depending on the iOS device, it would anyway crash after some reloads, at some point.
  • I happened to find a fix by avoiding the use of mipmap and harmonics generation. So, replacing this line :
scene.environmentTexture = new BABYLON.HDRCubeTexture(url, scene, 1024, false, true, false, true);

By this :

scene.environmentTexture = new BABYLON.HDRCubeTexture(url, scene, 1024, true, false, false, true);

seems to be fixing the bug, or at least I have no more crashes. So there must be an issue on using these params but could not yet find the time to test if it’s either the one (mipmap) or the other (harmonics) or both combined.

2

The ibl texture itself assuming full float requires:

w * h * numberOfFaces * rgbaSize * perPixelBytes * added size of mip maps

1024 * 1024 * 6 * 4 * 4 * 1.33 = 133882183 e.g. 133 Mb

And we need 2 at peak due to prefiltering from source into an equivalent cubeTexture so about 260Mb per run which would probably fail pretty fast if not recollected fast enough :slight_smile:

I am seeing with spector the call to delete but it could potentially be a Safari issue to not “GC” fast enough which I kind of doubt or we would see even more issues ?

image
image652×517 21.8 KB

As you can see 2 textures have been deleted during a play button click: the HDR and the pbr brdf texture that we can see in the inspector

image
image455×122 8.47 KB

Tracking down count of createTexture vs deleteTexture leads as well to 0.

We can see on a replay:

image
image474×229 11 KB

and

image
image438×223 10.2 KB

Basically, we delete id 2 and 3 from the previous run, then create 4 and 5 which are the brdf source texture and the hdr source texture.

We then create an additional 6 and 7 to hold the rgbd decoded brdf texture and the prefiltered cube texture.

Finally, we delete back both sources as we go (4 and 5) so we only keep what we need and recollect on play.

Not only the memory usage is insane but the processing here is crazy for a phone too :slight_smile: Creating the harmonics is done on the CPU and we need a huge amount of it due to the heavy none SIMD maths in place (an acceptable size is usually 256 and not 1024 in this case). Also, the prefiltering shader is running number of faces * number of mips = 6 * 10 = 60 times and includes a per pixel loop of:
image

image
image1095×524 78 KB

which includes heavy maths and texture sampling:

image
image787×273 38.6 KB

I am actually surprised your phone is able to run it. On mine, the context was lost after the first try. The fact that it does not work on both Safari and Chrome is because they are the exact same on ios and webkit is now relying on angle so in the end they would probably anyway suffer from the same flow.

I am really not sure it is memory related but be probably more be linked to the heavy process at play where Safari might try to protect the users by killing WebGL. This tends to be proven by disabling the prefiltering and generation of harmonics.

You could simulate the same memory usage by creating a second texture of the same type/size (different url to bypass the cache) and notice the phone not crashing: https://playground.babylonjs.com/#4U4QH9#141

Keeping harmonics only seems to be ok too:

This would suggest the heavy shader processing is leading to the context lost here explaining why you can not simply recover from it.

Sorry for the long post but I hope it helps understanding the limitation.

3 Likes
4

Thanks for your complete answer, having a closer look ASAP :slight_smile:

1 Like
5

Hello @sebavan , coming back on your long answer :slight_smile:

Ok. To be honest on my “big” app it was 512. But a lot of other textures are being used as well. Here I have put 1024 as an example to trigger the context lost quicker in my almost “empty” PG. But I did have the problem with 512 too …

Oh really ? The 1024 is loading fine on my phone (Pixel 6A, 2022) even if it does take a while (a few seconds) to generate, I admit…

Ok got it, but what I’m trying to understand, is how it’s possible that it would load fine the first or second time, and then after the 3rd or 4th page reload, it would crash (since, appart from what I supposed to be a memory leak, there should be “nothing more” to be loaded in memory on page reload)

Or maybe your explanation is about a context loss after page reload, in that case I might have missed something, sorry ^^

No worries, it’s fine ! Thanks for the time !