Audio Engine tanking FPS in iOS Safari

Bugs
1

Steps to reproduce

  • Initialize AudioEngine (either v1 or v2). Playing music is not necessary. Happens with both static and streaming sounds.
  • Click on the render canvas of BabylonJS (or some page interaction).
  • Blue audio playing icon shows up in address bar of Safari.
  • FPS drops from 60 to 30.

Environment

BabylonJS : 9.3.1

iOS / Safari : 26.3.1

Code

    // V1

    const engine = new Engine(canvas, true, { audioEngine: true }, true);

    // V2

    this.audioEngine = await CreateAudioEngineAsync();
    await this.audioEngine.unlockAsync();

Things to Note

Doesn’t happen at all with Chrome and Firefox on Android. Runs smoothly at 60 FPS on similar hardware.

And in Safari iOS, the FPS drop doesn’t happen at all when I don’t initialize my audio engine.

2

cc @RaananW

3

let’s look into that!

4

Can you check with this snapshot and let me know if it is fixed? I couldn’t reproduce this on a local device (i get 60 fps), but i think i know where it might come from:

5

Thank you so much for the fix. After testing it a couple of times, I think it is fixed now. Glad that is over with. It was literally driving me nuts trying to pinpoint the root cause of the performance drop.

6

This may be related to the above issue. Hope you don’t mind me continuing the conversation here. In iOS Safari, stuff like this doesn’t work (which works in Android).

    document.addEventListener("visibilitychange", () => {
      if (document.hidden) {
        this.audioEngine?.pauseAsync();
      } else {
        this.audioEngine?.resumeAsync();
      }
    });

Before the fix, the audio engine failed to pause (keeps playing music).

After the fix, it does pause on visibility change. But now it throws an error when resuming.

Uncaught (in promise) {“name”: “InvalidStateError”, “message”: “Failed to start the audio device”, “stack”: “”}

7

of course! let me look into that.

EDIT - when you get the chance (and when the snapshot exists) can you check if this works?

8

Hmm, it still doesn’t work. It still pauses on quit but doesn’t resume. There is no error now though. Not sure if I am getting this right, it appears both the promises aren’t resolved at all (but the audio did get paused). And if you try to quit a second time, the two pause promises got resolved at the same time (but still no resume promise).

This is quite a headache. I may just use the setVolume method if it doesn’t burn that much resources in the background.

EDIT: It appears the audio gets paused immediately on losing visibility without the need for pauseAsync().

EDIT 2: Found this.

9

can you share the playground you are testing with? i will do my best to test is fully.

And thank you Apple :slight_smile:

10

This should be good enough.

The previous forum post should also be a good point of reference.

As much as I hate to admit, I have switched to howler.js for my project for now. And even there, the problem exists. There seems to be some wonky interactions with AudioContext resuming and suspending in iOS. Safari doesn’t allow resuming without a user gesture.

11

From all of my tests - this seems like a bug in safari that we can’t overcome. Or at least I haven’t found a proper solution. With the current implementation in the draft PR i can switch between tabs and continue hearing audio, but going to the home screen and back to the scene - and then unmuting the scene again, doesn’t help with the audio. Funny enough, when navigating to a new tab after that, the audio starts playing. This is why i think it’s a safari bug and not some limitation or a serious issue in the code. I can and will continue experimenting, but i am not sure what else i can add without starting to add hacks that I don’t want to see (and not even sure they will help)

12

Thank you so much for your work!

That seems to be the case for me as well. In Safari, AudioContext.state becomes interrupted when returning to home screen and it appears calling suspend() or resume() during that state makes it throw an error.

This is my solution at the moment. Howler.ctx is just AudioContext.

    document.addEventListener("visibilitychange", () => {
      if (!Howler.ctx) return;

      if (document.hidden && Howler.ctx.state === "running")
        Howler.ctx.suspend();
      else if (Howler.ctx.state === "suspended") Howler.ctx.resume();
    });

Of course, the audio still doesn’t resume on return. But I can easily resume it by binding resume() to a button. This appears to work consistently without any funny behavior.

Hope this helps.