3d spatial audio glitches

Bugs
#1

Hello, I’m a musician and software dev trying out some different WebGL and WebAudio frameworks for future art projects. I like Babylon.js a lot but I’m getting audio glitches in the 3d spatial sound playground examples @ Play Sounds and Music - Babylon.js Documentation . I’m on a 2 year old Windows 10 gaming PC and I don’t have any trouble with other frameworks so I’m thinking it’s a bug in Babylon.js which is a bummer because the sound glitches are a deal-breaker for me. Are there known issues and workarounds for these glitches? Is there a quick fix for them? Does anyone else hear them on their systems?

Cheers,
~ Andy Fillebrown

#2

Hey Andy, can you point me to an example that works with another framework

I know that lately win10 had a lot of issues with sound
I see nothing wrong in our implementation :frowning:

#3

I reproduce the glitches here as well: Web Audio Spacialisation

You need to let the boombox rotate (keep pushing the rotation button)

#4

I think you are correct. What I heard initially at Babylon.js Playground seems to be a bad fade out in the audio sample being used, and now that I’m listening more closely to other demos and examples I’m hearing lots of glitches on every framework based on WebAudio on both my Mac and my Windows machines across all browsers, particularly when the panning node crosses the threshold between the right and left ears.

#5

Is it a random audio clipping (when it shouldn’t be clipping) noise? I’ve noticed the same thing, and I’m on OSX. It seems to happen the worst when I a game scene is loading up, and the soundtrack starts playing. If I kill the soundtrack and leave just my ambient noise layer which is really soft, I can still hear it clip. The first 30 seconds or so seem to have multiple noticeable clippings, but it eventually stabilizes as time goes on.

#6

I pinged the team working on webaudio here at Microsoft. We will see :slight_smile:
I slightly reduce the speed I update the position and orientation (you can control it with scene.audioPositioningRefreshRate which is 500ms by default now)

#7

Thanks, the new scene.audioPositioningRefreshRate property seems to reduce the number of glitches but they’re not gone. I also tried using different buffer sizes and sample rates when creating the AudioContext, but maxing out the buffer size on Chrome doesn’t help, even to 8000+ samples!

Here is an A-Frame example that works well with no glitches at normal listener movement speeds, but the audio still glitches when moving the mouse rapidly.

(continued in next post because I’m currently limited to 2 links per post…)

#8

Here are some A-Frame examples using Google’s Resonance Audio that run silky smooth no matter how fast I move the mouse!

This makes me think the issue is in the web browser’s pan nodes since Google Resonance Audio doesn’t use the pan nodes. It basically creates it’s own pan node.

[edit] If I move the mouse extremely fast I do hear small glitches in the A-Frame and Google Resonance Audio examples, too, but I don’t think people can turn their heads that fast in VR and I can just rate limit other input devices and sound source speeds to avoid the glitches. I don’t think those workarounds will work in Babylon.js, yet. I might try hacking a Google Resonance Audio node into a Babylon.js sound component to see what happens.

#9

In my case I still hear glitches without using any panning or spatial sound, when just playing static soundtrack and or ambient noise, so I don’t think panning is entirely to blame. Especially since I am using spatial audio for the SFX but once the game loads, cpu settles down, and gpu starts working harder, glitches become fewer and fewer. To me it feels like the CPU usage is directly affecting the number of glitches I hear

edit - then again, the boombox demo, which is glitching much harder than anything I’m experiencing, isn’t using much cpu when panning so IDK.

#10

Excellent idea!

#11

might try hacking a Google Resonance Audio node into a Babylon.js sound component to see what happens.

Excellent idea!

I got a Google Resonance Audio hack working and it runs smooth as glass. No glitches in the orbiting cube playground example.

I gutted a lot of the Babylon.js audio system to get it working, though, so it might not be an apples to apples A/B test against the builtin pan node.

Regardless, I’d like to use Google Resonance Audio source nodes for all my sound sources and I’d like to be able to chain up custom Web Audio nodes for real time synthesis. Can you give me any pointers on how I can add those features to Babylon.js in a way that makes sense? …hopefully in a way that can be pulled into the main Babylon.js repo or can be added on as an extension/plugin?

#12

Hello we have a doc to help you contribute:
https://doc.babylonjs.com/how_to/how_to_start

I truly appreciate your willingness to contribute back… This is pretty rare !

#13

Thanks. The “Start Contributing to Babylon.js” page will be helpful. Right now I’m wondering where to add the code needed for integrating Google’s resonance audio SDK. Does something like that belong in the BabylonJS/Extensions repository? Would it go in the main BabylonJS/Babylon.js repository? Somewhere else?

There are also some things I’d like to change in the main repository like being able to specify the sample rate and buffer size for the audio engine’s audio context. How would I do that given the audio engine is created automatically? Would I add the audio context creation options to the scene? …maybe add them as properties on the AudioEngine that would be set before the audio context is instantiated?

These are the kind of questions I have right now.

#14

Alright :slight_smile:

  1. For Google resonance audio sdk, this is definitely an extension so Extensions repository will be the right place
  2. Can you change these values afterwards or do they need to be set at creation time?
#15

Ok, thanks.


The sample rate and latency/buffer-size options are set in the AudioContext constructor. They can not be changed after the audio context is created. It’d be nice to have those options exposed by the BabylonJS engine so they can be set before the audio context is created.

#16

Makes sense then we can probably add them here:
https://doc.babylonjs.com/api/interfaces/babylon.engineoptions

If you want you can submit a PR that will add a new audioOptions field as part of this interface. We will then be able to use data from this field to initialize the audio engine

#17

Hey @andy.fillebrown and @Deltakosh -any updates related to integrating “Google Resonance Audio” as an extension? Is there a repo I can review?

#18

No, during testing I was not able to resolve the audio glitch issues to my satisfaction so I reimplemented the Google Resonance Audio first order ambisonics in Csound WASM. That implementation works without audio glitches so far but I am not planning to make a Babylon extension with it.

#19

OK, no problem. Looks like A-Frame + Three for my project in the short term then. I’m happy to offer a small financial incentive to bring Babylon + Resonance Audio forward - if that would help.

#20

FYI - GitHub - etiennepinchon/aframe-resonance: 🅰️ Google Resonance Audio for A-Frame