Browser audio cancellation doesn't work using babylon's sound component

Hi

Our app provides two-way communication b/w browsers via WebRTC with the spatial audio of the peers playing via the Babylon Sound component.

We’re noticing that there is a lot of feedback (echo) that we get in this process.

This is in stark contrast to what happens if we use the native audio element and push it to the DOM. There is almost nil echo in that case. We cant use this because we want spatial audio, which is why we use Babylon’s Sound component.

My guess is this is a deeper issue within chrome, where it tends to ignore audio that is playing in the browser but is not part of the DOM.

Curious to know your thoughts on it.

Whooo this sounds really weird, could you create a repro in the playground ??? maybe using the webcam mike as an input ???

You can find the two implementations we use here - https://playground.babylonjs.com/#1F6J38#1
The first one (getStream1) is how we handle remote audio via Babylon’s Sound component.
The second one (getStream2) is an experiment we tried with the audio DOM component that got rid of the echo

Did you try like this? https://playground.babylonjs.com/#1F6J38#6

I guess the echo was from both the element playing it + babylon???

Thankyou for the updated example @sebavan

So the audio element wasn’t injected inside the DOM, so it wouldn’t play unless we do a
document.body.append( audioEl ) , which we didn’t in the first function (getStream).

I think I may not have described the issue well.
The issue is NOT that you hear the other peer twice
The issue is that when you talk, you hear your voice come back to you.
This happens because when I talk, my audio goes to a remote peer and comes back to me from their mic.

This should not happen because of chrome’s echo cancellation feature.

However, that specific feature does not seem to work when you inject a remote peer's audio into babylon’s sound component.

That does work if you inject the remote peer's audio to the native DOM element (we could just do this, but then we lose the spatialisation feature that babylon’s sound component provides)

We have noticed the exact same thing when using AFRAME’s Sound component, so this could be an underlying issue in WebGL.

Just to point out, we do get echo even if we use non-spacialised audio in Babylon’s Sound component.

Hope this information helps.

Is this also happening as soon as you use webaudio outside of babylon ?

Cause it might be an issue there ?

Hi @sebavan @Harkirat_Singh,

We are also facing the same issue to this use case.
We are using the spatial audio with an audio function to it and we can hear ourselves when speaking.
Is there now a solution to this?

A repro would be great cause we never got to the bottom of this one.

This isn’t an issue with Babylonjs, but with Chrome. Chrome doesn’t cancel web audio right now.

2 Likes

I encountered something that might be related to this in a demo I’ve been working on recently. Are these two links relevant?

Chrome bug: 933677 - chromium - An open-source project to help move the web forward. - Monorail
The workaround I used: javascript - Chrome won't play WebAudio getUserMedia via WebRTC/Peer.js - Stack Overflow