setDirectionalCone from a WebRTC MediaStream doesn't update the soundPanner directionalCone

Hello all, new guy here!

On v4.1.0, I’m using this code to attach a WebRTC coming from a remote stream:

// all of this code below requires user interaction (button) to activate

window.handleRemoteStreamAdded = ({stream}) => {
    // need to attach the stream to an audio object otherwise it won't play!
    var audio = new Audio();
    audio.muted = true;
    audio.srcObject = stream;

    var remoteSound = new BABYLON.Sound("RemoteSound", stream, scene, null, { streaming: true, autoplay: true, spatialSound: true });
    remoteSound.attachToMesh(box);
    remoteSound.setDirectionalCone(90, 180, 0); // sets remoteSound variable only, not updating remoteSound._soundPanner directionalCone settings
    remoteSound.setLocalDirectionToMesh(new BABYLON.Vector3(1, 0, 0));
    remoteSound.play(); // need this here otherwise remoteSound._soundPanner directionalCone settings won't be updated!
};

As you can see from the comment above, I found out that while using a remote MediaStream coming through via WebRTC:

  • the hack some people proposed few months ago (there is a post in the forum and a GitHub merge) didn’t worked for me
  • setDirectionalCone only updates the directional cone variables of the remoteSound object, but not the ones in the attached soundPanner

In order to get the soundPanner updated, remoteSound needs to get called with play, because Sound.play() forces the update of soundPanner.
Otherwise sound will come from the connected mesh as omnidirectional.

I tried what’s on master but got the same results.
Although, I didn’t understood how is it possible that in this playground https://www.babylonjs-playground.com/#1BO0YS, using static asset, it works.

I believe the reason it doesn’t work is this line:

if​ (​this​.​isPlaying​ ​&&​ ​this​.​loop​)

In the set Directional cone function. If it’s a loop it updated automatically, but if it isn’t, it won’t update until you force-play.

This is why it works in the example… Can you try with loop set to true and see if it works? Then we’ll know if that’s the issue or not.

BTW, I believe this is the best title for a bug post I have seen in a long time

I tried:

    var remoteSound = new BABYLON.Sound("remoteSound", stream, scene, null, {
    streaming: true,
    autoplay: true,
    spatialSound: true,
    loop: true
});

and it works without calling:

remoteSound.play();

Good call!

BTW, I believe this is the best title for a bug post I have seen in a long time

Lol thanks I thought it was too long

2 Likes

Ok, so we know what the issue is.

It should work even when it’s not defined as a loop. I’ll add a GitHub issue for that