V2 AudioEngine Not Working With multiple spatial StaticSound

Hey @docEdub … I am having a weird issue with more than one StaticSound playing at same time and both set to spatial.

I have Setup listenerAutoUpdate options and the StaticSounds set spatialAutoUpdate.

I it works perfect with only one spatial sound.

I have a spatial Static Sound attached to the body of the car, and another attached to Rear Right Wheel.

The camera that the listener is attached to is FOLLOWING the car.

So i hear the engine and wheel skid sounds perfectly if the car is close to ZERO center scene. as i move aways from the center the WHEEL SKID marks sound drops off like it is not respecting the listerner postition or something/

IT IS SO HARD TO REPRODUCE ALL MY WORK IN THE PLAYGROUND.

But there is SOMETHING wrong with multiple static sounds that have Spatial Enabled.

If i disable one of them… Either one… The volumes works great as a move away from center… because the camera is following me and the listener is attached to the camera.

AND V1 works perfect with multiple sounds.

So @docEdub or @sebavan you guys got any clue about mutiple staticSound with spatial enabled ?

1 Like

So i forced to only be able to use the Legacy V1 Audio (I hope V1 that never goes away, because its the only AUDIO version that works for my toolkit that I been using for YEARS).

That is why i am trying to enable legacy audio on the playground for typescript:

At home it works pretty well. I do it like this:

this.audioEngine = await BABYLON.CreateAudioEngineAsync({ listenerEnabled: true, disableDefaultUI: true});
        this.audioEngine.volume = 1.0; 
        if (this.scene.activeCamera) {
            this.audioEngine.listener.attach(this.scene.activeCamera, true);
        }

const defaultOptions = {
            spatialEnabled: true,
            spatialPosition: mesh.position,
            spatialRotation: mesh.rotation,            
            spatialMinDistance: 0,
            spatialMaxDistance: options.sizeZone,
            spatialDistanceModel: "linear",
            spatialRolloffFactor: 1,
            spatialPanningModel: "HRTF",
            loop: true,
            autoplay: true,
            maxInstances: 3,
            volume: 1.0,
            preloadCount: 1
        }; 
        const sound = await BABYLON.CreateStreamingSoundAsync(nameSound, pathSound, defaultOptions);

The spatialEnabled: true, spatialDistanceModel: 'linear', and spatialPanningModel: 'HRTF' options are important, and so are this.audioEngine.listener.attach(this.scene.activeCamera, true);

This is my setup:

        /**
         * Set audio data source (BABYLON.StaticSound)
         */
        public async setAudioDataSource(source: string | ArrayBuffer): Promise<void> {
            if (this._audio != null) {
                this._audio.dispose();
                this._audio = null;
            }
            const spatialBlend: boolean = (this._spatialblend >= 0.1);
            this._initializedReadyInstance = false;
            this._lastmutedvolume = this._volume;
            // Note: spatialMinDistance must be >= 1 for well-defined WebAudio PannerNode linear rolloff behavior.
            // A value of 0 produces undefined gain at close range and causes multi-source ducking artifacts
            // because the browser's implicit limiter triggers when multiple sources sum past 0 dBFS.
            // spatialConeInnerAngle/OuterAngle 360 forces omnidirectional (no cone attenuation).
            this._audio = await TOOLKIT.AudioSource.CreateStaticSound(this._name, source,
            {
                loop: this._loop,
                volume: (this._mute === true) ? 0 : this._volume,
                autoplay: false, // Note: Never Auto Play Here 
                playbackRate: this._pitch,
                spatialEnabled: spatialBlend,
                spatialPosition: new BABYLON.Vector3(0,0,0),
                spatialAutoUpdate: true,
                spatialPanningModel: "equalpower",
                spatialDistanceModel: "linear",
                spatialMinDistance: this._mindistance,
                spatialMaxDistance: this._maxdistance,
                spatialRolloffFactor: TOOLKIT.AudioSource.DEFAULT_ROLLOFF,
                spatialConeInnerAngle: 360,
                spatialConeOuterAngle: 360
            });
            if (this._audio != null) {
                this._initializedReadyInstance = true;
                if (spatialBlend === true)
                {
                    this._audio.spatial.attach(this.transform, false, BABYLON.SpatialAudioAttachmentType.PositionAndRotation);
                    this._isAudioSpatial = true;
                }
                if (this.onReadyObservable && this.onReadyObservable.hasObservers()) {
                    this.onReadyObservable.notifyObservers(this._audio);
                }
                if (this._playonawake === true) this.play();
            }
        }

Was not setting maxInstances or preloadCount but itrs pretty close.

And the first StaticSound work great, the second StaticSound drops off

I see you have this:
spatialPosition: new BABYLON.Vector3(0,0,0),

That’s the position of the center of the scene, not the position of your object.
I think that’s the problem.
Unless all the sounds have to be in the center of the scene.

You also have the option to attach a sound to a moving object.
sound.spatial.attach(meshToAttach, true);

I hope it helps.

Yo @Dad72 .. I do:

this._audio.spatial.attach(this.transform, true, BABYLON.SpatialAudioAttachmentType.PositionAndRotation);

FYI.. here is my wrapper class i use to implment Legacy and V2 audio for my toolkit:

AudioSource.ts.zip (5.5 KB)

And it works perfect if playing ONE StaticSound with spatial enabled.

But when I play TWO spatial sounds, Like the ENGINE sound and the Wheel Skid sound, when i am far away from center scene. ONE of them drops off and you can barley here, even thou the camera and therefore the audio listener is following the car. Like the second sound it not recognizing that the audio listener is following the car an NOT at the center of scene.

If i remove one, then you here the other one at proper level, no matter where in the scene you are, again the listener is following the car.

REALLY WEIRD

I think it would be nice if you could make a reproduction on the playground with the parameters you use to try to reproduce it as simply as possible.
2 sounds that move with the same parameters you use.

1 Like

@MackeyK24 I also encountered a problem that I had not paid attention to by studying the question more closely.
And so, with that, it works perfectly well.

const defaultOptions = {
	spatialEnabled: true,
	autoUpdate: true,
	loop: true,
	autoplay: true,
	maxInstances: 3,
	volume: 1.0,
	preloadCount: 1,
	
	spatialAutoUpdate: true,
	spatialPosition: mesh.position,
	spatialRotation: mesh.rotation,            
	spatialMinDistance: 0,
	spatialMaxDistance: 50,
	spatialDistanceModel: "linear",
	spatialRolloffFactor: 1,
	spatialPanningModel: "HRTF"
}; 

const sound = await BABYLON.CreateSoundAsync("mySoundName", pathSound, defaultOptions);		
sound.spatial.maxDistance = 50;
sound.spatial.position = mesh.position;
sound.spatial.rotation = mesh.rotation;		
sound.spatial.distanceModel = "linear";
sound.spatial.panningModel = "HRTF";
sound.spatial.attach(mesh, true);

I think that’s what corrected the problem I encountered like you.
autoUpdate: true, and spatialAutoUpdate: true,
And I added the options : sound.spatial.xxx...

I think the autoUpdate corrected my problem and it should be the same for you. I wanted to share this, because it should solve your problem.

Yo @Dad72 .. Quick question, why are you setting the properties TWICE. Once via detaultOptions, then again after creating teh sound, is that required to make things work correctly ?

As of Version 8.52.0, I get an error at compile that autoUpdate does not exist:

Object literal may only specify known properties, and 'autoUpdate' does not exist in type 'Partial<IStaticSoundOptions>'.

I’m using version 8.52 and I don’t have this autoUpdate error that doesn’t exist.
And I used it 2 times to make sure it works just in case.

Once in the options of the sound and another time on the sound.spatial because with the options in the sounds it doesn’t seem to me to work properly, but with sound.spatial.xxx, there it worked as desired.

CRAP… :frowning:

That did not fix it. But it is actually working, and playing spatial sounds. But when I play more that one, in my case the Car engine sound and the wheel skid sound. The second sound (wheel skid) spatial volume drops off the furthur i am away from the center. The listener is being updated, the spatial sound is being updated. But the first sound (engine sound) plays at perfect volume no matter how far from center. If I come back closer to center, I start to hear the wheel sound.

Here is the code I create the sound with:

        public async setAudioDataSource(source: string | ArrayBuffer): Promise<void> {
            if (this._audio != null) {
                this._audio.dispose();
                this._audio = null;
            }
            const spatialBlend: boolean = (this._spatialblend >= 0.1);
            this._initializedReadyInstance = false;
            this._lastmutedvolume = this._volume;
            const defaultOptions: any = {
                loop: this._loop,
                volume: (this._mute === true) ? 0 : this._volume,
                autoplay: false, // Note: Never Auto Play Here 
                autoUpdate: true,
                playbackRate: this._pitch,
                spatialEnabled: spatialBlend,
                spatialAutoUpdate: true,
                spatialPosition: new BABYLON.Vector3(0,0,0),
                spatialRotationQuaternion: new BABYLON.Quaternion(0,0,0,1),
                spatialPanningModel: "HRTF",
                spatialDistanceModel: "linear",
                spatialMinDistance: this._mindistance,
                spatialMaxDistance: this._maxdistance,
                spatialRolloffFactor: TOOLKIT.AudioSource.DEFAULT_ROLLOFF,
                spatialConeInnerAngle: 360,
                spatialConeOuterAngle: 360
            }            
            this._audio = await TOOLKIT.AudioSource.CreateStaticSound(this._name, source, defaultOptions);
            if (this._audio != null) {
                this._initializedReadyInstance = true;
                if (spatialBlend === true && this._audio.spatial != null)
                {
                    this._audio.spatial.position = new BABYLON.Vector3(0,0,0);
                    this._audio.spatial.rotationQuaternion = new BABYLON.Quaternion(0,0,0,1);
                    this._audio.spatial.panningModel = "HRTF";
                    this._audio.spatial.distanceModel = "linear";
                    this._audio.spatial.minDistance = this._mindistance;
                    this._audio.spatial.maxDistance = this._maxdistance;
                    this._audio.spatial.rolloffFactor = TOOLKIT.AudioSource.DEFAULT_ROLLOFF;
                    this._audio.spatial.coneInnerAngle = 360;
                    this._audio.spatial.coneOuterAngle = 360;
                    this._audio.spatial.attach(this.transform, true, BABYLON.SpatialAudioAttachmentType.PositionAndRotation);
                    this._isAudioSpatial = true;
                }
                if (this.onReadyObservable && this.onReadyObservable.hasObservers()) {
                    this.onReadyObservable.notifyObservers(this._audio);
                }
                if (this._playonawake === true) this.play();
            }
        }

As you can see, the setup is pretty straght forward.

NOTE: V1 works perfect, for both spatial sounds:

        public setLegacyDataSource(source:string|ArrayBuffer|MediaStream):void {
            if (this._audio != null) {
                this._audio.dispose();
                this._audio = null;
            }
            const spatialBlend:boolean = (this._spatialblend >= 0.1);
            const htmlAudioElementRequired:boolean = (this.transform.metadata != null && this.transform.metadata.vtt != null && this.transform.metadata.vtt === true);
            this._initializedReadyInstance = false;
            this._audio = new BABYLON.Sound(this._name, source, this.scene, () => {
                this._lastmutedvolume = this._volume;
                (this._audio as BABYLON.Sound).setVolume((this._mute === true) ? 0 : this._volume);
                (this._audio as BABYLON.Sound).setPlaybackRate(this._pitch);
                (this._audio as BABYLON.Sound).setPosition(this.transform.absolutePosition.clone());
                this._initializedReadyInstance = true;
                if (spatialBlend === true && this._audio instanceof BABYLON.Sound)
                {
                    this._isAudioSpatial = true;
                    this._audio.attachToMesh(this.transform);
                }
                if (this.onReadyObservable && this.onReadyObservable.hasObservers()) {
                    this.onReadyObservable.notifyObservers(this._audio);
                }
                if (this._playonawake === true) this.play();
            }, { 
                loop: this._loop, 
                autoplay: false, // Note: Never Auto Play Here 
                refDistance: this._mindistance, 
                maxDistance: this._maxdistance,
                rolloffFactor: TOOLKIT.AudioSource.DEFAULT_ROLLOFF,
                spatialSound: spatialBlend,
                distanceModel: "linear",
                streaming: htmlAudioElementRequired
            });
        }

This really sucks, because I cant run ANY of my kool Havok Vehicle Physics Demos on the playground. Because either the V2 spatial audio is NOT WORKING properly with two or more spatial sounds or I cant get the playground to properly run CreateEngine option with TypeScript mode so I can enable the V1 legacy audio flag on engine construction.

I am trying to get this setup in time for V9 release video. But these two issues are preventing me from doing so :frowning:

And its so hard to get my exact source to run on the playground WITHOUT all the rest of the project.

cc @docEdub , @sebavan and @Deltakosh

It is strange indeed. And the wheels of your vehicle, are they children of the vehicle? wheel.parent = car; I wonder if it would not be linked.
With you trying with 2 vehicles to see if the sounds are playing (without the wheels) ?
If both vehicles play their sounds and not the wheels, it is probably the way the wheels are attached to the vehicle.
This may be an avenue to explore.

In your options i see :

spatialPosition: new BABYLON.Vector3(0,0,0),

and

this._audio.spatial.position = new BABYLON.Vector3(0,0,0);

You could do:

spatialPosition: this.transform.position,
spatialRotation: this.transform.rotation,

and

this._audio.spatial.position = this.transform.position;
this._audio.spatial.rotation= this.transform.rotation;

From what I see it should work. Especially if it works for the vehicle. But for the wheels, I wonder if the wheels are children of the vehicle and that could be the problem. But maybe I’m wrong.

Sorry that I haven’t been able to help you more. Without a reproduction in the playground it is quite difficult to detect the problem precisely.

Yep, the wheels are children of the vehicle. I am going to try to slam my driving demo with broken audio in a playground a bit later today, so you can at least see what i am talking about.

Again… V1 works perfectly (for years) with the same exact setup.

Thanks for all your help man :slight_smile:

To go further in what I suspect, the wheels being attached to the vehicle, the position of the wheels is never updated with the vehicle, it remains at the vector3 0, 0, 0.
That’s what I think.
The wheels probably don’t provide a valid world position for WebAudio.

Maybe a test could verify this theory by doing:

scene.onBeforeRenderObservable.add(() => {
    const p = wheel.getAbsolutePosition();
    skidSound.spatial.position.copyFrom(p);
});

Tried that:

        protected async updateAudioSource(): Promise<void> {
            if (this._isAudioSpatial === true && this._audio instanceof BABYLON.StaticSound) {
                // Note: Using spatialAutoUpdate: true on the sound and attaching it to the transform node allows the audio engine to handle updating the spatial position based on the transform node's world position and rotation, so we don't need to manually update it here in the update loop.
                if (this._audio.spatial.position != null) {
                    this._audio.spatial.position.copyFrom(this.transform.absolutePosition);
                } else {
                    this._audio.spatial.position = this.transform.absolutePosition.clone();
                }
                this._audio.spatial.update();
            }
        }

Still no joy :frowning:

At this point, I think you need to use logs to check the position of the vehicle and the wheels to see the results you have.
You’re on unity, I don’t know if you have something like console.log() in unity, but you need to display everything that the values return to detect the cause and better target the problem.
This will allow you to proceed by elimination and see what really is why the sound is not playing on the wheels.

Try to see if sound.spatial exists on the wheels, the relative and absolute positions and everything that will allow you to understand.

Because I confirm that on my project it works. The only difference is that I don’t have a parent-child relationship for my sounds.

FYI… I am not on Unity. Using BAbylonJS. Just my models are exported from unity as GLTF

Yo @Dad72 thanks for all you help thus far. I know its extremly frustrating to track down.

Let me recap.

First. I am using babylonjs with my toolkit runtime, that basically gives me script component (Unity-Style) life cycle for which i iwrite my BJS code. I only export models from unity.

Second. The setup (even the partent/child stuff) worked perfect in V1 for years. All the spatial stuff, linear rolloff, min and max distances. Both sounds are spatial. Both sound when far away the sound is lower, when camera is following the car, both sounds are aat the desired volume levels. I used this setup in Ammo.js using btRaycastVehicle for YEARS, then when got Havok and I switched the physics engine and created my own havok vehicle controller. That also worked fine.

Then we switched to V2 audio and made that the default. That is when shit blew up. I have been following @docEdub edits and progess on how to implement V2 audio. I made the AudioSource.ts wrapper class that basically support V1 and V2 for the same thing, for example:

My legacy audio setup:

        /**
         * Set legacy audio data source (BABYLON.Sound)
         */
        public setLegacyDataSource(source:string|ArrayBuffer|MediaStream):void {
            if (this._audio != null) {
                this._audio.dispose();
                this._audio = null;
            }
            const spatialBlend:boolean = (this._spatialblend >= 0.1);
            const htmlAudioElementRequired:boolean = (this.transform.metadata != null && this.transform.metadata.vtt != null && this.transform.metadata.vtt === true);
            this._initializedReadyInstance = false;
            this._audio = new BABYLON.Sound(this._name, source, this.scene, () => {
                this._lastmutedvolume = this._volume;
                (this._audio as BABYLON.Sound).setVolume((this._mute === true) ? 0 : this._volume);
                (this._audio as BABYLON.Sound).setPlaybackRate(this._pitch);
                (this._audio as BABYLON.Sound).setPosition(this.transform.absolutePosition.clone());
                this._initializedReadyInstance = true;
                if (spatialBlend === true && this._audio instanceof BABYLON.Sound)
                {
                    this._isAudioSpatial = true;
                    this._audio.attachToMesh(this.transform);
                }
                if (this.onReadyObservable && this.onReadyObservable.hasObservers()) {
                    this.onReadyObservable.notifyObservers(this._audio);
                }
                if (this._playonawake === true) this.play();
            }, { 
                loop: this._loop, 
                autoplay: false, // Note: Never Auto Play Here 
                refDistance: this._mindistance, 
                maxDistance: this._maxdistance,
                rolloffFactor: TOOLKIT.AudioSource.DEFAULT_ROLLOFF,
                spatialSound: spatialBlend,
                distanceModel: "linear",
                streaming: htmlAudioElementRequired
            });
        }

and V2 audio setup:

        public async setAudioDataSource(source: string | ArrayBuffer): Promise<void> {
            if (this._audio != null) {
                this._audio.dispose();
                this._audio = null;
            }
            const spatialBlend: boolean = (this._spatialblend >= 0.1);
            this._initializedReadyInstance = false;
            this._lastmutedvolume = this._volume;
            const defaultOptions: any = {
                loop: this._loop,
                volume: (this._mute === true) ? 0 : this._volume,
                autoplay: false, // Note: Never Auto Play Here 
                autoUpdate: true,
                playbackRate: this._pitch,
                spatialEnabled: spatialBlend,
                spatialAutoUpdate: true,
                spatialPosition: new BABYLON.Vector3(0,0,0),
                spatialRotationQuaternion: new BABYLON.Quaternion(0,0,0,1),
                spatialPanningModel: "equalpower",
                spatialDistanceModel: "linear",
                spatialMinDistance: this._mindistance,
                spatialMaxDistance: this._maxdistance,
                spatialRolloffFactor: TOOLKIT.AudioSource.DEFAULT_ROLLOFF,
                spatialConeInnerAngle: 360,
                spatialConeOuterAngle: 360
            }            
            this._audio = await TOOLKIT.AudioSource.CreateStaticSound(this._name, source, defaultOptions);
            if (this._audio != null) {
                this._initializedReadyInstance = true;
                if (spatialBlend === true && this._audio.spatial != null)
                {
                    this._audio.spatial.position = new BABYLON.Vector3(0,0,0);
                    this._audio.spatial.rotationQuaternion = new BABYLON.Quaternion(0,0,0,1);
                    this._audio.spatial.panningModel = "equalpower";
                    this._audio.spatial.distanceModel = "linear";
                    this._audio.spatial.minDistance = this._mindistance;
                    this._audio.spatial.maxDistance = this._maxdistance;
                    this._audio.spatial.rolloffFactor = TOOLKIT.AudioSource.DEFAULT_ROLLOFF;
                    this._audio.spatial.coneInnerAngle = 360;
                    this._audio.spatial.coneOuterAngle = 360;
                    this._audio.spatial.attach(this.transform, true, BABYLON.SpatialAudioAttachmentType.PositionAndRotation);
                    this._isAudioSpatial = true;
                }
                if (this.onReadyObservable && this.onReadyObservable.hasObservers()) {
                    this.onReadyObservable.notifyObservers(this._audio);
                }
                if (this._playonawake === true) this.play();
            }
        }

and to play and audio track:

        /**
         * Play the sound track
         * @param time (optional) Start the sound after X seconds. Start immediately (0) by default.
         * @param offset (optional) Start the sound at a specific time in seconds
         * @param length (optional) Sound duration (in seconds)
         */
        public async play(time?: number, offset?: number, length?: number): Promise<boolean> {
            await TOOLKIT.AudioSource.UnlockAudioEngine(); // Note: Always Attempt To Unlock Current Audio Engine (V2 Preferred) On Play
            this.internalPlay(time, offset, length);
            return true;
        }
        private internalPlay(time?: number, offset?: number, length?: number): void {
            if (this._audio != null) {
                if (this._initializedReadyInstance === true) {
                    if (this._audio instanceof BABYLON.StaticSound) {
                        this._audio.play({ waitTime:time, startOffset:offset, duration:length });
                    } else if (this._audio instanceof BABYLON.Sound) {
                        this._audio.play(time, offset, length);
                    }
                    this._isAudioPlaying = true;
                    this._isAudioPaused = false;
                } else {
                    this.onReadyObservable.addOnce(() => {
                        if (this._audio instanceof BABYLON.StaticSound) {
                            this._audio.play({ waitTime:time, startOffset:offset, duration:length });
                        } else if (this._audio instanceof BABYLON.Sound) {
                            this._audio.play(time, offset, length);
                        }
                        this._isAudioPlaying = true;
                        this._isAudioPaused = false;
                    });
                }
            }
        }

So if legacy V1, I am using the BABYLON.Sound class and when V2, I am using the BABYLON.StaticSound class

I also am not sure about the parent child relationship. We never had this kind of issue in V1.

Again really frustrating trying to track down what is V2 doing that spatial sound is so much different.

If I simply, Enable Legacy V1 Audio Engine on construction { audioEngine: true } and force V1.

EVERYTHING IS PERFECT, so positions of my car and its wheels are no different.

I dont know man, i am bout to give up on V2 audio for now :frowning:

But hey man, I really appreciate all you have done to try and help me figure this out, thanks so much :slight_smile:

@Deltakosh , please dont completly remove V1 audio in near future, until i can figure out these V2 spatial audio issues.

I have to return to try and get my Havok Vehicle Demos working for the V9 video release. So i guess i will jut have to host on my own index.html pages where I can control the V1 legacy audio options.

adding @RaananW to help :slight_smile:

1 Like