Properly Setup Spatial Audio

Questions
1

Yo @docEdub Can you please out line what i need to do to properly setup and use spatial audio.

Am I required to update the spatial audio position each frame, or does the class that handles attaching to the mesh do that?

Can you make a small playground that properly sets up and update spatial Audio example ?

Its hard to duplicate my toolkit usage on the playground, but this how i cam currently trying to setup spatial audio, but I dont know if its right or not, because i can only hear sound at position 0,0,0 or what ever position i specify with spatialPosition option. But it does NOT move the sound around with the attached mesh

        public async setAudioDataSource(source: string | ArrayBuffer): Promise<void> {
            if (this._audio != null) {
                this._audio.dispose();
                this._audio = null;
            }
            console.log("### Setting Audio Data Source: " + this._name);
            const spatialBlend: boolean = (this._spatialblend >= 0.1);
            this._initializedReadyInstance = false;
            this._lastmutedvolume = this._volume;
            this._audio = await TOOLKIT.AudioSource.CreateStaticSound(this._name, source,
            {
                loop: this._loop,
                volume: (this._mute === true) ? 0 : this._volume,
                autoplay: false, // Note: Never Auto Play Here 
                playbackRate: this._pitch,
                spatialEnabled: spatialBlend,
                spatialAutoUpdate: true,
                spatialDistanceModel: ((this._rolloffmode === "logarithmic") ? "exponential" : "linear"),
                spatialReferenceDistance: this._mindistance,
                spatialMaxDistance: this._maxdistance,
                spatialRolloffFactor: TOOLKIT.AudioSource.DEFAULT_ROLLOFF
            });
            if (this._audio != null) {
                this._initializedReadyInstance = true;
                if (this.onReadyObservable && this.onReadyObservable.hasObservers()) {
                    this.onReadyObservable.notifyObservers(this._audio);
                }
                if (spatialBlend === true && this._audio.spatial != null) this._audio.spatial.attach(this.transform, false, BABYLON.SpatialAudioAttachmentType.PositionAndRotation);
                if (this._playonawake === true) this.play();
            }
        }

And my CreateStaticSound helper function

Screenshot 2025-03-11 at 6.21.24 PM
Screenshot 2025-03-11 at 6.21.24 PM1920Ă—961 145 KB

2

Yep, I’m working on the docs now. I’ll post you a playground when I get to the spatial audio docs.

4 Likes
3

Here’s an example playground for spatial audio: https://playground.babylonjs.com/#VP1B9P#28.

4

…and here’s a spatial audio settings visualizer: https://playground.babylonjs.com/#VP1B9P#34

1 Like
5

Hi @docEdub All PG of documentation for the V1 engine no longer works. I believed that I understood that it had been correct, but should we do something special in the PG of the V1?
For example :

I also look at the V2 Audio Space PG above and I do not see any difference depending on the distance or sounds right and left ear.
I have changed the PG above. The mesh is a more important distance than the spatialMaxDistance option and yet the sounds plays, whether the camera is moving away or not, the sound does not take into account the distance.
The result must be: the more the object moves from the camera, the more the sounds are reduced. And if the distance from the camera exceeds spatialmaxeditance, the sounds must be cut.

If I comment: //bounce.spatial.attach(...) the sounds is playing in the same way as if the sound was not attached to the mesh. I think he doesn’t have it in fact.

6

Well in the playground , it kinda works… But it looks like maxDistance is not being respected:

In this play ground, i am moving mesh forward and backward some units… and the sound does get lower and higher as the mesh gets closer to camera… but it is the same whether i set maxDistance to 5 or 150

And i cane get the spatial part working at all outside the playground. I am working on another external project example

7

Yo @docEdub … So this repo is setup to test both the new and legacy spatial audio:

Note: At the top of App.tsx you can set enableLegacyAudio to enable legacy mode. Try them both. You can use keyboard to move camera back and forth as well.

For me legacy works fine, but the new audio engine, does not change based on mesh location:

import React, { useEffect, useRef } from 'react';
import 'babylonjs';
import 'babylonjs-gui';
import 'babylonjs-loaders';
import 'babylonjs-materials';
import TOOLKIT from 'babylon-toolkit';
import Viewer from './Viewer';
import './App.css';

function App() {

  ////////////////////////////////////////////////////////////////////////
  // CONFIGURATION
  ////////////////////////////////////////////////////////////////////////
  const moveBackAndForth = false;   // MOVE SPHERE BACK AND FORTH
  const enableLegacyAudio = false;  // ENABLE LEGACY AUDIO ENGINE
  ////////////////////////////////////////////////////////////////////////

  const onSceneReady = async (scene) => {
    // This gets the engine and canvas references (non-mesh)
    const engine = scene.getEngine();
    const canvas = scene.getEngine().getRenderingCanvas();

    // This creates and positions a debug camera (non-mesh)
    const camera = new BABYLON.FreeCamera("camera1", new BABYLON.Vector3(0, 5, -10), scene);
    camera.setTarget(BABYLON.Vector3.Zero());
    camera.attachControl(canvas, true);
    scene.activeCamera = camera;

    // This creates ambient light, aiming 0,1,0 - to the sky (non-mesh)
    const light = new BABYLON.HemisphericLight("light1", new BABYLON.Vector3(0, 1, 0), scene);
    light.intensity = 0.8;

    // Our built-in 'ground' shape. Params: name, options, scene
    const ground = BABYLON.MeshBuilder.CreateGround("ground", {width: 6, height: 6}, scene);

    // Our built-in 'sphere' shape. Params: name, options, scene
    const sphere = BABYLON.MeshBuilder.CreateSphere("sphere", {diameter: 2, segments: 32}, scene);

    // Move the sphere upward 1/2 its height
    sphere.position.y = 1;

    // Setup spatial audio test
    if (enableLegacyAudio)
    {
      if (!BABYLON.Engine.audioEngine?.unlocked)
      {
          BABYLON.Engine.audioEngine?.unlock();
      }    
      const sx = new BABYLON.Sound(
          "loading",
          "https://raw.githubusercontent.com/onekit/gardener/master/public/sounds/loader/loading.ogg",
          scene,
          null,
          { autoplay: true, loop:true, spatialSound: true, maxDistance: 25, rolloffFactor: 1 }
      );
      window["sx"] = sx;
      sx.attachToMesh(sphere);
      console.log("Legacy Sound Created: ", sx);
    }
    else
    {
      const audioEngine = await BABYLON.CreateAudioEngineAsync({ volume: 0.25 });
      await audioEngine.unlock();
      const sxx = await BABYLON.CreateSoundAsync("bounce",
        "https://raw.githubusercontent.com/onekit/gardener/master/public/sounds/loader/loading.ogg",
        { spatialEnabled: true, spatialMaxDistance: 25, spatialRolloffFactor: 1 }
      );
      window["sxx"] = sxx;
      sxx.spatial.attach(sphere);
      sxx.play({ loop: true });
      console.log("New Sound Created: ", sxx);
    }

    // Move sphere back and forth
    if(moveBackAndForth)
    {  
      let moveSpeed = 0.05;
      const moveDistance = 25;
      scene.onBeforeRenderObservable.add(()=>{
        if (sphere.position.z > moveDistance) {
          moveSpeed = -0.05;
        }
        if (sphere.position.z < -moveDistance) {
          moveSpeed = 0.05;
        }
        sphere.position.z += moveSpeed;   
      });
    }

    // Focus the render canvas
    TOOLKIT.SceneManager.FocusRenderCanvas(scene);
  };

  return (    
    <div className="root">
      <Viewer legacyAudio={enableLegacyAudio} antialias={true} adaptToDeviceRatio={true} onSceneReady={onSceneReady} className="canvas" id="my-canvas" />
    </div>
  );
}

export default App;
8

The maxDistance property doesn’t do anything with the default distanceModel of “inverse”. Set the spatialDistanceModel: "linear" option for maxDistance to take effect.

See https://playground.babylonjs.com/#VP1B9P#40.

Maybe it makes more sense for the default distance model to be “linear” instead of “inverse”? What do you think?

9

I’m looking into this. I see why it’s happening but I’m not sure of the right fix, yet.

10

Ya, getting the spatial audio settings right can be difficult. Try this playground that exaggerates some of the settings to make the spatial qualities more apparent: https://playground.babylonjs.com/#VP1B9P#39

Also note that setting the referenceDistance and maxDistance to the same value is probably not what is intended since there will be no volume ramp as the distance increases in that case. The volume will just snap from 100% to 0% when the max distance is reached.

Maybe instead of referenceDistance it should be called minDistance?

11

All PG of documentation for the V1 engine no longer works.

I’m looking into this. I see why it’s happening but I’m not sure of the right fix, yet.

Thanks for reporting this! It should be fixed by PR 16311.

1 Like
12

Thanks.
I was trying to make comparisons with PG Audio V1 and Audio V2.
I managed to make things work. I hope it’s the right way to do.

I think you could add this PG in the examples of your current documentation. It takes up what existed in the PG of the V1

From what I understand you must also add audioEngine.listener.attach(scene.activeCamera); :+1:

This seems to work well once you understand the use. I love this new audio engine to the. It seems more flexible.

1 Like
13

Awesome! Thanks. This will be a good PG to use for the migration guide!

1 Like
14

Image result for grooving cat gif
Image result for nodding yes gif
Image result for chill musician saying yes gif

1 Like
15

So i think i got it working as well. But we were missing CRUCIAL information:

Spatial audio will not work WITHOUT this:

      const audioEngine = await BABYLON.CreateAudioEngineAsync({ 
        volume: 0.25,
        listenerAutoUpdate: true,
        listenerEnabled: true,
        resumeOnInteraction: true
      });
      audioEngine.listener.attach(scene.activeCamera);

That is what you were missing when we asked how we setup V2 Spatial Audio.

1 Like
16

Yo @docEdub … since you now have to explicitly attach your camera. Can you just set a new camera over top… For example I set to use the scene.activeCamera, but then later decide switch to a different camera, let say another view for a car cockpit… Do I have to detach the current camera and re-attach to the audioEngine.listener.

Basically, do I have to call audioEngine.listener.attach(scene.activeCamera); every time i switch cameras ?

17

The audioEngine.listener.attach function detaches from the old scene object automatically before attaching to the new scene object.

18

Yes.

The listener doesn’t switch with the active camera automatically. You could do it like this https://playground.babylonjs.com/#CHD078, but you will run into trouble when Babylon switches between active cameras every frame like it does for WebXR.

19

Heh ya that example I gave could have been better. I mean, spatial sound sources work fine with the default listener stuck at position (0,0,0). The listener doesn’t have to move with the camera :slight_smile: .

Also:

  • listenerAutoUpdate defaults to true, so that’s not needed.
  • volume and resumeOnInteraction don’t do anything about spatial audio. Just saying. Not sure if that’s what you mean or not.
20

Thanks all for being on the bleeding edge of the new audio engine. It’s been a little bumpy but your feedback is making it better!

Based on your feedback, I’ve made some changes to the new audio engine’s API to address some of the issues you’re running into. See PR 16318 for the list of improvements and note that some of the changes may break code you wrote against the new audio engine, so watch out for these changes and thanks for your patience as I tweak things to 100% before Babylon 8 is released.

Thanks again!

1 Like