Web Audio API error related

[onekit]

Hi @sebavan, let me explain using a specific case how I sometimes catch this error. In the game, the character moves through space; a FreeCamera is attached to it, and a Spatial Sound listener is attached to that camera. As long as the coordinate precision stays at a reasonably whole-number level, everything is fine. As soon as the fractional part becomes extremely precise, an error occurs because of a very large fractional component.

The position is updated continuously, and the issue is quite rare, but it can happen from time to time. As I’ve studied in detail, this is not specific to Firefox only - the difference is that Firefox completely freezes when this error occurs, while Chrome allows the error to happen. However, recently Chrome has also started freezing on this error.

The solution as I see it: either we add a service layer between assigning coordinates to the Spatial Sound listener, or we add a filter directly to the Sound object.

But of course, considering that this breaks attachToMesh, there’s no way for me to plug in my own custom service here. I’d like this to work at the level of BabylonJS itself, without my own hacks or workarounds.

Thank you in advance.

[onekit]

BTW: It related bug, but isn’t audio problem at all. Time to time when I read Position from character mesh, I got Y position is NaN and can’t set this position to the Camera.

This error also supressed by my custom safe setter:

  private updatePOVCamera(camera: any) {
    if (!this.player.mesh || !this.player.eyes) return
    const headPosition = this.player.eyes?.getAbsolutePosition(this.player.mesh).clone()
    const forwardVector = this.player.mesh.forward.clone().normalize().scale(0.2)
    const targetPosition = this.player.mesh.position.add(forwardVector)
    targetPosition.y = headPosition.y - 0.45
    const threshold = 0.9
    if (targetPosition.y < this.player.mesh.position.y + threshold) targetPosition.y = this.player.mesh.position.y + threshold
    const lerpedPosition = Vector3.Lerp(camera.position, targetPosition, 0.1)

    this.safeSetCameraPosition(camera, lerpedPosition, 'Prevented setting invalid POV camera position')
  }

Cause I can’t assign Vector3 position to the camera until check is it valid Vector3:

private safeSetCameraPosition(camera: any, position: Vector3, warningMessage: string = 'Invalid camera position'): boolean {
    if (CameraValidators.isValidVector3(position)) {
      if ('copyFrom' in camera.position) {
        camera.position.copyFrom(position)
      } else {
        camera.position = position
      }
      return true
    } else {
      Logger.warn(warningMessage, position)
      return false
    }
  }

My conclusion: Time to time getting mesh position from scene give for Y the NaN result cause of Lerp maybe.

[docEdub]

Cool, ya, that will kill the error message, but it will not set the parameter’s value. The PR I made for this (https://github.com/BabylonJS/Babylon.js/pull/17579) does something similar but falls back to setting the parameter’s value directly instead. Try out the latest version of Babylon.js and you should not see the setValueCurveAtTime errors anymore.

[onekit]

Thank you so much. I’m really looking forward to the fix being included in the next BabylonJS release. I tried 8.44.1, and it doesn’t seem to have this fix yet. Firefox shows:

Uncaught DOMException: AudioParam.value setter: Can't add events during a curve event
    setTargetValue webAudioParameterComponent.ts:91
    set webAudioParameterComponent.ts:41
    _updatePosition spatialWebAudioSubNode.ts:175
    _updatePosition spatialAudio.ts:234
    set spatialAudio.ts:152
    setPosition sound.ts:647
    _onRegisterAfterWorldMatrixUpdate sound.ts:875
    _registerFunc sound.ts:857
    notifyObservers observable.ts:439
    computeWorldMatrix transformNode.ts:1326
    getWorldMatrix node.ts:495
    computeWorldMatrix transformNode.ts:1197
    isReady mesh.ts:1415
    isReady scene.ts:2401
    start RenderLoop.ts:30
    _renderFrame abstractEngine.ts:1013
    _processFrame abstractEngine.ts:989
    _renderLoop engine.ts:653
    _boundRenderFunction abstractEngine.ts:916
    b2 abstractEngine.ts:87
    _queueNewFrame abstractEngine.ts:1029
    _renderLoop engine.ts:666```

[sebavan]

Assigning back to @docEdub as it sounds 8.44.1 should be working

[onekit]

Anyway we have huge progress. In the Chrome this error disappears completely.
Now we see another error (in same place spatialAudio, but another). Firefox, oh, Firefox.

[docEdub]

Oh interesting. This is a slightly different error occurring when setting the value directly. I had not expected this would also fail when curves are active, but it does on Firefox (of course).

I will try again to repro this on Firefox since it’s doing strange things. If I can get it to repro then I have a dev build of Firefox I’ll debug it on to come up with a better solution.

[docEdub]

I can get it to repro on Firefox about 40% of the time with this playground:
https://playground.babylonjs.com/#P1FP6L

Interestingly, as I increase ItemCount, it repros less often.

Also interesting this only happens close to the start of audio. After a few seconds the error stops happening.

[docEdub]

There is some issue with Firefox’s implementation of cancelScheduledValues causing this. It works 99.999% of the time, but fails in some rare cases when there are a lot of calls to cancelScheduledValues and setValueCurveAtTime with overlapping time ranges. Since this happens so rarely I’m going to remove the fallback that’s also throwing an error on Firefox in this case and only log a warning.

[docEdub]

PR 17641 should fix/workaround this.

[onekit]

I see in Firefox these warnings, but anyway they are in console under cover. No freezes.

[sebavan]

@docEdub the frequency of the warnings is pretty high, should we keep them ?

[docEdub]

ya, maybe lower the log level or stop warning about it after the first time

[docEdub]

The lowest log level we have is still noisy, so I’ll change it to warn once and then be quiet.

[docEdub]

PR 17669 makes it warn once and then shut up about it