Copy camera rotation matrix to omnitone rotation matrix

Questions
1

I’m getting this error attempting to get the camera rotation and copy it to the listener position in Omnitone.
uisng : https://googlechrome.github.io/omnitone/#home
omnitone.min.js:17 Uncaught TypeError: Cannot read properties of undefined (reading '0') at Object.A.invertMatrix4 (omnitone.min.js:17:446) at z.setRotationMatrixFromCamera (omnitone.min.js:17:18117) at eval (Game.js:118:20) at e._renderLoop (babylon.js:16:98406) A.invertMatrix4 @ omnitone.min.js:17 z.setRotationMatrixFromCamera @ omnitone.min.js:17 eval @ Game.js:118 e._renderLoop @ babylon.js:16 requestAnimationFrame (async) e.QueueNewFrame @ babylon.js:16 e._renderLoop @ babylon.js:16 requestAnimationFrame (async) e.QueueNewFrame @ babylon.js:16 e.runRenderLoop @ babylon.js:16 doRender @ Game.js:114 eval @ index.js:10

My Constructor :

constructor(canvasId) { this.canvas = document.getElementById(canvasId); this.engine = new BABYLON.Engine(this.canvas, true); this.time = 0; this.audioCtx = new AudioContext(); this.FOH = Omnitone.createFOARenderer(this.audioCtx, { // The example audio is in the FuMa ordering (W,X,Y,Z). So remap the // channels to the ACN format. channelMap: [0, 3, 1, 2], });
my render :
doRender() { this.engine.runRenderLoop(() => { const shaderMaterial = this.scene.getMaterialByName("shader"); shaderMaterial.setFloat("time", this.time); this.time += 0.02; this.FOH.setRotationMatrixFromCamera(this.camera);
shaderMaterial.setVector3( "cameraPosition", this.scene.activeCamera.position ); this.scene.render(); }); window.addEventListener("resize", () => { this.engine.resize(); }); }

Camera Declaration :
this.camera = new BABYLON.FreeCamera( "camera1", new BABYLON.Vector3(0, 5, -10), this.scene ); this.camera.setTarget(BABYLON.Vector3.Zero()); this.camera.attachControl(this.canvas, false);

I would like to be able to rotate the camera view and have the audio mix rotation copy the camera rotation matrix… I’m hot sure how to translate these two types any help would be appreciated.

Bless,
-Gennaro

2

Could you ask the omnitone ppl what do they expect in their matrix?

3

sure

4

great then let s see how we can make it work :slight_smile:

5

this may be helpful… it looks like it was used with ForgeJS library to do something similar. I’m getting to processing matrices … still working on vectors :stuck_out_tongue: at the moment :slight_smile:

maybe this source code can help from forge JS…

Maybe this code from three-js and vr Omnitone can help…

I was using three.js and it’s great, but Babylon JS is progressive at the moment and scalable, gives me the ability collaborate with non technical artists a little better too.

I want to install this engine so I can mix pre baked audio mixes with spatial audio and synchronize with animation and video on web.

1 Like
6

In their GitHub readme they show how to do it for ThreeJS like this:
renderer.setRotationMatrix4(camera.matrixWorld.elements);

So then I think the BabylonJS equivalent would be like this, where renderer is your Omnitone renderer:
renderer.setRotationMatrix4(camera.getWorldMatrix().m);

And if that’s working for you, it might be good to try to PR it into their readme too, and have that little example for each engine… :slight_smile:

7

also it would mean they might work in right handed system and you could set the babylon scene in this mode with scene.useRightHandedSystem = true

1 Like
8

I’m still working on figuring out the exact orientation but it IS WORKING SO FAR!!! I really appreciate the help and will definitely do a PR when I sort everything out. This means a lot, I’ve been working on understanding how to wire this up for a while and really value the help and your time. Thank you!

Bless,
Gennaro

2 Likes