# How to Stream Audio into the Engine

**URL:** <https://forum.babylonjs.com/t/how-to-stream-audio-into-the-engine/11784>\
**Category:** Questions\
**Created:** [June 17, 2020, 8:39pm UTC](https://forum.babylonjs.com/t/how-to-stream-audio-into-the-engine/11784 "2020-06-17T20:39:41Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![missionz3r0](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/missionz3r0/32/9155_2.png) [@missionz3r0](https://forum.babylonjs.com/u/missionz3r0)\
**Post date:** [June 17, 2020, 8:39pm UTC](https://forum.babylonjs.com/t/how-to-stream-audio-into-the-engine/11784/1 "2020-06-17T20:39:41Z")

</div>

I am working on a peer to peer application, and one of the things I would like to be able to do is have a user share audio with the others. Currently looking to do this with webtorrents.

Now, with a webtorrent I can get a readable stream which gives me chunks of data in the appropriate order, with each chunk being a node Buffer. And each chunk of data I can turn into an arraybuffer on either on their own, or all together to get the completed audio file. I’ve not figured out how to convert a node Buffer into a mediastream either.

My problem comes when I want to consume these chunks. After the audio from a chunk has played, I’d like to discard the chunk so I’m not just holding the audio in memory. But, from what I can tell that’s easier said than done.

The sound constructor can accept a url, an arraybuffer, or a mediastream.

I could do a blob uri for the url, but that’d require having the entire audio clip already loaded to do that which defeats the purpose. An arraybuffer is slightly better. Things can be added into it over time, so I don’t need everything loaded already to run it. But to my knowledge, it doesn’t consume and discard the data as its used. It holds onto it. Mediastreams are a bit more of an enigma to me, they look like they are used specifically for pulling media from a user’s camera and microphone which isn’t what I’d like to do. And I’m not even sure if they would do what I’m wanting either because I don’t know if they discard media, or hold onto it.

My current thought is to create several sound objects in babylon, each of which holds a chunk of the audio, and play them back to back. However this seems clunky, and I’m not sure it could work.

Some additional notes. The audio files could be of variable length, anywhere from a few seconds long to hours. Also, I’m not tossing the chunks completely after they’ve been used. That would defeat the purpose of using webtorrent. I just want to pull it out of memory.

So with this in mind, is there a way to stream music into the babylon engine? And what about other media, like videos?

---

<div class="post-metadata">

**Author:** ![thomlucc](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/thomlucc/32/4046_2.png) [@thomlucc](https://forum.babylonjs.com/u/thomlucc)\
**Post date:** [June 17, 2020, 9:55pm UTC](https://forum.babylonjs.com/t/how-to-stream-audio-into-the-engine/11784/2 "2020-06-17T21:55:58Z")

</div>

Hello @missionz3r0 - and welcome to the Forum!

I’ll start by sharing some general information, before some more expert folks jump-in to help you:

- A good starting point is the the “[Play Sounds and Music](https://doc.babylonjs.com/how_to/playing_sounds_and_music)” page of the documentation where you’ll find a few examples like [Loading a sound from an ArrayBuffer](https://doc.babylonjs.com/how_to/playing_sounds_and_music#loading-a-sound-from-an-arraybuffer) and streaming sound captured from the microphone: [Babylon.js Playground](https://playground.babylonjs.com/pg/MB5S2C/revision/2).

- Video streaming can be done using the videoTexture like explained in the [solution of this thread](https://forum.babylonjs.com/t/video-streaming-to-texture/6672).

---

<div class="post-metadata">

**Author:** ![missionz3r0](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/missionz3r0/32/9155_2.png) [@missionz3r0](https://forum.babylonjs.com/u/missionz3r0)\
**Post date:** [June 17, 2020, 11:25pm UTC](https://forum.babylonjs.com/t/how-to-stream-audio-into-the-engine/11784/3 "2020-06-17T23:25:44Z")

</div>

Yep, read over those and I don’t know how many other docs while working on this project.

Shows how to stream things in, but not out. If I’m playing a 2 hour long piece of audio, and I’m 70 minutes in, I don’t really need the last 60+ minutes loaded into memory right at that moment.

---

<div class="post-metadata">

**Author:** ![Deltakosh](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/deltakosh/32/26635_2.png) [@Deltakosh](https://forum.babylonjs.com/u/Deltakosh)\
**Post date:** [June 18, 2020, 12:06am UTC](https://forum.babylonjs.com/t/how-to-stream-audio-into-the-engine/11784/4 "2020-06-18T00:06:00Z")

</div>

That will probably require a server capable of streaming music directly. From the client standpoint, if you give a url to a stream then it will work transparently

---

<div class="post-metadata">

**Author:** ![missionz3r0](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/missionz3r0/32/9155_2.png) [@missionz3r0](https://forum.babylonjs.com/u/missionz3r0)\
**Post date:** [June 18, 2020, 12:11am UTC](https://forum.babylonjs.com/t/how-to-stream-audio-into-the-engine/11784/5 "2020-06-18T00:11:02Z")

</div>

I can tell you that a server isn’t needed in this case. I’ve already tested uploading and sending a file from client machine to client machine. I can also pass the file into Babylon, however it requires either putting the whole file into it, or eventually putting the whole file in. But what I’m trying to do is stream in some while streaming out parts already used.

I’m starting to think I could just split the file into something like 30 second chunks, and playing them back to back, destroying the already played audio as I go.

---

<div class="post-metadata">

**Author:** ![Deltakosh](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/deltakosh/32/26635_2.png) [@Deltakosh](https://forum.babylonjs.com/u/Deltakosh)\
**Post date:** [June 18, 2020, 5:04pm UTC](https://forum.babylonjs.com/t/how-to-stream-audio-into-the-engine/11784/6 "2020-06-18T17:04:21Z")

</div>

probably the best option if you do not want to dig into RTMP protocol

---

<div class="post-metadata">

**Author:** ![ycw](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/ycw/32/4315_2.png) [@ycw](https://forum.babylonjs.com/u/ycw)\
**Post date:** [June 19, 2020, 5:28pm UTC](https://forum.babylonjs.com/t/how-to-stream-audio-into-the-engine/11784/7 "2020-06-19T17:28:16Z")

</div>

Use `<audio />` if that is a browser based app. Set `audio.src` to object url of a **MediaSource** , then add a **SourceBuffer** to mediasource, then stream data into source buffer. To get **MediaStream** from audio, use `audio.captureStream()`.

```auto
const media_source = new MediaSource()

audio.src = URL.createObjectURL(media_source) 

const media_stream = audio.captureStream() 
const sound = new BABYLON.Sound(.., media_stream, ..)

media_source.onsourceopen = function () {
   const source_buffer = media_source.addSourceBuffer(..)
   // ...
   source_buffer.appendBuffer(/* buffer here */)   
}

```