Obviously each format (mp3, ogg, wav, etc) have their pros/cons in terms of file size and quality, but are there anything specific to babylon that should be considered? For example, in Unity I was under the impression (and I don’t even know if this is true) that compressed formats like mp3 would take more CPU to run and thus it was better to use wav files for SFX and mp3 files for longer things like music/voice. Is that similar here or should there be no difference?
I’m asking in general, but my extreme use case would be a school chromebook which is both slow on the CPU and also has a slow wifi. I’m already struggling to keep 30fps on the CPU so I am hesitant to introduce anything that may negatively impact performance but I suspect the slow wifi is going to be the bigger issue and thus mp3/ogg may still be the way to go.
I’m far from an expert in Babylon audio, but I think it is using Web Audio API under the hood.
Also interesting is the decoding efficiency. Using decoding speed as a proxy for CPU burden, mp3 seems pretty low burden (and has great browser support).
I think the recommendation on wav for SFX is based on the latency and CPU burden of decoding mp3. If you need to keep an audio file in an ArrayBuffer (such as loading mp3 SFX into an ArrayBuffer then playing when needed from there), check out this reddit post which links to this Web Audio API call (decodeAudioData()).
I’m not aware of any Babylon-specific considerations, but look for an expert to come by and corroborate that. Of course, this may be different for BabylonNative.
For Babylon, the default Sound options make compressed audio formats get decompressed before they are played. So, any CPU overhead occurs just once. There is no CPU used for decompression while they’re playing.
This changes if you set the sound’s streaming option to true, though. The streaming option decompresses the audio as it’s received, and browser implementations vary on this, so it’s possible for the decompressed audio to be thrown away by the browser even if you know you’re going to need it again later. When this happens, the same audio stream will need to be decompressed again.
The streaming option uses less memory for large audio files because it doesn’t store the entire decompressed audio buffer, but it uses more CPU during playback because it has to decompress the same audio repeatedly.
Also be aware that the streaming option does not loop seamlessly on some browsers, so it’s not possible to guarantee there will be no gap in the audio at the loop point. If a sound must have no gap at the loop point, then do not use the streaming option.