Downloads are async and shouldn’t hurt the main thread at all. I’d preload audio to ensure it’s ready when needed: Play Sounds and Music - Babylon.js Documentation
Also, perhaps this post will be useful to you: Good practice to load and access an asset (e.g. sound) from code?