I have been lurking here a few months now and shared a few pictures of my VR project. Babylonjs is very nice to work with and I feel there is great potential here to use it as a platform for game development or any kind of 3d experience. I have tried making something like this in Unity before but the slow build cycle there is a killer as well as the license. Plus I love JavaScript.
As I am exploring all kinds of interactions in VR and slowly building my engine there have been a lot of focus on performance. My target platform in PCVR with a moden GPU like 4000 series or higher as I really want as high fidelity as possible through Babylonjs. All my meshes today are just stuff I have gotten from Sketchfab, except a few models that I have made myself. I build it in a Tauri container which is using WebView2 from Edge that is based on Chromium.
But I am having a bit of a hard time figuring out where I should optimize as when I chat with Gemini and other AI’s they keep reminding me that this is slow and that is slow (like alpha clipped textures), only to see that both Oculus Debug Tool and SteamVR reports that my GPU is practically idling when rendering my scene, hardly ever going over 1ms even. I cant really hear the fan spinning up either although windows seems to report my 4070 Ti Super GPU is at 50% and CPU at 5% while running it. So it also feels like the Oculus tool or the SteamVR GPU reporting might be wrong?
In the scene above I have cranked up the amount of thin instances per 32x32m chunk in my world to 1000 and similarly many trees. Interestingly there is no real correspondence here, at 200 instances per chunk the amount of CPU render time reported by oculus debug tool is only a tad lower.
Here is a list of things I have done:
Quest 3: 72 Hz refresh set, running at 100% res
Oculus Debug Tool, ASW turned off, H.264 encoder at 900 Mpbs over USB-C 3 link cable
My Tauri app has been set to run at max GPU in both windows and Nvidia Control Panel
All meshes have world matrices frozen (I only unfreeze dynamic ones)
Extensive use of AssetContainers so that only chunks and their meshes in the frustrum is actually in the scene (AssetContainers is fantastic and blazingly fast and easy to use)
And yes the scene used Cascaded Shadow Maps now (4 cascades), turning that off reduces mostly the CPU Render time and has little effect on actual GPU usage it seems. It feels like it is very CPU bound (which I know is an issue since this is a single thread JavaScript engine). It is ofc sad to see my CPU is also idling at 5% usage, but I guess that is because those 5% are the single core allocated for the JS engine.
Gemini keeps suggesting stuff like turning on dual-view rendering but that is default on now right? (Babylonjs also has this in their XR docs but none of the code in those work with v8). Also foveated rendering which when I test for it I get that it is not supported so possibly yet another link cable limitation of the WebXR implementation? Still it seems I need more CPU optimization than GPU at this point so.
If anyone has any tips on what I should try out to improve performance it would be nice.
EDIT: I did some tests with a somewhat filled scene with many meshes (mostly thin instances) (1500 draw calls) and no doubt CSM with 4 cascades is rather expensive on the CPU side. Oculus debug tool render time drops from around 25ms to 18ms if I turn off CSM totally (but then everything looks rather poor imo). Although I did freezeWorldMatrix() on the base meshes for my thin instances as well as doNotSyncBoundingInfo = false (I do my own culling of full chunks) it looks like perhaps if you remove the meshes from an asset container and then re-add them these settings are lost? Anyway I got some improvement in performance if I set those on every root thin instance mesh every time I swap them in from their assetcontainer.
I have been trying most of the things here except freezing materials. Is there a big benefit to that as well?
There are 2 paths you need to check t improve performance. The first is WebXR, the second is general rendering issues.
WebXR - There are a few differences between a WebXR scene and a “Regular” 3D scene in Babylon. The main difference is te startup process and the features that are enabled in WebXR, namely - the controller selection feature and the near-field picking feature. These 2 can be a little expensive, if all meshes are pickable in your scene. But there are pure CPU-related features.
Another issue with WebXR is something we can’t avoid - we are rendering twice. once for each eye. it is true that the viewport is reduced (and babylon takes care of it) but there is no way around the two passes on the entire rendering pipeline for each eye. Now, this IS GPU related :). Babylon does support multiview (and layers - that use it), however, they are only supported in very limited number of devices. And surely not through a link connection. So - they cannot be used to improve performance.
Foveated rendering, reduced number of frames, lower resolution - those are all tradeoffs that can be used to improve your scene’s rendering time with clear consequences - less quality, less fluid work. This is up to you to decide what to do with it. TBH - foveated scene only makes sense when you also have eye tracking, otherwise you expect the user to always look to the center - and this is not the case. and, since (AFAIK) the only device that actually supports it fully in the quest 3 (with the quest browser) I don’t think using it makes sense. But this depends on your experience - it might make sense to your scene and your usage.
Now, when getting out of the XR world and into the babylon world, there are so many things you can do to improve rendering and there are much mroe talented people than me that can help with these types of optimizations. A good start could be - Babylon.js docs
And a side note - I see you talk a lot about the state of the quest link and support for specific features. And i totally agree with you. I am still amazed the there is no official hand support when using link. The only thing we can do is complain, but as Meta is shifting from XR to AI (heck, soon they will change the name of the company to LLMWorld or something) there will be less investment in XR features and less need to implement such features that are only used by a handful of people. I wanted to add “sadly” in every sentence i have written in this paragraph. Just imagine I did.
Thanks for the long reply. Yes it is a shame there isn’t a serious push for good WebXR support for the devices GPU rendered through a link cable or wireless as I feel it is a lost opportunity here for hobby devs like myself to do something nice with a JS based framework. Most LLM based development is also way easier when all you do is writing code - which is a big limitation in how you would work with tools like Unity.
I am trying all the optimizations now, but it seems to come down to reducing draw calls and number of meshes the CPU part needs to traverse every frame. Since there is one draw call per material, heavy use of texture atlases seems to be a must. Naturally thin instances everywhere you can use them, and locking down static meshes in any way possible to reduce Babylonjs having to check them at all. Naturally I have gone a bit overboard with CSM using 4 cascades (will lower it to 2-3) and I need to pick more carefully which meshes actually can cast shadows using that even though they can receive them.
I thought my problem was the alpha clipped stuff like grasses and foliage on trees, but even swapping for simple imposters did nothing really since the problem was never on the GPU, but the single thread JS architecture.
Also if anyone has any tips on how to reduce shimmering on alpha clipped meshes that would be nice - do I perhaps have to consider switching to real alpha blend for them instead? All other solid meshes are very nicely antialiased, but the alpha clipped stuff has shimmering around them as well as the shadows.
Do you know if there is any progress on WebGPU support for WebXR VR? I understand that could speed up the CPU part of the engine considerably?
