Reducing memory consumption in BabylonJS apps

Hey!

I was solving this issue nowadays. Our app was crashing on non-Pro iPhones and mid-level Android devices. The FPS was around 14-16 on an iPhone 10 (the model we choose as the bottom line device the app has to run on). After I did some optimizations on the model/code the app runs on nearly all devices we could put our hands on (except some really old androids) and the FPS is in range of incredible 45-50 FPS on the iPhone 10.

DO THIS FIRST!

Change the powerPreference to default. BabylonJS sets it to high-performance by default. Especially iOS is quite unforgiving when set to high-performance and Safari immediatelly kills your app if it detects high battery usage, so do not force high-performace mode by default but go with default which gives the browser the permission to set it’s own rules how to handle the webgl stuff in your app.

  createEngine(canvas: HTMLCanvasElement): Engine {
    const engine = new Engine(
      canvas,
      true,
      {
        powerPreference: 'default',
      },
      false
    )
    return engine
  }

Just this one step pushed the app compatibility towards the Low profile devices and the app was not crashing so often as it used to.

Profiles

First of all I decided to create 3 groups of devices we gonna support or let’s call them profiles instead of groups.
The Low profile includes the older non Pro iPhones and their Android equivalents.
The Mid profile includes Pro iPhones and their Android equivalents.
The High profile includes Desktop and high range phones like the iPhone 15 Pro.

Packing

I realized that the first bottleneck seems to be always the textures when it comes to memory consumption as @cedric suggested. Put all your textures into the glb (I was loading some textures dynamically by code) and PACK the glb file. This already helped a lot to reduce memory footprint.

This is how I pack the glbs: GLTF/GLB texture and compression support in babylon.js - #2 by roland The FPS was poor but it was running on even more devices!

Summoning @labris for more elaborated information. He is the king of packing on the forum :wink:

Texture resolution

We create our assets always for the High profile devices.

We use Blender which has Python scripting support so you can do pretty nice things before you export the glb. Basically we have different resulting glbs for different device profiles. Downscaled textures to 2k or 1k. The level of packing is also set differently for each of the profiles. I do not include lightmaps in the low profile glbs for example. So you are basically trading in graphics fidelity.

Start to test with 4k packed textures and go lower if the app is crashing until you find which texture resolution works for you.

Go low poly, use instances

Remove as many geometry as you can. You can use the packer to do lossy packing of the geometry too but it can create glitches. Sometimes it is not perfect just good enough but it will run even on your fridge :stuck_out_tongue:

Another thing already mentioned by @cedric is instances. Use them wherever you can.

HTML layers vs FPS

If your app uses HTML UI, you render something above the webgl canvas, the GPU needs to compose the UI layers together with the webgl canvas. The more HTML layers the more GPU cycles your app needs. Our app uses React. We had like 40+ HTML layers on some screens. After rewiriting the UI we ended up just by 2! One is the webgl canvas and the second one is the HTML layer. You can achieve this by using React portals or VUE teleport. I’m not familiar with Angular or Svelte but I bet they have something similar.

Another thing you have to keep in mind is to optimize your UI components rehydrations. A wrong component design can lead to a lot of unnecessary rehydrations and redraws of the UI layers which triggers GPU composition every time.

This change ramped up the FPS quite a lot!

Use any tool available to debug

Connect your phone to your computer and use desktop browser tools to inspect what is happeninng on your phone. Try different browsers, different tools…

I used Safari to debug issues on iOS a lot. If you gonna connect your phone to you computer don’t forget to use a datacable not a simple charging cable. Enable the Web inspector in Safari on your phone. Turn off Wifi/Bluetooth on your phone before starting to debug.

All about performance profiling in Chrome:
https://developer.chrome.com/docs/devtools/performance/overview

FPS in general

Use everything you can:

https://doc.babylonjs.com/features/featuresDeepDive/scene/optimize_your_scene

On demand rendering

Power consumption is another concern for a mobile app developer. If your scene becomes static, when nothing moves/changes on the scene you stop the rendering loop.

A simple event handler on the canvas can trigger starting the render loop again. For example when the users clicks on something, moves the mouse, etc… You can start the render loop manually when you know that something needs to move on your scene. For example a navigation agent was added and started to move towards it’s destination point. When your agent reaches the destination you can stop the rendering loop again. Etc… babylonjs has a lot of observables to use in scenarios like this.

Misc

You can crash Safari on iOS by loading too much assets at a time. I had a hardtime to load 8 textures at once. 8 subsequent calls to new Texture(url) crashed the app on iOS. If you need to load textures this way do it in batches of 3-4 for example.


This approach applies to desktop first apps as well so keep optimizing!!

18 Likes