Throttle processing power/bandwidth to keep stable FPS during loading

Questions
1

Hello,

I am loading in textures that are rather big and thus have impact on the FPS when they are getting added. I am loading in the models with a basic color/texture and want further detail to be loaded on the fly without impacting performance too much. That way you are supposed to be able to control the camera in a basic scene while it gets “prettier”. Right now though the FPS drop to a point that’s not toleratable and it also sometimes freezes completely, leaving you with no control whatsoever. The reason is to eliminate a long loading screen in the beginning.

Is there a way to distribute the dedicated processing power/bandwidth used? Or should I persue another loading technique? I am currently chaining promises after another to have an order of the more important stuff being added first while less important stuff is added afterwards.

Looking forward to tackling this issue!

2

I don’t know if textures can be loaded in a separate worker thread and passed to the main thread once they are created… I think that would be the only way to avoid hiccups on the main thread.

Note that the problem is not the loading of the texture itself as it is done by the browser in separate native threads but the creation of the texture object once it is done, and maybe the loading into the WebGL layer / GPU.

This thread can also help: Texture loading blocks thread - #2 by Deltakosh

3

Hey,

I read a bit, seems it’s just not possible with javascript being single threaded.

Now for a compromise and to still evade a long loading screen in the beginning I would like to have a short loading animation when a texture is being created. I tried around and there doesn’t seem to be a callback function for the assets manager for that case, since onProgress gets called AFTER. Maybe I can just check any significant FPS drops and then have a loading animation appear?

4

I might be very ignorant here of how the texture object is created, but can’t it be split into multiple operations so that the main thread has some time to breathe in between? @Evgeni_Popov would love to read any details on how the texture object is created (not sure if there is any doc or where to look in code).

5

Once the texture is loaded by the browser and handed to the js code there’s some operations going on but not much (except if you are loading some cube textures and some preprocess must be applied on them) and then the texture is created/setup in the gl layer. I suspect it is where most of the time is spent as we really don’t do a lot on our end.

6

Is there a way I can prevent the creation process after the download of the texture? From what I saw in the processing/network tab of chrome the download is not the issue for performance loss. It’s when the download finishes and probably like you said the texture really gets created. If i was to download them all beforehand and create/apply them when I want to I could setup another loading screen.
I guess that also makes up my previous question: Is there a callback or some other way to handle when something is downloaded before the creation process?

7

No, there’s no callback here.

What you can do is downloading the data “by hand”, and when ready create the texture by handing these data. See this post: Texture loading blocks thread - #3 by Evgeni_Popov

8

I don’t really get what you want me to do from your linked post although I tried it. What good does it do to have an empty texture if it lags when I’m loading in the real texture. Every download option follows a creation (Chromes profiler lists: e._prepareWebGLTexture) or am I missing something? How do you split up the downloading and the creation?

9

You can create an empty RGBA texture and later on update it with the data you downloaded: it’s the fastest you can do on the creation side as some code will be avoided because of the empty data provided at construction time.

10

You can create an empty RGBA texture and later on update it with the data you downloaded

That still makes it sound like you can just download the texture and then wait for when you want to apply it to then have the lag. But I’m getting the lag when the download finishes.

11

Could you setup some code on the Playground so that we can see how you are doing it?

How are you downloading the texture?

12

I tried
new BABYLON.Texture("dropbox url")
and
assetsManager.addTextureTask("", "dropbox url"); assetsManager.load();

Really nothing to show off in a playground other than the filesize of the texture (roughly 35MB).
It doesn’t matter if you apply it, it just lags when it finishes downloading.

13

Is the decode from .jpg / .png to expanded occurring in the download / fetch thread?

14

I tested the Basis compression earlier on a project with a similar issue and have some success!

15

Applying it to a property (like material.diffuseTexture) basically takes no time, it’s just setting a variable to the value of another variable.

Everything is done when creating the texture by new BABYLON.Texture("dropbox url").

You should check that by simply creating an empty raw RGBA texture (const texture = BABYLON.RawTexture.CreateRGBATexture(null, width, height, scene)) you don’t experience the hiccups. If there are still some hiccups, you can’t do anything against them. If not, then you can try this:

  • create an empty RGBA texture in the main thread
  • download your picture from a worker thread, see for eg Loading Images with Web Workers - DEV Community. You will need to generate raw RGBA data, though, not a blob as it is done in this link (maybe you can create an ImageBitmap from the blob, then draw this bitmap in an offscreen canvas and finally get the RGBA data from the canvas)
  • pass the raw RGBA data from the worker thread to the main thread and update the raw RGBA texture with those data

I think it’s the fastest it can be. If there are still some hiccups, it’s because of the texture update processing, which is unavoidable.

5 Likes
16

Been running into this as well during runtime of our app because we are constantly streaming in new geometry. For now, we limit the number of textures that can be processed a frame and give a breather after each one, that helps performance a lot. I have put the glb download into a worker and transferring the array buffer over which helps but not much.

Been contemplating for a while separating the ktx2 textures from the glb and trying that in a separate thread as well. @Evgeni_Popov that is very helpful, thanks. Would not have guessed about the rendering to off screen canvas part.

Edit: The way we slow them down is by limiting the number of promises that can happen at once in loadMaterialPropertiesAsync and give them a timeout before they can resolve.

2 Likes
18

Hi Matt,

Do you perhaps have a code example on how you did this part? I would be very interested in seeing that :slight_smile:

1 Like
19

Sure, it’s in some other thread too but I forget. The following is inside a GLTFLoader extension:

    const pLimit = require('p-limit');
    const materialLimit = pLimit(1);

    this.loadMaterialPropertiesAsync = function (context, material, babylonMaterial) {
       let promises = [];
        promises.push(
            materialLimit(() => new Promise((resolve) =>
                loader.loadTextureInfoAsync(
                    context + '/pbrMetallicRoughness/baseColorTexture',
                    material.pbrMetallicRoughness.baseColorTexture,
                    function (babylonTexture) {

                        // ...apply the texture

                        setTimeout(() => {
                            resolve(true);
                        }, WAIT_TIME_PER_TEXTURE_BLOB);
                    }
                ))
            )
        );

        promises.push(
            materialLimit(() => new Promise((resolve) =>
                loader.loadTextureInfoAsync(
                    context + '/normalTexture',
                    material.normalTexture,
                    function (babylonTexture) {

                        // ...apply the texture

                        setTimeout(() => {
                            resolve(true);
                        }, WAIT_TIME_PER_TEXTURE_BLOB);
                    }
                ))
            )
        );

       return Promise.all(promises);
    }

plimit is here: p-limit - npm

20

Perhaps not a complete solution, but if your 3D app is the 2nd thing the user will load/see, you could start secretly pre-loading resources on the 1st landing page.

Then when your 3D app starts, some if not all resources have already been downloaded to the user, so they will load more quickly from browser cache (manifest).

Setup:

  1. Page 1: Simple landing page with some teaser text and a ‘start app’ button (+ hidden simple scene that will only start pre-downloading certain assets).
  2. Page 2: Actual 3D app (scene etc.) that requires the high FPS performance.

As a 1st time user will experience the ‘reading teaser, looking at trailer images, then decide to start app’-step as time consciously spent instead of time waited/wasted.

Nonetheless would be interesting to know about ways to non-intrusively load new resources during app usage (e.g. when your scene needs to be enhanced with new assets when the user moves closer towards the next area).