Upgrading from 3.1 alpha5 to 4.03, take 12, multiple issues

So this is like my 12th and final try upgrading from 3.1.a5 to newest stable babylon version. I am closer than I’ve been before, but I’ve been working on upgrading it all week and am at the point where I need to ask for help. – If you remember from before, I’ve been version locked at 3.1.alpha5 for over a year now. I’ve tried upgrading probably 12 times, and everytime there would always be huge FPS issues. (like half frame rate is best I ever got, and at worst, many memory leaks and <10 FPS).

So the good news is, it’s at least able to hit 60FPS sometimes now. The bad news is performance still sucks compared to 3.1a5 overall, and on top of that I have other bugs I still haven’t quite figured out (the performance increase is likely also due to the many fixes/optimizations I’ve done since my initial tries also) – I could open a bunch of threads but I figure centralizing it and tagging @Deltakosh and @sebavan would be the most humane thing to do, rather than clutter the forum with a bunch of topics explaining ^ every time. If you’d rather have me open a bunch of topics instead LMK.

Before we get to any general performance issues, there are still a few outright broken things I’d like to solve so I’ll post those first, and I’ll number each so you can just respond to the number. And I won’t post all of them at once. – Also I’ll create a PG for things I can create a PG for, after I ask the question, because there may or may not be an obvious answer to some of them.

1. Animation events no longer firing for me – Actually haven’t dug into this one too far, but basically I have a method like this: (which works 100% of the time in 3.1, but works 0% of the time in 4.0.3 for some reason

Object.defineProperty(BABYLON.Animation.prototype, "$$onPercentCompleted", {
  value: function(percentage, callback) {
    let max_frame = this.getHighestFrame(),
        target_frame = _.round(max_frame * percentage);

    let _event = new BABYLON.AnimationEvent(target_frame, callback, true);
    this.addEvent(_event);
    return this;
  }
});

//example of use below, basically never hits 0.5 % and thus never completes the promise I'm resolving within the animation
    return new Promise((resolve,reject) => {
        this.attack_animations.move_back.$$onPercentCompleted(0.5, () => {
          console.log('on 0.5 % complete');
          resolve() 
       });
   });

^ I figure it might be something obvious that was changed as I know there were a lot of things added to animations, but does anything obvious stand out that could have changed and cause that not to work any more?

2. General GUI issues, but mainly, AdvancedDynamicTexture not updating, particularly in my main full screen texture.

I’ve had to do the following several times, which I didn’t have to do previously, but the biggest source of confusion is why I have to do the onAfterNextRender callback to get it to update properly.

    this.gui = BABYLON.GUI.AdvancedDynamicTexture.CreateForMesh(this.mesh, 300, 100, false);
    this.gui.backFaceCulling = true;
    this.rectangle_control = new BABYLON.GUI.Rectangle();
    this.text_control = new BABYLON.GUI.TextBlock();
    this.text_control.text = "whatever"
    this.rectangle_control.addControl(this.text_control);
    this.gui.addControl(this.rectangle_control);

    this.scene.onAfterRenderObservable.addOnce(() => {
      this.gui.markAsDirty();
    });

everytime I change the button text, and after I initially set the text, I have to run the addOnce callback or it not only won’t update, but will actually show up blank, even though the underlying text block has the same text value. Any idea what that could be off the top?

3. Video Texture various issues
Lets call this a placeholder as I’ll have to dig into it futher, but basically video textures are playing slower, throwing uncaught errors, and are failing to play to completion and or are freezing when played. (The large majority of my videos I lazy load, but it happens very fast because of electron/local, and I’ve had virtually no issues in 3.1.5 w/ the lazy load strategy). – Anyways I’ll post more when I’ve dug into it further, but one specific thing that tripped me up hard when debugging to bring up now:

Video texture got switched to use fat arrow function binding on the class itself, which binds the methods at runtime each time it’s instantiated, and makes it hell to debug, inherit from, or overwrite. Literally none of those 3 things can be done except for me workaround below, which is super hacky. – In case you’re not familiar with the issue this post describes fairly well Arrow Functions in Class Properties Might Not Be As Great As We Think

But to illustrate my issue directly, I was trying to debug _createInternalTexture method, so I find the source method in the code and paste compiled code in (paraphrasing code). – Never hits the first console.log

function _createInternalTexture() {
  var _this = this;
    console.log("CREATING INTERNAL TEXTURE", _this.name);
    if (_this._texture != null) { }
}

Object.defineProperty(BABYLON.VideoTexture.prototype, "_createInternalTexture", {
  value: _createInternalTexture
});

So I try to inherit from BABYLON.VideoTexture, replace all my references to BJSVT with my VT, and do the same, still no luck –

export class DMVideoTexture extends BABYLON.VideoTexture {
  constructor(...args) {
    super(...args);
  }

  _createInternalTexture() { console.log('still wont get hit') }
}

Later I realize the why, is because:

getEventListeners(myvideotexture.video).canplay[0].listener === myvideotexture._createInternalTexture
=> false

Because when you bind like

private _createInternalTexture = (): void => {

Not even a subclass will be able to override the method call. And you can override the method with defineProperty, but it won’t actually work at runtime, as it will call the first one bound anyways – So my immediate question here is, can we please stick to:

addEventListener('canplay', this._createInternalTexture.bind(this))

in the constructors, because it’s a literal nightmare to work with / debug / comprehend what is happening otherwise. You basically have to overwrite the entire constructor AND the method if you are adding an event listener this way, because the parent constructor will never call the overwritten instance method even though it’s the lowest member of the ancestry chain.

(the only way to do w/o overwriting entire constructor it is my stupid trick I finally figured out below which I’m currently resorting to)

  function __createInternalTexture() {}

  set _createInternalTexture(val) {
    this.__createInternalTexture = __createInternalTexture.bind(this);
  }

  get _createInternalTexture() { return this.__createInternalTexture }

Anyways, there will be more to come but I’m tired and I figure I should probably annoy you with my questions in chunks rather than all at once :slight_smile:

Hello this is quite a big one :wink:
For all of the issues, we will need a repro in the playground to be sure to help you as fast as possible :slight_smile:

Some thoughts:

  1. Ok this is unclear to me but I see no change on our events so repro is mandatory
  2. Same. This works for instance: https://www.babylonjs-playground.com/#XCPP9Y#1108
  3. We did these changes to support treeshacking. But I’m absolutely fine to move to AddEventListener + bind. Please do a PR and I’ll merge it

Cool I’ll see if I can find time to put together a pr this weekend and thanks, I’ll look into trying a playground if can’t figure out

So update on issue #1 – Solved that, issue was that I was starting the animation, and then calling the $$onPercentComplete method after it had started, which apparently does not work in newest version. Actually haven’t checked against PG yet, but I changed my extended code to do it as part of the animation creation process anways, which fixes the issue

1 Like

Issue #4 - My mesh freezing strategy no longer works in 4.0.3 – I can’t reproduce this one on playground, but this is a big one for me, as I saw a fairly large performance improvement when I put this in originally, so it’s possible that once I get this working again that performance will be comparable to old version.

Anyways, since I can’t reproduce on PG, I’m wondering if you have any guesses, hunches due to changes in versions, or even an idea as to place to start looking to debug it – Basically what happens is: scene.freezeActiveMeshes() is working fine for me, when I call it in the console, nothing breaks. – But when I use my $$initializeFreezeCallbacks() method as seen in PG below, all the instanced meshes in my scene disappear, including any of their children. No errors or anything. The draw calls get cut in half, but the active meshes array is same length.

https://playground.babylonjs.com/#W2ZA5S#4

Any ideas @Deltakosh ?

We did a large rework of our internal architecture to boost performance of frozen scenes so it could be many things:(

Just checked your code… Nothing pops to my eyes
Let me check

Well I do not repro the problem of spheres that disapears in your PG. It seems to work as expected.

How do you get spheres to disapear?

Video here: https://youtu.be/_xDQkr3J_rc

Coolio, thanks for looking. No the pg is working correctly but in my actual project I’m having the issue with my meshes disappearing, but the code I posted to freeze the meshes is the same as in my local project. I’ll dig into it tomorrow and report if I figure it out.

As an update to #4, no good news to report, unfortunately. I am however optimistic that upon solving this, I may be able to match performance of 3.1 alpha, as before it breaks, I can see a very noticeable performance gain, just on the smoothness of the card hover animations alone. That said, the issue is more confusing than I initially though:

I can’t reproduce the issue outside of my actual game scene. The freeze strategy does seem to be working in my other scenes, however, something in the actual game scene is breaking it. Also to expound on the original problem:

Calling $$initializeFreezeCallbacks(), while there is nothing going on, seems to be working. It’s only after doing something, whether hovering on a mesh or progressing the game, that it breaks, BUT: it’s not right after, sometimes I can hover on a few, sometimes I can hover on all of them, and then at some point while progressing the game state it breaks – I generally can reproduce it after triggering hover states on X=random number of meshes in my scene, but not always.

Either way, it leads me to believe it’s somehow related to creating instance meshes, and/or cloning meshes, as my hover states create new instanced meshes when triggered if no hover state exists for the card. – It almost seems like some operation is failing to complete in its entirety before babylon starts to render, causing an invalid render that it is unable to recover from? The strange part is that it doesn’t break the entire scene, only the rendering of the instanced meshes, and they still seem to be there, just no longer getting rendered.

edit: actually may have finally isolated

False alarm, did not isolate, still completely broken

@Deltakosh – I still can’t reproduce in pg, BUT, I found the source of where things are breaking and I’m not sure what to make of it. – It is in _activate method of instanced mesh. I can confirm if I replace the method with the following, trigger the bug so everything disappears, then set $$force_true on one of the disappeared meshes, call unfreezeActiveMeshes(), mesh shows up again.

Object.defineProperty(BABYLON.InstancedMesh.prototype, "_activate", {
  value: function(renderId, intermediateRendering) {
    if (this._currentLOD) {
        this._currentLOD._registerInstanceForRenderId(this, renderId);

        if(this.$$force_true) { return true }

        if (intermediateRendering) {
            if (!this._currentLOD._internalAbstractMeshDataInfo._isActiveIntermediate) {
                this._currentLOD._internalAbstractMeshDataInfo._onlyForInstancesIntermediate = true;
                return true;
            }
        } else {
            if (!this._currentLOD._internalAbstractMeshDataInfo._isActive) {
                this._currentLOD._internalAbstractMeshDataInfo._onlyForInstances = true;
                return true;
            }
        }
    }
    return false;
  }
})

Any ideas what could be causing this? Also what is “intermediate rendering”?

Intermediate rendering is used when rendering into render targets

Progress on #4 @Deltakosh – Remember this issue Performance optimization for activeMeshes / memory leak issue with freezeActiveMeshes

?

I noticed this while debugging today, I thought it had gone away but memory leaking was back when trying to freeze. I assumed these two issues were linked before, as they are executing same code path, but I am not so sure anymore, because I figured out that issue and removed the leak, but freezeActiveMeshes still makes my scene disappear. Here is playground reproducing the memory leak/calling _activate even though scene is frozen issue. (It is godrays, so I removed them for now and it’s fine).

https://playground.babylonjs.com/indexStable.html#W2ZA5S#12

The meshes disappearing is still an issue, that I cannot reproduce outside my main scene, but I’ve been now debugging this for over a week and a half, not to mention the other 2+ weeks of time I spent trying to upgrade previously, so I could really use some help even though I can’t repro via playground. I need ideas on things to try or understanding as to why a mesh would stop rendering even though it didn’t leave the activeMeshes array. Stepping through it alone is very difficult because there are so many different render related methods in different classes/places, so it’s hard to know if I’m even looking in the right place.

The things I’ve narrowed down (which may or may not be totally accurate) are:

It seems to be occurring in _activate method of instanced mesh.

    InstancedMesh.prototype._activate = function (renderId, intermediateRendering) {
        if (this._currentLOD) {
            this._currentLOD._registerInstanceForRenderId(this, renderId);
        }
        if (intermediateRendering) {
            if (!this._currentLOD._internalAbstractMeshDataInfo._isActiveIntermediate) {
                this._currentLOD._internalAbstractMeshDataInfo._onlyForInstancesIntermediate = true;
                return true;
            }
        }
        else {
            //I THINK THIS IS CULPRIT, it's already marked as active so it returns false, and never render again?
            if (!this._currentLOD._internalAbstractMeshDataInfo._isActive) {
                this._currentLOD._internalAbstractMeshDataInfo._onlyForInstances = true;
                return true;
            }
        }
        return false;
    };

Which if I’m understanding it correctly, is to prevent a mesh that is already in activeMeshes array, from being rendered again? But that also seems to be what is causing it not to render if so, as if I mark the instanced mesh mesh._currentLOD._internalAbstractMeshDataInfo._isActive=false, it will render again when I unfreeze active meshes. However: It will only render itself, not any children, even though they should be active as well.

I was able to “fix” the issue (make things worse, but at least get meshes to show up again), by overriding this in _evaluateActivemeshes

        if (mesh.isVisible && mesh.visibility > 0 && ((mesh.layerMask & this.activeCamera.layerMask) !== 0) && (mesh.alwaysSelectAsActiveMesh || mesh.isInFrustum(this._frustumPlanes))) {
            this._activeMeshes.push(mesh);
            this.activeCamera._activeMeshes.push(mesh);
            if (meshToRender !== mesh) {
                meshToRender._activate(this._renderId, false);
            }
            if (mesh._activate(this._renderId, false)) {
                if (!mesh.isAnInstance) {
                    meshToRender._internalAbstractMeshDataInfo._onlyForInstances = false;
                }
               //forcing to false causes meshes to show up again. But it created 200+ more draw calls in the process
                meshToRender._internalAbstractMeshDataInfo._isActive = false;
                // meshToRender._internalAbstractMeshDataInfo._isActive = true;
                this._activeMesh(mesh, meshToRender);
            }
            mesh._postActivate();
        }

Any new ideas based on this?

You must understand that I cannot help with partial code. This is why a repro in the plyaground is mandatory. If I was you, I would have spent all the time you mention to build a sample in the playground that demonstrates the issue. Not being able to repro means it is in your code.

Worse case, if you can share a running example somewhere that reference babylon.max.js, I could try to dig into your code. But honestly. put yourself in my shoes if I asked you to help me with just a few lines of code without a repro

1 Like

Why do you say this one is leaking? How do you see it?

Pinging @julien-moreau as he is the author of the godrays (FYI)

Haven’t read the code yet (on my phone) but I can see the sphere disappearing some few seconds when I:

  • rotate the camera enough so the sphere is culled
  • return to the previous rotation and target the sphere

Once I re-target the sphere, the sphere is not visible and reappears like 1s later.

Reading the post it looks like it is the issue you are looking for. Have you reproduced @Deltakosh? I’m on an iPhone 7+ now

1 Like

The leak and the disappearing issue I believe are two separate issues. You can see the leak by looking at the console log, I am summing the items in sourceMesh.visibleInstances array. Normally only 1 array is kept around, the most previous renderId. When godrays are in scene, the number of arrays grow with every render, a new array added each time. And will grow exponentially by the number of source meshes in the scene I believe. But I don’t think it’s causing the disappearing act. –

@julien-moreau that sounds similar, except I never see the meshes again after they’ve disappeared unless I manually force that internalAbstractMeshData.isActive value to false and unfreeze manually.

I think I finally have a reproduction of the disappearing issue however.

@Deltakosh

I believe this replicates what I am seeing. Two different ways to replicate, first is more realistic probably. (realistic as in I’m not actually adding meshes in the afterRender callback like in second playground).

In this one, click the center sphere fast enough/enough times, and it should cause all the meshes to disappear eventually.
https://playground.babylonjs.com/indexStable.html#W2ZA5S#29

In this one, I can trigger consistently by doing before render/after render. You should see the meshes appear initially, but disappear very quickly after.
https://playground.babylonjs.com/indexStable.html#W2ZA5S#22

I finally narrowed it down in my own app to be pure timing based which is why it was so hard to reproduce, and why nothing made sense. But basically once you trigger it, all the instanced meshes will disappear unless you force that value back to false so that it reevaluates

both PG does not load for me because of a missing _ function