$500 if you can Diagnose and help to fix this crash that is happening on iPhone only

We’ve got an app called Lightning Bridge. It’s a multi asset crypto gateway with a UI based around Babylon.js. We’ve never had problems deploying to mobile, but recently our app crashes almost immediately when loaded on an iPhone 13 Pro on Safari and Chrome. It doesn’t crash on an iPad Pro. It doesn’t crash on desktop. I’ve been able to remote debug the iPhone, but the debugger detaches as the crash happens, so that I haven’t been able to see the cause. Our app loads various 3D meshes, images, and sounds at startup. It may be related to this but I am not sure.

Acceptance Criteria:

First, diagnose the problem. What exactly is happening when we crash. Second, provide a solution so our app will load in iPhone, or at least detail the reason why it cannot in it’s current state.

Resources:

We’ve deployed a simulation mode version of our app here:

It bypasses user login and wallet connection. The UI is partially functional but it will not do any real transactions.

To trigger the crash, press a few buttons on the app. 1. Select Send or Receive Token or Network. 2. Press the Switch button. 3. Press the Send Button. That one should take you to a confirm screen. Press Confirm and you will see either a progress animation with 1s and 0s flying through the air, or else a prompt to approve a wallet transaction. This is as far as the app goes in simulation node. The app usually crashes either just after loading or when pressing a few of these buttons. Sometimes it will go quite a while without crashing though, while sometimes it will crash almost immediately.

I think it might be possible to diagnose the problem by remote debugging an iPhone loading this site. If you want access to the code, please contact me and we can set that up. Likewise, if you want to try a live version of Lightning Bridge, we can set that up too. The fastest way to reach me in on Telegram. My username is james_global_liquidity.

2 Likes

It might be a memory leak of some sort resulting in a too big memory usage this is usually the culprit on those devices. Like too big textures, too many of them and such. Good luck to the bounty hunter !!!

I’m actually looking at that right now. I suspect it may be too many PNG images. Disabling gzip compression on them improved it a little. Maybe I’ll collect my own bounty. :slight_smile:

Thanks!

AhAh :slight_smile:

No one ever does but connect your phone to a mac physically and open XCode and you can see its console and the moment it crashes. It’s depressing for me the reason why but perhaps yours is a fixable message !

2 Likes

i only have an ipad and macmini so no bounty hunting for me… hope you find it or someone can help. I know it is frustrating.

Thanks for the replies. And thanks for the lead @sebavan. The problem is much reduced after using 128x128 PNGS for my icons. They had been 512x512. I’m calling this closed.

@br-matt I did that already and I’m able to see my iPhone logs remotely, but the whole thing goes away as soon as the crash happens, so no useful information that way.

1 Like

yup memory issue makes it freak out. May I ask if it only started happening recently ? as we had quite a few similar reports and I am wondering if we are doing smthg wrong or if apple does ? I am leaning towards them as it does not repro on other browsers

I mean connecting it to XCode, not the inspector of safari which yea, loves to quit like that

1 Like

i found these test scenes on github. babylon’s memory use is much higher compared to three. if you open both pages, you can see three’s peak memory use is about the same as bjs but then three frees up memory after everything is loaded, where babylon doesn’t. I didnt try very hard, but i did try a few things to get bjs to free memory and couldn’t.

three: 174mb
babylon: 940mb
three lite: 91mb
babylon lite: 533mb

.GitHub - AstroPilot97/babylonjs-test-scene: A 3D performance test scene built using Babylon.js library and utilising WebGL2 renderer.
.GitHub - AstroPilot97/threejs-test-scene: A 3D performance test scene built using Three.js library.

Of course, freeing memory isnt always desired and usually a tradeoff between time and space complexity. I prefer to have things run faster at the cost of more memory, but maybe some other default on mobile could be appropriate, if even possible.

2 Likes

Turns out my app still crashes, but now it’s always after opening one of our selection menus ten times, and so it’s almost certainly a leak.

I feel like I’m really bad at destroying objects in TypeScript.

I think the problem is here:

This is the List Item class. Each time we open our menu, we clear it and rebuild it. That means we call destroy on each list item, which is intended to free the memory we use for the texture. It does successfully cause the item to be gone from the scene tree, but I think it’s not actually freeing up the texture memory. In any case, I can improve this design by caching the textures so we are not newing them each time we open the menu.

1 Like

Hi, I wanted only to say that you are right. :sweat_smile:
You are using AdvancedDynamicTexture which is a dynamicTexture under the hood, and …TextMeshString is a dynamic texture too?

If yes, I’m pretty sure that this is the issue, and I recommend you to use an atlas texture and SPS, or try to avoid recreating them too often, even update

3 Likes

You are totally right. It’s the AdvancedDynamicTexture. Without creating one, I’ve got no issues. As soon as I create one, I get crashes on iPhone, but never yet on Desktop and never even on my iPad.

So what can I do here? My goal is to make these buttons work. I don’t want to use a single full-screen gui, because I run in to so many problems accounting for every screen dpi/resolution. This is why I’m using the AdvancedDynamicTexture here. It’s so I can use the 2D GUI buttons, attached to each of these button plane meshes. When I do it this way, I get perfect rendering on ever device.

Should I be using MeshButton3D here maybe? It seems like a lot to have to set up the whole 3D gui system just to be able to respond to clicks/taps on a plane, but maybe that’s the right approach?

You can also play with the live deployment that currently does not crash, but that also has no button responses at : https://mobile.lightningbridge.com

PS Along the way, I did manage to optimize some things. I thought it was my png textures crashing me. Before I was loading them from a url each time I create the panel. No I load them as Textures using the AssetsManager at app start, and just pull them from there each time I need them. Much nicer.

sorry if im missing somthing … but why are you using a 3d engine to create 2d gui anyway?

i understand this in the context of 3d gui in 3d scenes , or engines where you dont have any other alternative but this is a web app. ?

Have you tried to use its optimizations on the controls and on the dynamicTexture of your GUI

control.useBitmapCache = true
advancedDynamicTexture.useInvalidateRectOptimization = false

Maybe that could help.

Hi @shaderbytes. It’s a great question, and one that I have spent a lot of time debating in my own mind, event to the point of prototyping our current UI using a standard html/css based react component.

The TLDR is,
“It comes down to the “feel” that we can get, when using a 3D engine, with a particular appreciation for babylon’s excellent, PBR support, as compared to anything we can do using html/css or even a 2D engine like Pixi.js”

And if you want to hear the full story, here it is.

In a very real sense this is a 3d gui.

Our text is driven by our “TextMeshString” which uses pooled and cached instances of high-poly extruded True Type font characters, that we create in Cinema 4D. All of the various borders, including the outer bezel, are 3D meshes, and everything has PBR materials. Lighting for the scene is driven by a custom HDR map. This way, we get an interface that feels distinctly not like a web page. Each surface has a specific metallic/roughness that combines with the HDR lighting to give all these little glints and highlights across the scene, giving me the look I want. Also, we do use “3D” effects all over as the user goes through the process of transferring money using Lightning Bridge. Various meshes appear and disappear with animation, and we’ve got this cool SPS based indicator that we show when we are waiting for a response from the server. A cloud of 3D ones and zeroes floats by. And then there are these subtle effects that I can do because of our 3D/PBR. When the user first connect’s his wallet, I animate a spot light so as to gradually illuminate the scene, over 500 ms. It feels nice. It makes the scene feel more alive and less static.

There is also practical reason, particular to my myself and my small team, that makes it appropriate for us to use this technique vs a more traditional html/css workflow.

We’ve been working with babylon.js for close to four years now, on several products that we have built as a company, on the way to our current “Lightning Bridge” product. These were each more obviously 3D dependent than our current product. For example, this clip of TRADE : The Game, shows our use of babylon.js to create this social multiplayer game, based on real-time competitive simulated trading of the “order book” aka “level 2” data which represents real trading activity. In this case, it is the market for Bitcoin vs Dollars on Binance.

Along the way, we have developed our own internal framework for building reactive, data-driven 3D interfaces using babylon.js,with. Combined with a tone.js based sound engine, and an xstate based logic engine, it’s become quite powerful over the years. It’s called GLSG for “Global Liquidity Scene Graph.”

Back in 2019, I posted about it here, with the idea that I might open-source it. I never did it it yet, but I’m still very open minded about doing that, if I ever find a few people willing to help me to do it.

With our other products, I’ve always designed their interfaces to be forward-compatible with VR and AR modalities. What that means mostly, is that we have designed our user interfaces so that they can be rendered with a single camera’s view. For 2D style heads-up display elements we composite into a “real” space out in front of this camera. This way, when moving to VR/AR, we can easily attach these elements to the user’s field of view, with inertial smoothing, and we get a user interface that works particularly nicely in VR/AR but that also projects nicely to a 2D screen. A while back, I realized that we can design for the screen and for VR at the same time, by regarding the screen as a portal into a 3D-space, and with a 3D UI projected “just right” so that it fills the dimensions of the screen. Then I realized, that even without considering VR/AR, that this a cool way to make a “2D” UI.

For example, one of GLSG’s features is our “SceneUI” system. Using it we can make a hierarchy of “UIControls”. Each of these is made up of one or more babylon.js based 3D elements. The Scene UI system projects these controls out in front of the first-person camera, using a docking system that allows us to snap these controls to the edges of the view frustum. With this simple docking system in place, we can start to compose scenes, and we can define regions of the screen as “stages” that express 3D activity, such as using Babylon’s animation system to create transitions based on positioning and/or scaling UIControls.

I think there is a bright future for this approach, especially as we start to see devices hit the market that will demand that interfaces are 3D aware. I also think that Babylon is positioned to be in important building block for this new era of interfaces.

1 Like

That was such a great guess, but no, it still crashes just the same. Thanks for pointing me to these optimizations though. I wasn’t yet aware of them.

It works fine on iPhone 11 iOS version 16.1 besides some missing textures.

I think the issue is from using too many advanced dynamic textures.
I had the same issue on iOS, but not with a total crash, just black regions.
I fixed it by using fewer advanced dynamic textures.
You can also use a lower resolution for each texture: bjsgui.AdvancedDynamicTexture.CreateForMesh(this.buttonPlane, 512, 512);
Try to convert your PNGs to ultra compressed JPG with https://squoosh.app/ if transparency is not an issue.

Are you pooling and recycling the 0s and 1s?

thanks for the detailed answer. I appreciate your view on your workflow and possible extended use into xr etc.

Thanks. This is also great advice and I was also getting crashes before when my PNGs were too large.

I ended up fixing the problem today by using MeshButton3D, which is what I probably should have started off using.

As far as the 0s and 1’s, yes. It’s an SPS with a set number of instances and I reuse them.

Thanks again!

2 Likes