What are the best practices for environment meshes made for Babylon?

Questions
1

My prior experience is with desktop and 2D projects, so this is a new world to me. We’ve outsourced the modeling, but the modeler’s niche is architecture, not games, meaning he’s used to creating high-fidelity content and rendering static shots. I’m trying to figure out what to tell him in order to get meshes that load and run reasonably well on the web, on mobile, in VR. Things like:

  • file sizes;
  • vertex & triangle count;
  • materials, etc.

I won’t know everything before I test it extensively, but I want to figure out a starting point. Get some context.

Current Stats

File Size

The scariest thing is the total file size of 48 MB. I feel like that’d take ages to load on mobile connections. What’d be a good size to aim for, generally?

Vertex & Triangle Count

In total, we have 150k vertices and 170k triangles. I’m guessing we should try to reduce those numbers to at least below 100k.

Materials

The 3D Viewer app on Windows says there are 267 Material IDs. I’m not sure if that means there are 267 unique materials, but if so, that looks like an enormous number. But if that’s indeed the case, I can see a lot of elements that could reuse the same material—low-hanging fruit here.

I ask this because I’m still trying to wrap my head around mesh optimization. I’ve been reading up on the Scene Optimizer and auto-LODs, but these target runtime performance, not loading time.

There’s also progressive loading for GLTF files, but it seems MSFT_lod is all but unsupported by major 3D software. Incremental loading looks promising, and I’m probably going to implement it, but a large portion of our models can be seen from the spawn point, so I don’t know if that’ll be enough.

So, my main question is: should we figure out a default, high-fidelity level of detail for the models—the same way artists work on large images and downsize in exports—and subtract from that for lower-end devices? Because I said the we need the app to run on mobile and in VR, but it could also be accessed from a desktop, without VR.

PS: Speaking of GLTF, is that the recommended type for Babylon?

1 Like
2

Pinging @PatrickRyan and @PirateJC

I do think the high-fidelity for desktop and lower fidelity for mobile/VR is a good aim - trying to adapt to your expected user is always a positive!

GLTF is quickly becoming the 3D standard everywhere, including in Babylon, and for good reason, it’s a very well planned format.

4 Likes
3

First you need to optimize your model at model level. Check if there are some objects with repeated geometry and use instances on them (or even, with some extra code, thin instances).
Check why do you have so much materials; I believe that you can reduce their number in 10 times, so you’ll have at the end about 30 materials.
Check if these selected materials really need a texturing or it is enough just to use their PBR properties without undesired visual degradation.
Seems that in your case you’ll need mobile detection and different models for mobile and high-quality version, so you will be able also to use smaller textures for mobile.
To reduce load time you may put into GLB only geometry and materials and load relevant textures async.
So, shortly, there is a lot of what could be done with your model even before it will need programmatical optimisation for Babylon use.

3 Likes
4

Yeah. Try to replace duplicate assets with instances is an important step. You can do this in Blender or 3DMAX. At the bottom of this page.
Instances | Babylon.js Documentation.
We can simply do a test You can find that using instances can significantly reduce the file size.

20211209231637
202112092316371334×702 125 KB

39968kb reduced to 184kb :rocket:
Maybe we have more articles on similar topics?
glTF Tips for Artists – Cesium
The number and resolution of textures also affect asset size and performance.

3 Likes
5

Hey @VerifiedTinker

First - your username is the BEST! LOL I love it!

Second - thanks for bringing up this topic. I love that you’re getting your feet wet with real time optimized 3D asset generation. Your instincts are already really solid. As you’ve pointed out, high poly/vert count as well as having lots of materials will increase file size which decreases load time.

All of the things you’ve mentioned are great things to investigate.

Now for the hardest part of the answer. There is no one single way to learn how to create real time assets, it’s an endless space of tinkering (see what I did there? LOL) and experimenting.

The process for creating real time assets is also not unique to Babylon. Babylon is an awesome rendering engine, but it will have some of the same pros and cons as any real time engine.

This is good news because it means that there are COUNTLESS resources out there that can teach tips and tricks of how to create great assets for real time environments.

I know this is kind of a cop-out answer, but doing a search in YouTube for creating game ready assets will be an awesome place to start.

Then as @carolhmj mentioned, getting your 3D objects into the .glb or .gltf format will make them load and run fast in Babylon.

I hope this helps a tiny bit!

4 Likes
6

@VerifiedTinker, the points that @labris and @musk bring up are great ways to optimize your model, but the core of the problem you are describing goes back further than just optimization. If the artist you are working with specializes in pre-rendering (rendering with a ray tracer to produce a single, high-fidelity image or a sequence of high-fidelity renders for video) they are not likely a good candidate to use for real-time rendering. The reason is that the optimization for your asset starts at the planning stage. You can use tools to reduce your file size after the fact, but the truth is that an artist that specializes in real-time rendering is going to produce an asset that is far more optimized before you even get to the final performance tweaks like compression.

Planning for a real-time asset will ask the following questions:

  • What is the target device (desktop, mobile, HMD, console) and what are the performance limitations of the device my users will be on?
  • What is the shading model, PBR, Blinn-Phong, Cell Shading, Custom Shading? How would the chosen shading model influence my texture creation or material breaks in the model?
  • What materials in the model need advanced rendering techniques like transmission, clear coat, sheen, etc. and how will that affect my material breaks? How can I batch materials with similar requirements to save on material counts?
  • What is the triangle budget of my asset? Is it a hero asset that will be close to the camera and needs extra resolution or detail or is it an environment asset that will be far from camera or not on screen for long?
  • Where do I spend my triangle budget? How can I maximize my budget for areas that support silhouette or parallax in the model while stealing triangles from other parts of the model that do not need the resolution to support the silhouette or parallax?
  • Where can I eliminate triangles and what tricks can I employ? Can I delete faces that are not seen? Manifold geometry is not a must so there are always places with unneeded triangles. Can I float a piece of detail over simpler mesh? Sometimes you can get away with a smaller triangle count if things like panel lines are slightly floating above a surface rather than embedded in the surface since you don’t need any of the supporting edges to support the higher-frequency detail. Can I just collide submeshes to create more complex silhouettes rather than making it manifold? For example, a cylinder colliding in the middle of a quad is fewer triangles than adding support loops/edges to make the two meshes manifold.
  • What detail can I bake into a tangent space normal texture to save triangles? Your normal texture can do a lot of heavy lifting from baked detail from a high-resolution mesh to adding material-level high frequency detail like pits, scratches, rust, grain, pebbling, etc. This detail is usually the type that does not rely on parallax to look believable, does not affect the silhouette of the model, and adds realism when the camera is close to the mesh. However, you also have to plan how to use your textures well as high-frequency detail often needs high resolution textures to be effective. You also need to plan for how close your camera will be to the mesh to determine the largest texture size you will require so you don’t see artifacts. This would be your largest texture LoD if you are implementing an LoD system.
  • What is the largest texture size I can use? Just because you can render an 8K texture does not mean you can use one. You need to check to see if there are limitations for your file format (4k for glTF) or you may decide that the trade-off for extra texture loads of smaller textures gives a better result than fewer large textures because you can get better material breaks and higher texel density with the smaller textures.
  • Can I use tiled textures or do I have to have altased textures? Or can I use a combination of both? Tiled textures are going to be smaller, but do not allow for things like AO or lightmap baking. You also may need a unique unwrap for another reason and so can you make use of multiple UV sets to give you the ability to use tiling textures for their smaller size and greater texel density while still being able to have an atlas for baking AO or lightmaps.
  • Can I use non-square or non-power-of-2 textures? The engine you are rendering with sets this limitation, but if you can use them (as you can with Babylon.js) using a non-square texture may give you a better UV unwrap by making better use of the space and maximizing texel denisity. A non-power-of-2 texture may also allow you to keep the target texel density for your scene while just using a smaller texture. Why save a 1024x1024 texture if you only need 800x800 pixels to hit your target texel density?
  • Do I need to skin and rig this asset? Are there things that need to be added, parented, or swapped in the model at any time? Do I need to share animations across models? Are there customization options to the model that need to be accounted for in materials or textures?
  • How can I combine meshes to reduce draw calls while allowing for needed material separations due to advanced rendering requirements? Should I combine meshes based on maximizing UVs? Is there a hybrid solution in there?

I have converted many pre-render models to be real-time models in the past and it is a lot of manual work and in many cases easier to start from scratch. The main reason is that pre-render models pay technical debt with time in render. You can have hundreds of thousands of triangles and hundreds of materials that only contain factors for color, roughness, and metalness because you are only impacting the time for the render to complete which does not impact the final product for the user. In real-time rendering, however, you pay your technical debt with your frame rate. If your target is 60 frames per second, but you have too many triangles, draw calls, textures, etc, you will see your frame rate drop especially on low-end devices. This does impact your end user as the experience will feel choppy or broken. For HMDs, you need to render at even higher FPS counts like 90 and above to make the experience feel smooth.

In the same way you would not go to your dentist for an appendectomy, you should choose an expert in the type of rendering you want to do. Find a pre-render expert for static renders or pre-rendered video. Find a real-time rendering expert to get the most out of your real-time experience. And if your pre-render expert is going to be your real-time rendering expert, then they will need to expand their toolbox as I have seen many pre-render experts be surprised at how different the workflows are for real-time rendering.

There are resources out there to walk you through some considerations for best practices like 3D Commerce Working Group Releases Real-time Asset Creation Guidelines to Assist Artists Create Efficient, Reliable Models for Retail and E-Commerce - The Khronos Group Inc which is mainly setting guidelines for single products in a viewer. But looking at game art forums, and online training for game art is likely needed to fully understand all of the levers that need to be controlled in real-time rendering.

I hope this helps start the process of evaluating your art pipeline because taking a measured approach to your assets from the start will save you a lot of time in the end trying to find ways to massively optimize your assets.

13 Likes
7

This answer must go somewhere into the Documentation, I believe :slight_smile:

4 Likes
8

I think we are talking about doing a post expanding on this for Dev Stories.

5 Likes
9

YES
YES
YES
YES

2 Likes
10

Amazing info @PatrickRyan

I’d like to add: one of my frustrations, having had to optimise many bought 3D assets myself (often that I haven’t chosen myself), is that even marketplaces that have filters for “low poly”, “real-time” and/or “VR/game ready” like Sketchfab and TurboSquid, are often far from it. Maybe I’m just getting old, but many people seem to have forgotten what these terms really mean, and how to create a quality 3D asset that will work well for real-time 3D rendering.

3 Likes
11

Haha, thank you. It’s a book reference.

I agree completely! I brought up the same concerns, but was told the artist is out of our hands, so we’ve got to do the best with what we have. Which is why I came searching here, so I can relay my findings to the artist. This wouldn’t be my job in a perfect world, but I’d rather put in the time than trust in a stranger to do good work.

Thanks for the ton of info. Exactly what I was looking for. One question: you mentioned lightmaps; does Babylon come with tools for baking them?

The only mention of lightmaps I could find in the docs is in the Unity toolkit, and while I love Unity, I’m still not sure if I should use the toolkit, given that I made my way to Babylon in part due to Unity’s shoddy WebGL support. Probably out of this question’s scope, though.

Edit: I found something. But it’s about using lightmaps, not creating them, so I suppose not.

12

Babylon.js can ingest light maps but we don’t bake them. I would typically just bake them in my preferred DCC (digital content creation) tool like Maya, Max, or Blender and then load them into my Babylon scene. This thread has a good breakdown of my process for them.

2 Likes
13

You can try to optimize your model with https://rapidcompact.com/
Here are some demos - Demos & Showcases - RapidCompact Community

2 Likes