Dynamically Loading in Materials

[johntdaly7]

Hey everyone,

So I am trying to figure out a way to decrease our large furnished-office-space models, beyond just compression, and the current angle we are looking at is dynamically loading in furniture sets and materials upon in-app selection.

I can load in another glb easily enough, but what I am struggling with is figuring out how to dynamically load a material. Correct me if I’m wrong, but it seems for the most part a material must be associated with a mesh in Blender. Because of this, we have created a lot of duplicate meshes just to carry variant materials based on design selections.

So, in Blender, when exporting a glb/gltf, we are looking at creating an extension that decouples the duplicate meshes that carry material variants for the same type of object (ie a chair) from the scene, and instead, create a reference JSON that links a URL of a texture, its associated property data, UVMap, and shader data, etc. to the mesh. Then, when in Babylon, when that material is selected, I want to create a new PBRMaterial on the fly by loading in the texture and setting all the associated property values. My issue, however, is that when grabbing data from the .gltf, the UVMap and atlassing data needed to create the material is obviously missing (in the .bin I presume).

Does anyone know where I can find how Babylon reads in all the data necessary to identify and create a Material from the gltf/glb binary?
I’ve been looking through the github here: Babylon.js/babylon.glTFFileLoader.js at 6eddbac20810c74a2a6e08ba632f50842ae83f07 · BabylonJS/Babylon.js · GitHub
but there’s so much going on that I could use a bit of direction.

Also, during my research into this process, I discovered the Khronos extension for material variants, which is cool and probably useful:

Streamlining 3D Commerce with Material Variant Support in glTF Assets

However, I am wondering if, given the size of our office building models with furniture, it would still make sense to decouple the variant materials from the geometry and load them in as needed based on what the user selects. In which case, even if I use the Khronos extension, I need to figure out how to load in a material (shader, texture, properties, UVMaps for atlassing, etc.) dynamically (possibly from a JSON file that was created via our export extension in Blender) and create it during runtime.

Finding out what to extract from the .gltf and especially .bin when exporting from Blender, based on how the exported material data may be read/dynamically created on the Babylon side, is the hard part, and any guidance would be very helpful!

[sebavan]

Adding @bghgary as I wonder if Variants could be split out of the main material ?

[johntdaly7]

Thanks! And even if they can be or can’t be, just figuring out all the data in the .gltf/.bin that I would need to extract in order to create a new PBRMaterial in Babylon from scratch based on the variant is what I am probably looking for.

@bghgary, I noticed you are involved in the Khronos forum as well. Do you think it would make more sense to ask this question there? I figured I’d ask it here first because I think this implementation is reliant on how Babylon reads in the data from the file.

[bghgary]

@johntdaly7

I can load in another glb easily enough, but what I am struggling with is figuring out how to dynamically load a material. Correct me if I’m wrong, but it seems for the most part a material must be associated with a mesh in Blender. Because of this, we have created a lot of duplicate meshes just to carry variant materials based on design selections.

I don’t know much about Blender, but can you associate the material with a super simple geometry just to carry the pay load for the material?

@johntdaly7

So, in Blender, when exporting a glb/gltf, we are looking at creating an extension that decouples the duplicate meshes that carry material variants for the same type of object (ie a chair) from the scene, and instead, create a reference JSON that links a URL of a texture, its associated property data, UVMap, and shader data, etc. to the mesh

This sounds exactly like the material variants extension you already found.

@johntdaly7

Then, when in Babylon, when that material is selected, I want to create a new PBRMaterial on the fly by loading in the texture and setting all the associated property values.

This is interesting. The current implementation loads all of the materials on load to avoid having to wait when switching materials, but I can see how that isn’t always the right thing to do. I wonder if we can add a flag to the existing implementation to load the material async upon request instead of at load?

@johntdaly7

My issue, however, is that when grabbing data from the .gltf, the UVMap and atlassing data needed to create the material is obviously missing (in the .bin I presume).

I’m not sure what this means. What do you mean by “grabbing data from the .gltf”? Are you trying to parse the glTF yourself? Can you explain this more?

@johntdaly7

However, I am wondering if, given the size of our office building models with furniture, it would still make sense to decouple the variant materials from the geometry and load them in as needed based on what the user selects. In which case, even if I use the Khronos extension, I need to figure out how to load in a material (shader, texture, properties, UVMaps for atlassing, etc.) dynamically (possibly from a JSON file that was created via our export extension in Blender) and create it during runtime.

When you say “decouple”, do you mean separate files? You can already do this. GLB files are typically a single file, but it doesn’t have to be. You can reference external buffers with a GLB. For example, you can put all of the binary data for each material in a separate buffer with a separate url. Another choice is to use glTF and not use GLB at all.

@johntdaly7

Finding out what to extract from the .gltf and especially .bin when exporting from Blender, based on how the exported material data may be read/dynamically created on the Babylon side, is the hard part, and any guidance would be very helpful!

I think given what I understand of your scenario, a good solution would be to generate a glTF/GLB with external references for each material and modify the Babylon.js glTF loader extension for material variants to support lazy loading of each material.

[bghgary]

@johntdaly7

@bghgary, I noticed you are involved in the Khronos forum as well. Do you think it would make more sense to ask this question there? I figured I’d ask it here first because I think this implementation is reliant on how Babylon reads in the data from the file.

I’m not sure. It depends if your question is about the Babylon.js implementation or something about glTF in general.