Hi guys,
I would like a option to include textures in .babylon files exported via sceneSerializer,
would this be possible to add to the core? 
BABYLON.SceneSerializer.Serialize(scene, withTextures?);
BABYLON.SceneSerializer.SerializeMesh(toSerialize, withParents?, withChildren?, withTextures?);
Would not require a change to the format at least, as it is being done from blender. The data would have to still be available on the cpu, and I am not sure it is. The un-decoded data as it existed in the file would be needed, and I do not think it is ever, always in JS.
Babylon.js/fileTools.ts at master Ā· BabylonJS/Babylon.js Ā· GitHub
I know conversion from to and from B64 is built into javascript, but cannot remember where.
yeah i was thinking just a base64 string included in the .babylon file
of course itās larger/size than separate binary image files but SO much easier to keep track of a single file during development at least.
Slowly starting to look at this,
@JCPalmer @Deltakosh
Iām looking at the babylon format itself, is there no āsmartā/global textures array support?
all i can find is material.xxxTexture = serializedTexture;
which is sub-optimal for base64 if multiple materials use the same texture, itāll be stored multiple times
Yes this is one problem. We would need to have a smart way to say that this texture is already stored and should not be stored again
My idea could be to keep the texture name only but add a new block in the babylon file with the base64 files indexed by the texture names
Personally, I am going the other way, meaning I plan on nuking all vertex data that has not been set as update-able once the scene is ready. Would also like to trash any texture data on the cpu, if possible, making later serialization impossible of course.
Interested in quantitatively reducing GC episodes on very small devices like Occulus Quest by providing the biggest area for the heap. Not sure texture data is on the heap, but a couple of loops to trash the references to this garbage should not have any negative impact.
Loading from browser cache 
if the texture url is saved and the image has been cached, it should work.
on/after texture load itās possible to delete the texture._buffer again.
But this is looking like a much bigger job than expected at first, many things need updating and serializer needs async support to wait for texture load before returning a base64 string.
I had to wrap the entire serialize process, preloading textures into each their own canvas before calling the serializer, and within texture.serialize(), check & get base64 string from itās canvas.
but it works
In my search to support multiple texture types/buffers,
I found the texture.readPixels() function,
refactored my code & got it working, passing pixels to a canvas, then converting to base64 without need for async functions or preloading textures from cache.
BABYLON.Texture.prototype.toBase64 = function(){
if(!this._texture || this.readPixels === void 0) return null;
const pixels = this.readPixels();
const width = this.getSize().width;
const height = this.getSize().height;
const canvas = document.createElement('canvas');
canvas.width = width;
canvas.height = height;
const ctx = canvas.getContext('2d');
ctx.fillStyle = 'rgb(0, 0, 0)';
ctx.fillRect(0, 0, width, height);
let wCount = 0;
let hCount = 0;
for(let i = 0; i < pixels.length; i+=4){
if(wCount > width-1){
wCount = 0;
hCount++;
}
ctx.fillStyle = `rgba( ${pixels[i]}, ${pixels[i+1]}, ${pixels[i+2]}, ${pixels[i+3]/255})`;
ctx.fillRect(wCount, height - 1 - hCount, 1, 1);
wCount++;
}
return canvas.toDataURL('image/png');
};
Now if i export a GLB mesh to .babylon with a base64 texture, it turns white when importing it again.
Iām having a tough time figuring out what causes it, the image data is present & it should workā¦
Edit 1; PG
https://playground.babylonjs.com/#GDGCSP#2
There is also a slight color/light difference from the standardMaterial to PBRMaterial convertion by GLB exporter. (@Drigax is this behaviour expected? )
Edit 2;
It looks related to PBRMaterial, rather than the textureā¦
Applying the b64 texture from PBRMaterial as diffuseTexture on a StandardMaterial works fine.
Loading & applying a new texture as albedoTexture on the PBRMaterial is still white.
Edit 3; ā¦Success?
It appears the problem was related to energy conservation,
more specificly the āuseSmithVisibilityHeightCorrelatedā option
https://playground.babylonjs.com/#GDGCSP#3
(texture is also flipped, but this is mostlikely a bug on my end)
Now this opens a new question,
Why would this work on a .glb mesh, but not a .babylon meshā¦
the materials are identical, useSmithVisibilityHeightCorrelated is also turned on, on the .glb material.
Furthermoreā¦ attaching .glb material to .babylon mesh without changing anything in the material, works tooā¦ this is too weird 
Maybe something happens in the PBRMaterial serializing?
Fixed texture flipping.
BABYLON.Texture.prototype.toBase64 = function(){
if(!this._texture || this.readPixels === void 0) return null;
const pixels = this.readPixels();
const width = this.getSize().width;
const height = this.getSize().height;
const canvas = document.createElement('canvas');
canvas.width = width;
canvas.height = height;
const ctx = canvas.getContext('2d');
ctx.fillStyle = 'rgb(0, 0, 0)';
ctx.fillRect(0, 0, width, height);
let wCount = 0;
let hCount = 0;
for(let i = 0; i < pixels.length; i+=4){
if(wCount > width-1){
wCount = 0;
hCount++;
}
let drawHeight = hCount;
if(this.invertY){
drawHeight = height - drawHeight - 1;
}
ctx.fillStyle = `rgba( ${pixels[i]}, ${pixels[i+1]}, ${pixels[i+2]}, ${pixels[i+3]/255})`;
ctx.fillRect(wCount, drawHeight, 1, 1);
wCount++;
}
return canvas.toDataURL('image/png');
};Ok.
The root of the white PBRMaterial problem came from PBRMaterial.environmentBRDFTexture not being the same as scene.environmentBRDFTexture
Can anyone confirm how these are supposed to act?
Iām not very familiar.
Is there only 1 allowed per scene?
Is it supposed to get serialized?
How is it ment to react on import/parse?
@Deltakosh (sorry for the many pings) 
Few things, yes there can be more than one environment texture per scene. It is rare, but each PBRMaterial is has a spot to assigned one, so they are not forced to be the same, at least I hope not. That is one of my back up plans for something.
One the other hand, there is also one for the scene, which is used by default and or skybox, if turned on. I also added the scene env texture to the .babylon format when I re-did the Blender exporter for 2.80.
BUT, I only added it as a separate file to be specified. You cannot currently specify an environment texture in base64, so I did not add for it in .babylon format. Also, the .env format is a special thing. Not sure it was ever made to serialize.
Might be a colorspacing issue, @PatrickRyan, @sebavan, when do we use linear vs gamma corrected color spaces for our materials/textures?
@Drigax, use gamma space for any color map like base color, emissive, sheen (color), clear coat(color) or something that is actually used for color (in terms of custom maps). For non-color data, use linear which would be AO, Roughness, Metallic, Normal, etc.
Also, IBL files also need to be read as linear as they are usually 32-bit to start and you donāt want to apply a gamma curve to that data.
@JCPalmer
PBRMaterial.environmentBRDFTexture (same as your link) is the property it use and gets serialized from.
(before serializing it points to scene.environmentBRDFTexture,
after import itās separate and not working)
itās odd since default scene.environmentBRDFTexture also have a base64 buffer from what i can see.
and the environmentBRDFTexture does have a serialize function.
But then the mesh serializer should not pass the environmentBRDFTexture for PBRMaterials, only for full scene serializerā¦
@Drigax
I notised diffuseColor (1,1,1) turns into albedoColor (0.5, 0.5, 0.5)
I donāt know if this is the reason, or even a flaw or intended.
Edit; changing light intensity causes odd result again after editing albedoColor, clearly something more complex at work
@Drigax @PatrickRyan
Did you have a chance to look at the gltf/glb export/material convertion?
Made a simple mesh while testing some functions,
It uses default material and vertexColors.
Exported three files directly from the same mesh,
.babylon (left in PG) (Looks as it should),
.gltf (middle in PG),
.glb (right in PG)
It would be nice if they looked simular after export, is it not possible?
https://playground.babylonjs.com/#GDGCSP#9
Color spaces are hard. Currently your original scene uses our Standard Material which interprets your color in sRGB color space, and once we import from glTF we use a PBR material for the imported mesh, which interprets color in a linear color spaceā¦oops. We should probably be converting your vertex color as wellā¦
Iāll look into fixing this, thanks!
That might expain why the difference is so big
Earlier examples using textures, the difference is much small / acceptable range
Currently we start with this:
https://playground.babylonjs.com/#GDGCSP#28
Iāve made some changes to convert to linear color space at export via:
But the result still appears perceivably different:
@PatrickRyan @sebavan any idea what we may be running into between our material models?
On our left is the original standard material, and on the right is our PBRMaterial.
Iāll see if i can see any difference with the unlit colors of the meshes as well
I am still seeing the same as you.
The vertex colors look identical in both models were I would expect standard in gamma and pbr in srgb ???
Oh it is local to your envā¦ Ok I need some sleep
The light computation happens in linear space for pbr and in gamma space for babylon
