Yo @Deltakosh … a while ago you were gonna have @bghgary add some kinda hooking into the GLTF parser so one doesn’t need to customize the GLTF project.
I basically have various node extras in my GLTF that I use for unity metadata … kinda like how I was using node.metadata previously in Babylon.
Are there any details so I can start hooking in and parsing my unity metadata that gets serialized as GLTF extras
You can use something like that:
https://www.babylonjs-playground.com/#8IMNBM#15
Does this fiction get called for EVERY node in gltf including the scene and material nodes… just trying to figure out how I can handle custom materials
Here are the hooks I implemented so far. IGLTFLoaderExtension - Babylon.js Documentation
Let me know if they work for you.
Yo @bghgary or @Deltakosh … I am trying to add support for GLTF Extensions in my new toolkit. A couple of questions:
1… So i should basically create a GLTF Extension for parsing my metadata. Like * KHR_texture_transform
2… How do you load support for my extension as well as all the others like * KHR_texture_transform
Yes. There are a bunch of loader extensions that add to the functionality of the core loader.
If you call the core loader’s respective function, it will call other loader extensions that haven’t been called yet.
I need to write some documentation for this. In the meantime, let me know what you’ve tried and I will help where I can.
So are all the other loaded automatically registered and ready to go or do I have to manually call a function to get them going.
Because I tried loaded a gltf that has scale and offset but I dont see that the texture has any tiling… my Amiga texture squares should be smaller and double the amount of squares for each cube face
Yo @Deltakosh or @bghgary … Im am trying to create a GLTF Extension but i am not to sure about the the promises and the call chain for parsing scene and node extras metadata. (We will come back to material extras metadata).
Can you plase look over this class and tell me if loadSceneAsync and loadNodeAsync are correct.
There is no function handler for this._loader.loadSceneAsync so i am not to sure i am setting babylonScene properties too early before the the promised function call to this._loader.loadSceneAsync finishes.
My GLTF Extension Skeleton Test Code:
const CVTOOLS_NAME = "CVTOOLS_unity_metadata";
/**
* Babylon canvas tools loader class
* @class CVTOOLS_unity_metadata
* [Specification](https://github.com/MackeyK24/glTF/tree/master/extensions/2.0/Vendor/CVTOOLS_unity_metadata)
*/
class CVTOOLS_unity_metadata implements BABYLON.GLTF2.IGLTFLoaderExtension {
/** The name of this extension. */
public readonly name = CVTOOLS_NAME;
/** Defines whether this extension is enabled. */
public enabled = true;
private _loader: BABYLON.GLTF2.GLTFLoader;
/** @hidden */
constructor(loader: BABYLON.GLTF2.GLTFLoader) {
this._loader = loader;
}
/** @hidden */
public dispose() {
delete this._loader;
}
/** @hidden */
public onLoading(): void {
console.log("CVTOOLS - OnLoading");
}
/** @hidden */
public onReady(): void {
console.log("CVTOOLS - OnReady");
}
/** @hidden */
public loadSceneAsync(context: string, scene: BABYLON.GLTF2.Loader.IScene): BABYLON.Nullable<Promise<void>> {
const promise = this._loader.loadSceneAsync(context, scene);
console.log("CVTOOLS - Parsing Scene: " + scene.name);
// ..
// TODO: Update Scene Properties
// ..
this._loader.babylonScene.ambientColor = BABYLON.Color3.Green();
// ..
return promise;
}
/** @hidden */
public loadNodeAsync(context: string, node: BABYLON.GLTF2.Loader.INode, assign: (babylonMesh: BABYLON.Mesh) => void): BABYLON.Nullable<Promise<BABYLON.Mesh>> {
const promise = this._loader.loadNodeAsync(context, node, (mesh: BABYLON.Mesh) => {
console.log("CVTOOLS - Parsing Mesh Node: " + mesh.name);
// ..
// TODO: Update Mesh Properties
// ..
mesh.visibility = 1.0;
// ..
assign(mesh);
});
return promise;
}
}
BABYLON.GLTF2.GLTFLoader.RegisterExtension(CVTOOLS_NAME, (loader) => new CVTOOLS_unity_metadata(loader));
This looks correct. The assign function callback sets the newly created object. Scene is created outside of the loader and doesn’t need this.
Sweet… Thanks Gary… How about Materials… Whats the rhyme or reason on all the material functions.
I basically want to add lightmap support (and a few other properties) to an existing material… Can i need to be able to support Standard and Custom materials based off the material.extras.metadata
How should the loadMaterialAsync functions look and how show the promises look for that?
For lightmap, override the IGLTFLoaderExtension - Babylon.js Documentation function the same way as what you have with the other functions. Then call GLTFLoader - Babylon.js Documentation to load the lightmap texture. Once the texture is loaded, assign it to the provided Babylon material. Use Promise.all to combine the promises together so that it will resolve when both asyn ops complete.
Crap… I dont think i understand the rhyme or reason behind GLTFLoader.LoadExtensionAsync and LoadExtrasAsync… All the 3rd party extensions seem to wrap everything in LoadExtensionAsync.
Also when trying to use Promise.all from my ES5 compile target. I get an error:
[ts] 'Promise' only refers to a type, but is being used as a value here. Do you need to change your target library? Try changing the `lib` compiler option to es2015 or later. [2585]
Is there a ES5 way of doing Promise.all ???
This is what i have so far WITHOUT the GLTFLoader.LoadExtensionAsync wrapper:
/** @hidden */
public loadMaterialPropertiesAsync(context: string, material: BABYLON.GLTF2.IMaterial, babylonMaterial: BABYLON.Material): BABYLON.Nullable<Promise<void>> {
const promises = new Array<Promise<any>>();
/* LIGHTMAP PSUEDO-CODE TEST
const properties = material.pbrMetallicRoughness;
if (properties.lightmapTexture) {
promises.push(this._loader.loadTextureInfoAsync(`${context}/lightmapTexture`, properties.lightmapTexture, (texture) => {
babylonMaterial.lightmapTexture = texture;
}));
}
*/
return Promise.all(promises).then(() => { });
}
But it does not compile using ES5 target 
I think i fixed the Promise.all issue using ‘lib’ in compilerOptions:
"compilerOptions": {
"target": "ES5",
"module": "system",
"lib": ["es5", "es2015", "dom"],
"experimentalDecorators": true
}
But still dont quite understand the workflow…
Should i be calling the internal _loader.loadMaterialPropertiesAsync from my loadMaterialPropertiesAsync function ???
like so:
/** @hidden */
public loadMaterialPropertiesAsync(context: string, material: BABYLON.GLTF2.Loader.IMaterial, babylonMaterial: BABYLON.Material): BABYLON.Nullable<Promise<void>> {
const promises = new Array<Promise<any>>();
promises.push(this._loader.loadMaterialPropertiesAsync(context, material, babylonMaterial));
/* LIGHTMAP PSUEDO-CODE TEST
const properties = material.pbrMetallicRoughness;
if (properties.lightmapTexture) {
promises.push(this._loader.loadTextureInfoAsync(`${context}/lightmapTexture`, properties.lightmapTexture, (texture) => {
babylonMaterial.lightmapTexture = texture;
}));
}
*/
return Promise.all(promises).then(() => { });
}
Yes. That looks right except for how you get the properties. Where is the lightmap stored in the gltf?
Will be on material.commonConstant.lightmapTexture (i using UnityGLTF Material.commonConstant)
Yo @bghgary … Here is my updated material parser for lightmap texture using UnityGLTF material.commonConstant json:
/** @hidden */
public loadMaterialPropertiesAsync(context: string, material: BABYLON.GLTF2.Loader.IMaterial, babylonMaterial: BABYLON.Material): BABYLON.Nullable<Promise<void>> {
const promises = new Array<Promise<any>>();
promises.push(this._loader.loadMaterialPropertiesAsync(context, material, babylonMaterial));
// ..
console.log("CVTOOLS - Parsing Material Info: " + material.name);
console.log(material);
// ..
const materialJson:any = material;
const commonConstant:any = (materialJson.hasOwnProperty("commonConstant")) ? materialJson.commonConstant : null;
if (commonConstant != null) {
if (commonConstant.lightmapTexture) {
promises.push(this._loader.loadTextureInfoAsync(`${context}/lightmapTexture`, commonConstant.lightmapTexture, (texture) => {
if (babylonMaterial instanceof BABYLON.PBRMaterial) {
let pbrMaterial:BABYLON.PBRMaterial = babylonMaterial as BABYLON.PBRMaterial;
pbrMaterial.lightmapTexture = texture;
pbrMaterial.useLightmapAsShadowmap = true;
} else if (babylonMaterial instanceof BABYLON.StandardMaterial) {
let stdMaterial:BABYLON.StandardMaterial = babylonMaterial as BABYLON.StandardMaterial;
stdMaterial.lightmapTexture = texture;
stdMaterial.useLightmapAsShadowmap = true;
}
}));
}
}
// ..
return Promise.all(promises).then(() => { });
}
That seems to work… Is this the right way i should be doing my extra material properties ???
Next… How do i create a custom material.
So based on: material.extras.metadata.legacyMaterial = true
I wanna create a Standard Material instead of PBRMaterial…
Also i wanna be able to create Custom Materials … But i imagine its the same as getting a Standard Material working ???
To change what material is created, override the createMaterial function and return whatever material you want.
The code above looks good in terms of code. Putting a property on a glTF material without using extensions or extras is not allowed by the spec. Future versions of glTF may end up colliding with the names.
I know about the commonConstant on the material directly. I mainly went this route because the UnityGLTF Exporter use the material.commonConstant
I was gonna just add the commonConstat to the CVTOOLS_unity_metadata extension definition so when the extension goes public should not error on that commonConstant
Or if it’s a big deal… I’ll move to material.extras
Hey @bghgary and @Deltakosh and @sebavan … I am starting to notice something i dont really like in the GLTF Handness.
Every where in the C# Unity GLTF Exporter when dumping out position and rotation we SWITCH the handness. For example, this is out i dump out the node transform position in C#
Vector3 position = useLocal ? nodeTransform.localPosition : nodeTransform.position;
node.Translation = new XGLTF.Math.Vector3(position.x, position.y, -position.z);
// ..
Quaternion rotation = useLocal ? nodeTransform.localRotation : nodeTransform.rotation;
node.Rotation = new XGLTF.Math.Quaternion(rotation.x, rotation.y, -rotation.z, -rotation.w);
Notice the position.z, rotation.z and rotation.w always get NEGATED.
And every where i am using my ExportAccessor i can SWITCH the Handedness for positions and rotations.
public static Vector3 SwitchHandedness(this Vector3 input)
{
return new Vector3(input.x, input.y, -input.z);
}
public static Vector4 SwitchHandedness(this Vector4 input)
{
return new Vector4(input.x, input.y, -input.z, -input.w);
}
public static Quaternion SwitchHandedness(this Quaternion input)
{
return new Quaternion(input.x, input.y, -input.z, -input.w);
}
public static Matrix4x4 SwitchHandedness(this Matrix4x4 matrix)
{
Vector3 position = matrix.GetColumn(3).SwitchHandedness();
Quaternion rotation = Quaternion.LookRotation(matrix.GetColumn(2), matrix.GetColumn(1)).SwitchHandedness();
Vector3 scale = new Vector3(matrix.GetColumn(0).magnitude, matrix.GetColumn(1).magnitude, matrix.GetColumn(2).magnitude);
float epsilon = 0.00001f;
// Some issues can occurs with non uniform scales
if (Mathf.Abs(scale.x - scale.y) > epsilon || Mathf.Abs(scale.y - scale.z) > epsilon || Mathf.Abs(scale.x - scale.z) > epsilon)
{
Debug.LogWarning("A matrix with non uniform scale is being converted from left to right handed system. This code is not working correctly in this case");
}
// Handle negative scale component in matrix decomposition
if (Matrix4x4.Determinant(matrix) < 0)
{
Quaternion rot = Quaternion.LookRotation(matrix.GetColumn(2), matrix.GetColumn(1));
Matrix4x4 corr = Matrix4x4.TRS(matrix.GetColumn(3), rot, Vector3.one).inverse;
Matrix4x4 extractedScale = corr * matrix;
scale = new Vector3(extractedScale.m00, extractedScale.m11, extractedScale.m22);
}
// convert transform values from left handed to right handed
return Matrix4x4.TRS(position, rotation, scale);
}
}
then and example ExportAccessor function support switching the handedness:
private AccessorId ExportAccessor(Vector3[] arr, bool switchHandedness=false, string accessorName = null)
{
var count = arr.Length;
if (count == 0)
{
throw new Exception("Accessors can not have a count of 0.");
}
var accessor = new Accessor();
accessor.ComponentType = GLTFComponentType.Float;
accessor.Count = count;
accessor.Type = GLTFAccessorAttributeType.VEC3;
accessor.Name = accessorName;
List<Vector3> values = new List<Vector3>();
foreach (var vec in arr) { values.Add(switchHandedness ? vec.SwitchHandedness() : vec); }
accessor.Min = new List<double> { values.Select(value => value.x).Min(), values.Select(value => value.y).Min(), values.Select(value => value.z).Min() };
accessor.Max = new List<double> { values.Select(value => value.x).Max(), values.Select(value => value.y).Max(), values.Select(value => value.z).Max() };
var byteOffset = _bufferWriter.BaseStream.Position;
foreach (var vec in values) {
_bufferWriter.Write(vec.x);
_bufferWriter.Write(vec.y);
_bufferWriter.Write(vec.z);
}
var byteLength = _bufferWriter.BaseStream.Position - byteOffset;
accessor.BufferView = ExportBufferView((int)byteOffset, (int)byteLength);
var id = new AccessorId {
Id = _root.Accessors.Count,
Root = _root
};
_root.Accessors.Add(accessor);
return id;
}
That is the GLTF Spec i suppose… So the Babylon does a conversion on the position and rotation when importing GLTF scene file.
Now if using ALL GEOMETRY and CAMERAS AND LIGHTS coming from the GLTF scene file everything is fine.
But id i manually add geometry using code i gotta INVERT the position and rotation to get the same result.
Example… i have a plane and a cube at zero position in scene. In unity, i would make a Camera at position 0,1,-10 and rotation 0,0,0… That camera would be backup on the Z axis 10 units and it show the cube on plane just fine.
If i make the camera in code, for it to show the cube and plane at zero… i have to make the camera in code with position 0,1,10 and rotation 0, Math.PI, 0
Its like i have to think inverted of what i would normally expect a camera 10 units back on the Z-Axis
i think that is gonna be problem when write game mechanic code and where you expect to position and rotate things… I would think.
Is there a way to tell BABYLON GLTF parser NOT to invert the positions and rotations and i will export from Unity C# WITHOUT switching the handedness… I know it would not be a standard GLTF… but maybe i can keep a flag on the scene extras metadata that says DONT switch handedness then in the Babylon GLTF Extension i check that flag BEFORE i start parsing the seen and setup babylon gltf to NOT switch the handedness… Just positions and rotations as is.
What do you think i should do about this in the toolkit ???
You can set scene.useRightHandedSystem = true to change Babylon to right-handed. Does that make it better?