This glTF extension is a multi-vendor extension. This extension can be implemented using the glTF loader extension mechanism. I wouldn’t be opposed to adding this to the list of extensions we support if someone is willing to contribute the code.|
Ported from GitHub
Hello Thomas,
I’m currently working on adding support for GLTF 2.0 extensions to the Babylon.js loader, specifically EXT_mesh_features, EXT_instance_features, and the related EXT_structural_metadata.
Since I’m already familiar with the Babylon.js loader (having contributed to some standard and vendor extensions that are now in production), I’d like to ask the Babylon.js team about the preferred architecture for integrating these new extensions.
Currently, these are vendor-specific extensions. While they are being used in some standards like OGC 3D Tiles, there’s still an open question about whether they should be embedded directly into the core GLTF loader. Another key consideration is how to expose these new features to the user, given that they primarily provide IDs whose meaning is application-specific.
I have my own ideas on this, but if these extensions could eventually be submitted as a PR to the Babylon.js engine, I want to ensure they align with any architectural guidelines or constraints the team may have.
Here’s the base implementation I have so far:
import { Geometry, Mesh, Nullable } from "@babylonjs/core";
import { IGLTFLoaderExtension } from "@babylonjs/loaders";
import { GLTFLoader, IMeshPrimitive, IProperty, ITextureInfo } from "@babylonjs/loaders/glTF/2.0";
const NAME = "EXT_mesh_features";
export type FeatureIdAttribute = number; // An integer value used to construct a string in the format `_FEATURE_ID_<set index>` which is a reference to a key in `mesh.primitives.attributes` (e.g. a value of `0` corresponds to `_FEATURE_ID_0`).
// A texture containing feature IDs
export interface IFeatureIdTexture extends ITextureInfo {
channels: Array<number>; // Texture channels containing feature IDs, identified by index. Feature IDs may be packed into multiple channels if a single channel does not have sufficient bit depth to represent all feature ID values. The values are packed in little-endian order.
}
export interface IFeatureId extends IProperty {
featureCount: number; // The number of unique features in the attribute or texture.
nullFeatureId?: number; // A value that indicates that no feature is associated with this vertex or texel.
label?: string; // A label assigned to this feature ID set. Labels must be alphanumeric identifiers matching the regular expression `^[a-zA-Z_][a-zA-Z0-9_]*$`.
attribute?: FeatureIdAttribute; // An attribute containing feature IDs. When `attribute` and `texture` are omitted the feature IDs are assigned to vertices by their index.
texture?: IFeatureIdTexture; // A texture containing feature IDs.
propertyTable?: number; // the index of the property table containing per-feature property values. Only applicable when using the `EXT_structural_metadata` extension.
}
/**
* [Specification](https://github.com/CesiumGS/glTF/blob/3d-tiles-next/extensions/2.0/Vendor/EXT_mesh_features/README.md)
* [Playground Sample]()
*/
// eslint-disable-next-line @typescript-eslint/naming-convention
export class EXT_mesh_features implements IGLTFLoaderExtension {
/**
* The name of this extension.
*/
public readonly name = NAME;
/**
* Defines whether this extension is enabled.
*/
public enabled: boolean;
private _loader: GLTFLoader;
/**
* @internal
*/
constructor(loader: GLTFLoader) {
this._loader = loader;
this.enabled = this._loader.isExtensionUsed(NAME);
}
public dispose(): void {
(this._loader as any) = null;
}
public _loadVertexDataAsync?(context: string, primitive: IMeshPrimitive, babylonMesh: Mesh): Nullable<Promise<Geometry>> {
...
}
}
GLTFLoader.RegisterExtension(NAME, (loader) => new EXT_mesh_features(loader));
with the following augmentation
declare module "@babylonjs/loaders/glTF/2.0" {
interface IProperty extends GLTF2.IProperty {}
}
I’d appreciate any thoughts or guidance on how best to proceed. Thanks!
Hi @Guillaume_Pelletier ! Adding @bghgary and @ryantrem who have been working on the loader and might have some recommendations.
Hi @Guillaume_Pelletier! Good to hear from you. Hope everything is well.
I think it’s okay to put into core. Thanks to @ryantrem, we can now dynamically load extensions only when they are used in the glTF, so it shouldn’t be a problem.
For the mesh attributes, we can just add them to the geometry. The other metadata for different properties can follow what the ExtrasAsMetadata loader extension is doing.
Thanks for the input! And yes, I’m happy to hear from you guys. I missed you.
Given the example, I may easily make it work using the following code:
/**
* @internal
*/
public _loadVertexDataAsync(
context: string,
primitive: IMeshPrimitive,
babylonMesh: Mesh
): Nullable<Promise<Geometry>> {
this._assignFeatureIds(babylonMesh, primitive.extensions?.EXT_mesh_features);
return null;
}
private _assignFeatureIds(babylonObject: any, gltfProp: IProperty): void {
if (HasFeatureIds(gltfProp)) {
babylonObject.featureIds = babylonObject.featureIds ?? [];
for (const i of gltfProp.featureIds) {
babylonObject.featureIds.push(i);
}
}
}
However, the unknown attributes are not loaded. Even when returning null to allow the default behavior, the list of attributes is already defined.
In the context of the following glTF :
[
{
"primitives": [
{
"extensions": {
"EXT_mesh_features": {
"featureIds": [
{
"featureCount": 4,
"attribute": 0
}
]
}
},
"attributes": {
"POSITION": 1,
"NORMAL": 2,
"_FEATURE_ID_0": 3
},
"indices": 0,
"material": 0,
"mode": 4
}
]
}
]
The _FEATURE_ID_0 attribute is not loaded.
Moreover, when dealing with extensions for textures ids in a glTF model like:
[
{
"primitives": [
{
"extensions": {
"EXT_mesh_features": {
"featureIds": [
{
"featureCount": 4,
"texture": {
"index": 1,
"texCoord": 0,
"channels": [0]
}
}
]
}
},
"attributes": {
"POSITION": 1,
"NORMAL": 2,
"TEXCOORD_0": 3
},
"indices": 0,
"material": 0,
"mode": 4
}
]
}
]
You can spot a TEXCOORD_0 attribute, which collides with the standard GLTF and Babylon.js TEXCOORD_0 kind in this line
So what would be the best way to handle the loading of attributes without needing to rewrite the entire function? work arround with the TEXCOORD_xxxx duplicate ? I believe I might have to duplicate the entire function, but you may have a better idea, because many private or internal types are involved. …
**** EDIT ****
In the meantime, I duplicated the _loadVertexDataAsync function with a proper type workaround, and that did the job. However, I haven’t handled the TEXCOORD_xxx duplication yet.
**** EDIT ****
done the duplicate… will test and post the code tomorrow… late in Paris now…
You can duplicate the core function for testing, but ideally, we should modify this function to support what we need for this extension. I’m not sure what that should look like and will have to think about it.
I’m not sure I get the full picture of the duplication issue. What happens if there are UVs and texture feature ids? Is that allowed?