What plans does babylon.js have for digital humans？

Questions
1

Firstly, may I ask if babylon.js has any plans regarding digital humans?
Secondly, is there a faster and more convenient way for babylon.js to support the data combination of files with. obj and. npz suffixes?
Thank you for your response!

2

Could you elaborate your question in a bit more understandable way?

1 Like
3

chatGPT:

The person is asking two questions about Babylon.js and its capabilities:

1. Digital Humans in Babylon.js

“Firstly, may I ask if babylon.js has any plans regarding digital humans?”

They’re asking whether Babylon.js is planning to support or enhance features for “digital humans”, which usually refers to highly realistic 3D models of people, often used in:

  • virtual influencers
  • games
  • metaverse applications
  • digital twins or avatars
  • virtual assistants

They might be wondering if Babylon.js will offer:

  • specialized shaders or materials for skin, eyes, and hair
  • facial animation or rigging systems
  • support for motion capture or realistic behavior

2. Support for .obj and .npz files

“Secondly, is there a faster and more convenient way for babylon.js to support the data combination of files with .obj and .npz suffixes?”

This question is more technical. They’re asking if there’s a better or more efficient way to load and combine data from:

  • .obj files – common 3D mesh files
  • .npz files – NumPy compressed arrays (used in Python, often for machine learning or simulation data)

They’re likely working on something that combines 3D geometry from .obj and data (maybe animations, morph targets, physics, etc.) from .npz.

So they’re looking for:

  • existing tools, loaders, or workflows in Babylon.js that simplify importing and merging these two types of files
  • performance improvements or easier APIs
2 Likes
4

What would anser the babylon team?

ChatGPT:

If the Babylon.js team were to respond officially, it might look something like this — friendly, open, and a bit technical:

Hi there! Thanks for your questions — great ones! :blush:

1. Digital Humans in Babylon.js

At Babylon.js, we’re always excited about pushing the boundaries of real-time 3D on the web. While we don’t currently have a dedicated “digital human” pipeline, Babylon.js is fully capable of rendering high-quality human models, including:

  • Subsurface scattering and PBR materials for realistic skin
  • Morph targets, bones, and animation blending for facial expressions and body motion
  • Integration with GLTF, which is a great format for complex rigged characters

We’re also watching the digital human space closely, and if there’s strong community interest, we’d love to explore what we can do to make development even easier.

:backhand_index_pointing_right: In the meantime, you can already achieve some amazing results using GLTF + Babylon’s animation and shader systems.

2. Combining .obj and .npz files

Babylon.js has native support for .obj files, and you can load them using our OBJFileLoader. However, .npz is a Python/NumPy-specific format, which isn’t directly supported in the browser.

If you’re trying to combine .obj geometry with .npz data (e.g., for animation, simulation, or ML-driven morph targets), you’ll likely need to:

  • Preprocess the .npz file in Python (convert it to JSON, binary, or a custom format)
  • Load the .obj as usual, then apply the .npz data using your own JavaScript logic or shaders

There isn’t currently a built-in tool that directly combines .obj + .npz, but we’d be happy to help if you want to build a custom loader or workflow!

2 Likes
5

@CodingCrusader Well done! :wink:

1 Like
6

I wonder what is the main current tool to combine .obj and .npz if it exists.

1 Like
7

Can you write code to combine 3d obj files with npz files to modify the vertex positions of the obj file? Take animation into consideration as well. Babylon.js

Sure! Here’s a complete example of how to combine .obj files with .npz animation data to modify vertex positions over time in Babylon.js.

We’ll assume:

  • You have a .obj model loaded into the scene.
  • You have an .npz file (NumPy zipped array) that contains vertex positions for each animation frame.
  • You want to animate the model by updating the vertex positions every frame based on the .npz data.

Because .npz is a Python/Numpy format, it must be preprocessed or converted to a format Babylon.js (JavaScript) can handle — for example, you could convert it into a .json with an array of frames, each containing vertex positions.

:light_bulb: Step 1: Convert .npz to JSON (Python side)

You need to preprocess the .npz file like this:

import numpy as np
import json

data = np.load('animation.npz')  # contains e.g. frames['frame_000'], ..., ['frame_100']
frames = []

for key in sorted(data.files):  # assuming frame_000, frame_001, ...
    frame_data = data[key]  # shape: (num_vertices, 3)
    frames.append(frame_data.tolist())

with open('animation.json', 'w') as f:
    json.dump(frames, f)

:rocket: Step 2: Babylon.js code to apply animation

import { Engine, Scene, ArcRotateCamera, Vector3, HemisphericLight, SceneLoader, VertexBuffer, Mesh } from "@babylonjs/core";
import "@babylonjs/loaders"; // OBJ loader

const canvas = document.getElementById("renderCanvas") as HTMLCanvasElement;
const engine = new Engine(canvas, true);
const scene = new Scene(engine);

// Camera and light
const camera = new ArcRotateCamera("camera", Math.PI / 2, Math.PI / 3, 10, Vector3.Zero(), scene);
camera.attachControl(canvas, true);
new HemisphericLight("light", new Vector3(0, 1, 0), scene);

// Load OBJ and JSON animation
let frameIndex = 0;
let animationFrames: Float32Array[] = [];

async function loadAnimationData(url: string) {
    const response = await fetch(url);
    const json = await response.json();
    animationFrames = json.map((frame: number[][]) => new Float32Array(frame.flat()));
}

function applyFrame(mesh: Mesh, frameData: Float32Array) {
    const vertexData = mesh.getVerticesData(VertexBuffer.PositionKind)!;
    if (!vertexData || vertexData.length !== frameData.length) return;

    for (let i = 0; i < vertexData.length; i++) {
        vertexData[i] = frameData[i];
    }

    mesh.updateVerticesData(VertexBuffer.PositionKind, vertexData);
    mesh.refreshBoundingInfo(); // update AABB if needed
}

// Load model and start animation loop
SceneLoader.ImportMeshAsync("", "models/", "model.obj", scene).then(async result => {
    const mesh = result.meshes[0] as Mesh;

    await loadAnimationData("animations/animation.json");

    scene.onBeforeRenderObservable.add(() => {
        if (animationFrames.length > 0) {
            applyFrame(mesh, animationFrames[frameIndex]);
            frameIndex = (frameIndex + 1) % animationFrames.length;
        }
    });
});

engine.runRenderLoop(() => scene.render());

:white_check_mark: Folder Structure (example)

/public
  /models
    model.obj
  /animations
    animation.json

:brain: Notes

  • This assumes no topology change between frames: all frames must have the same number and order of vertices.
  • You can extend it to support .mtl files or blend shapes/morph targets for better performance.
  • For smoother animations, consider interpolating between frames.

:smiley: :smiley: :smiley:

2 Likes
8

For Babylon.js, efficiently combining .obj (3D model) and .npz (NumPy data array) files requires a streamlined workflow. Here’s the fastest and most convenient approach:

1. Convert .npz to Babylon-Friendly Format

Since .npz (NumPy) isn’t natively supported, preprocess it into a compatible format:

  • Option A: Convert to .json/.csv

import numpy as np data = np.load(‘data.npz’) np.savez_compressed(‘converted.npz’, **data)
Keep compressed if needed

Then parse in JavaScript using:

fetch(‘converted.npz’).then(response => response.arrayBuffer()) .then(buffer => { const data = new Float32Array(buffer); // Process as needed });

  • Option B: Use Python to generate .babylon or .gltf
    Tools like pybabylon or blender-scripting can merge .obj + .npz into a single export.

2. Load .obj and Attach .npz Data Dynamically

Use Babylon.js’s OBJFileLoader and link the data:

3. Optimize Performance

  • Compression: Use .glb (binary) instead of .obj + .npz for smaller files.
  • Web Workers: Parse .npz in a worker thread to avoid UI freezing.
  • Binary Serialization: Save .npz as raw binary (Float32Array) for direct GPU upload.

4. Use Add-ons for Direct Support

  • Babylon.js NPZ Loader Plugin
    Create a custom loader (example snippet):

BABYLON.SceneLoader.RegisterPlugin({ extensions: [“.npz”], importMesh: (meshes, scene, data) => { const parsedData = parseNPZ(data); // Your NPZ parser // Apply to meshes }, });

5. Best Practice: Unified File Format

For production, combine mesh + data into a single format:

  • .glTF + Embedded Data
    Use extras in glTF to store .npz data:

{ “meshes”: […], “extras”: { “npz_data”: […] } }

Tools to Simplify

  • Python Script (Automate conversion):

  • Blender Pipeline: Scripted export combining both files.

Final Tip

For real-time updates, use Babylon.js Observables to sync .npz data changes with the rendered mesh.

1 Like
9

This is the first I’ve heard of .npz. What kind of data is it storing about the model-- is it actually animation data? How does it work in conjunction in .obj? How does it fit into digital humans? :open_mouth:

10

@CodingCrusader That’s what I want to express, thank you!
@roland @labris This is the method I am currently trying, but I would like to know if Babylon.exe has a faster method, but it seems that there is no such method now.

11

Just use:

1 Like