Loading VRM Humanoid-based model

Demos and projects
#1

“VRM” is a file format for handling 3D humanoid avatar (3D model) data for VR applications . It is based on glTF 2.0. Anyone is free to use it. In addition, a standard implementation (UniVRM) in c# that can import and export VRM file in Unity is released as open source.
by VRM.dev

I’d just created UniVRM to TypeScript porting for babylon.js the amazing WebGL library!

image
image.jpg761×399 57.3 KB

Anyone can use babylon-vrm-loader npm package.

import * as BABYLON from '@babylonjs/core'

// this importing has side-effect
// ref. https://webpack.js.org/guides/tree-shaking#mark-the-file-as-side-effect-free
import 'babylon-vrm-loader'

// ... init and create Engine and Scene

// vrmFile is File object retrieved by <input type="file">.
// or you can use url too.
const scene = await BABYLON.SceneLoader.LoadAsync('file:', vrmFile, engine);
const vrmManager = scene.metadata.vrmManagers[0];

// Update secondary animation(Spring Bones)
scene.onBeforeRenderObservable.add(() => {
    vrmManager.update(scene.getEngine().getDeltaTime());
});

// Model Transformation
vrmManager.rootMesh.translate(new BABYLON.Vector3(1, 0, 0), 1);

// Work with HumanoidBone
vrmManager.humanoidBone.leftUpperArm.addRotation(0, 1, 0);

// Work with BlendShape(MorphTarget)
vrmManager.morphing('Joy', 1.0);

Codepen example is here(you need VRM model file).

Also you can see example here with sample model “Alicia Solid”.

Feel free to use and contribution! virtual-cast/babylon-vrm-loader : GitHub

9 Likes
#2

This is amazing !!! I think this would deserve an entry in the documentation so that ppl looking for loaders could find your great work.

I bet on @Deltakosh being pretty happy with that too.

1 Like
#3

You rock!!! would you be ok to add a page for it in our documentation so people can easily find it? (with link to your demo)

1 Like
#4

This is very nice! I’m glad you are using the glTF loader extension mechanism to do it.

2 Likes
#5

Of course.

#6

VRM has Unity Humanoid bone mapping data. And TransformNode is mapped to these bones(like LeftUpperLeg, Chest, Neck, …).

So you can animate any VRM model with same code!

vrm-motion

// example code
vrmManager.humanoidBone.leftShoulder.rotationQuaternion = Quaternion.FromEulerAngles(
    Math.sin(Math.PI / 4 * (elapsedTime / 200)),
    0,
    Math.PI / 3.5,
);
vrmManager.humanoidBone.rightShoulder.rotationQuaternion = Quaternion.FromEulerAngles(
    Math.sin(Math.PI + (Math.PI / 4 * (elapsedTime / 200))),
    0,
    -Math.PI / 3.5,
);
vrmManager.humanoidBone.leftUpperLeg.rotationQuaternion = Quaternion.FromEulerAngles(
    Math.sin(Math.PI / 4 * (elapsedTime / 200)),
    0,
    0,
);
vrmManager.humanoidBone.rightUpperLeg.rotationQuaternion = Quaternion.FromEulerAngles(
    Math.sin(Math.PI + (Math.PI / 4 * (elapsedTime / 200))),
    0,
    0,
);
vrmManager.humanoidBone.leftLowerLeg.rotationQuaternion = Quaternion.FromEulerAngles(
    -Math.PI / 6,
    0,
    0,
);
vrmManager.humanoidBone.rightLowerLeg.rotationQuaternion = Quaternion.FromEulerAngles(
    -Math.PI / 6,
    0,
    0,
);

Left model is IL-tan which is my company’s image character.

Here is VRM applications list almost in Japan.

2 Likes
#7

This is excellent:D

#8

https://playground.babylonjs.com/#K5W35Y#8

Playground example is here!

1 Like
#9

Haha this is really cool!! Do you want to add a page about VRM in our doc?

1 Like
#10

I made it!

3 Likes
#11

Is there an example with react.js loading VRM

1 Like
#12

Is it possible to use animation groups on the VRM model?
I tried loading animations from a GLB file (using Asset Containers) then copying the animation to the VRM model but I could not locate the skeleton object.

Also, if I create an avatar in Babylon.js can this library export the avatar as a VRM model?

#13

You can apply the animation groups to TransformNode using VRMManager.humanoidBone reference.

(I don’t know how to use Skeleton in animation :neutral_face:)

Sample code is here.

    const baseAnimGroup = assetContainer.animationGroups[0]
    const animGroup = new AnimationGroup(`${baseAnimGroup.name}_`, scene)
    animGroup.speedRatio = baseAnimGroup.speedRatio

    baseAnimGroup.targetedAnimations.forEach(targetedAnimation => {
      // refer animation name correspond to humanoidBone. example: head, leftUpperArm, ...
      const boneName: string | undefined = targetedAnimation.target.name
      if (!boneName || !targetVRMManager.humanoidBone[boneName]) {
        return
      }
      animGroup.addTargetedAnimation(
        // clone from original animation
        targetedAnimation.animation.clone(),
        // attach animation to TransformNode.
        targetVRMManager.humanoidBone[boneName],
      )
    })

Currently, you cannot export VRM Model using babylon.js. You need to use UniVRM for example.

1 Like