MMD file support, similar or better than the one in threejs

Feature requests
1

Threejs has this gem:
https://threejs.org/docs/#examples/loaders/MMDLoader
https://threejs.org/docs/#examples/animations/MMDPhysics
https://threejs.org/docs/#examples/animations/MMDAnimationHelper

demo
https://threejs.org/examples/webgl_loader_mmd.html

Was wondering if we can get it in babylonjs. With the huge amounts of community made content, it could make for some interesting web apps. The MMD content tends to be shared as video on the web, having it be shared in a more interactive way would be better.

It would be amazing if we can get runtime access to the skeleton, animation, model parts, materials, etc - be able to change them dynamically

I believe it is based on this:
http://takahirox.github.io/mmd-viewer-js/


but I might be wrong

The best thing about this is really how when loaded we get physics, animation and materials working out of the box. They are all set in the file.

Threejs has made this super easy.
So to summarize the selling points:

  • big amount of free quality content with community making it actively
  • little effort to load an animated model with physics set on it (traditionally a pain in the neck with other formats). Its all in the file
  • the content sort of works great for interactive/vr apps - it compliments that technology
  • would make for some fun babylonjs demos : ) (even Unity3d got one)
2

THis is not something I’m planning to personnaly work on but if anyone is motivated give it a try, I will be glad to merge the loader :wink:

3

Is there some simple example code we could look at for loading a new file type into babylonjs? What would be a good first place to start? Was wondering if we can adopt what threejs has to babylonjs

4

Looking into it, the threejs one uses this to parse the files

Here is the source code

perhaps it’s worth contacting takahiro (the author) to see if he would be interested in porting it to babylonjs?

1 Like
5

Hey @Deltakosh, I am planning to give this one a shot.
I’ve already forked the Extensions repo and built a basic framework for the code here: https://github.com/r-arvind/Extensions/tree/master/MMDLoader

I got the mmd parser running and hosted it here https://mmdtest.netlify.app/. On parsing, it gives out all the data like vertices, bones, morphs etc.

Now, I have a few things I wanted to discuss.

Once I have the data about the vertices, the next step is to create the mesh. I found this article on creating custom meshes https://doc.babylonjs.com/divingDeeper/mesh/creation/custom/custom, but the issue is that it requires us to pass a scene. I want the mesh to be standalone without passing it to any scene.

Also, I am not going to extend the scene loader for this extension. I think it would be better to keep this separate.

2 Likes
6

This is an AMAZING news 2021 is already starting better than 2020 :wink:

Could you create it similar to the obj or stl loader ?Babylon.js/objFileLoader.ts at master · BabylonJS/Babylon.js · GitHub

7

Hey @sebavan, that was my initial plan i.e to extend the scene loader plugin interface. But the issue is that, it essentially has three functions that needs to be implemented: importMesh, load and loadassetcontainer.

MMD is a bit different, when you load an MMD model, you generally attach a VPD(vocaloid pose data) for pose or VMD(vocaloid motion data) for animation. The mmd model file also has data for various morphs for facial expressions and other things.

For importing obj, we just pass the scene variable and the path to obj to the scene loader. But this method didn’t feel appropriate for mmd.

My plan was to build a class with a load function, which will read the mmd file and return a Babylon mesh, with rigs and skeleton when you pass the URL of the mmd file. Then you’ll have a function where you can pass the mesh and a VPD/VMD which will then assign the pose/animation to the mesh.

What do you think?

8

I am unfortunately not versed enough in the topic but I bet @Deltakosh will have some nice suggestions when he ll be back on Tuesday as he already had a look into it :slight_smile:

1 Like
9

Hey this is really great news!

Your loader can only implement ImportMesh actually and simply return null for other entry points :slight_smile:

1 Like
10

What about attaching a pose or an animation using .vmd file? How should that be implemented?

11

This is kind of brand new so I guess the extension can provide its own methods for that :wink:

1 Like
12

Thanks for the help :+1:! I’ll ping you if I bump into some issues

2 Likes
13

I have the same problem, how can I implement it

14

You can now load MMD model/motion from babylon.

7 Likes
15

w00000t this is huge !!! @PirateJC and @PatrickRyan I think we should make room in the doc for this one ?

1 Like
16

Here’s a working demo on PG.
There are still problems with physics and audio syncs.

If it’s really completed, I’ll post it separately.

2 Likes
17

Nice! Do you feel this is ready for me to add to my awesome-babylonjs list?

18

I think it’s ready. The 1.0.0 release is just around the corner.

I will introduce it in a week.

1 Like