MiKaPo: AI Pose Picker for MMD Model (Demo)

Demos and projects
1

I’m excited to share my project, MiKaPo (MikuMikuDance Pose Picker) , a web-based tool that automatically poses MMD models from video input in real-time. The tool currently supports pose and face detection, with both video upload and camera input options.


The tech stack includes BabylonJS for 3D rendering, babylon-mmd for MMD model support (big credits to @noname0310), Mediapipe for 3D keypoints detection, and Vite+React for the web framework. The key challenges revolve around efficiently controlling MMD models with 3D keypoints detected by Mediapipe, and optimizing performance for real-time updates.

My current plan is to develop an Electron version for desktop with more advanced functions like user model selection, animation export, and Ollama integration. I’m looking forward to hearing your perspectives on this, as well as any feedback, new function proposals, or suggestions on future deployment platforms as we explore the potential of this project together!

9 Likes
2

Wow!!
This is super cool buddy!

I am a big fan of MMD as well. In addition my daughter likes to dress like this :see_no_evil:

I can’t comment the tech stuff, I’m like two years behind in motion capture/face detection however I have a question. Could this be used as a poor man’s motion capture for gaming purposes as well? Sometimes you just can find the right one on Mixamo :smiley:

1 Like
3

Of course! check the mediapipe’s website Guía de detección de puntos de referencia de posiciones  |  Google AI Edge  |  Google AI for Developers

2 Likes
4

I forgot you to welcome to the community! So hey, we are already glad we have you here buddy! :wink:

2 Likes
5

This freaking rocks my man!!!

2 Likes
6

This is so coool :slight_smile:

1 Like