Realtime 3D Face Capture

So, this is a little tool I did some time ago and was here getting dust. =)

It uses Mediapipe for face mesh capture – plus a trick of mine to convert it into textured 3D face model on-the-fly =) – and Babylon.js for 3D rendering. It can export the captured face as gltf.

Demo: face
Source-code: GitHub - imerso/facecap: Babylon.js + Mediapipe face capture

Requires a webcam!

14 Likes

this is INSANE! SO! COOL!

Wow Amazing GIF - Wow Amazing Great - Discover & Share GIFs | Gif, Cool gifs, Amazing

1 Like

would be interesting if you published to github pages to make it more accessible!

Edit: :man_facepalming: click the demo link if you are as blind as me!

1 Like

It is on github pages, the link is on the original post. =)

1 Like

This is amazing! I believe this technique can be used for virtual space live conferencing calls by mapping webcam feeds to this and then map onto avatar meshes. Current approaches only display a square mesh of webcam feed beside the characters (could be for performance purposes).

2 Likes