I have been working on some xr tracking code and wanted to share a playground with some source for attaching meshes to face tracking landmarks. The code works on both desktop and mobile, although some issues remain with aligning the scene camera to a webcam.
There are hotkeys (wasd) for correcting the camera position, and I will post an update when I come up with a better solution to align the scene. I also added filtering to the tracked landmark position to reduce jittering in the mesh.
(Thanks to @bghgary for the background scaling code, and to @imerso for mediapipe resources)
Hey Carol, and thanks. It turns out that getting pose and position is much easier than going through the screen based landmarks, although I need to derive the camera vertical fov and distance. Both should be fairly straightforward to gather from the camera, and should fix the scale issue you are seeing.
Here is a playground showing the improved camera orientation. The trackers have been removed temporarily while I decide what to do about normal recalculation.