Hello, and thanks for any help you might lend. I am working with a background layer to capture a webcam into a fullscreen texture, and found some useful code for resizing in another thread (@bghgaryFullScreenLayer video; dynamically change the aspect ratio - #2 by carolhmj). The webcam image is being sent through a facemesh camera to allow me to track face landmarks in the image, and is showing some distortion on mobile. It is possible that the error is being introduced by scaling the texture which facemesh is using to create the landmarks, although the desktop version doesn’t show the issue.
I have tried pushing content through the scene to see if the distortion is in the 3d render, but it is only in the landmark positions (and generated mesh).
The offset, I believe, is the result of the difference in angle between the scene and my desktop camera. I am looking at how to correct it using the face normals (and perhaps intraocular distance). On mobile (portrait) the facemesh generated model looks distorted in y, although the tracking objects appear to have the right proportions.
It is also possible I have been looking at this too long.
Thanks, and I should be able derive the perspective camera using the webcam focal distance, but will need to correct the camera pose to account for the location of the webcam. There are some tools included within mediapipe/camera_utils which are for that purpose.
The pose transform matrix is included with the geometry in face mesh, and can be used to transform the vertices into correct 3d positions. I will update the demo in the other thread.