@vijay_krishna, Iām not sure if I follow what you are asking to do. When you say āchangeā the face by āapplyingā the user photo, are you talking about morphing the geometry to match the user photo or changing the texture of the face only by adding the the user photo at the pixel level? In either case, there will need to be some tech for identifying the features in the face so you can either morph the geometry or for fine tweaking the position of the user photo since you will get a wide variety of source images. But further clarification of your goals would be great.
and Thanks @PatrickRyan for your attention, We are looking for face morphing by applying user photo. The input will be .glb file shared earlier and user will upload his photo as per our guidelines. Can you share some logic/example how to accomplish this.
Hey, welcome.
Sorry that I have to give you this answer but ā¦ āIt is not possible yetā. Trust me, you are not the first who want to do this. I heard about this atleast 10 times till now.
2 big reasons.
1 - let say you have the object and you want to aply the photo on it, you canāt simply put an image over a mesh, ā¦you need uvs, and the image need to look in some way to be aplied corectly, and the image will not look good if the geometry was done for other āfaceā
2 -let say you want the mesh to change according to image, how do you think youāll do this? A mesh with 8 morphtargets for eye, nose, mouth is very big, and the result is a very limited type of faces
As a conclusion, you canāt do this for random photo and random faces, maybe you can do this but for few only
But we have a hope, Iām sure that one day an AI will do this for us.
I donāt want to discourage you, but unfortunately thatās about the situation right now. You will find mobile apps or maybe web apps too, which say that they can do this, but ā¦ they donāt, or their result is poor, not even close to a really human face. Or maybe youāll find something which I donāt, so you can still search for it
Actually, I understand that it can currently be done by a few. Not with a single photo, but a group. It cannot be done in this framework though. Perhaps try a search of 'deep fakes ā will lead to place that has more emphasis on this.
For consistency, you could start with a pool of known face types, similar to the concept of ecosystem biomes. Iām sure thereās some research out there on this topic. After you do feature detection, pick the prebuilt model closest to your user and apply a transform to it based on whatever feature detection algorithm you copy/pasted, making sure to keep everything symmetric.
Using .Merging Meshes | Babylon.js Documentation
Mmm, more or less agree with you.
It is one thing to read an image with opencv and based on some properties like hair color or wearing glasses return an 3d mesh from a galery which looks like, and other thing to create a 1 to 1 3d object based on an image, like photogrammetry.
It depends on point of view, I was talking about the second scenario