@vijay_krishna I am still a little confused about the 3D model you are looking to create. Let me level set here.
- I assume do you not want a quad holding the image as that’s easy to create and doesn’t help with the problem of dynamically changing the material of the clothes.
- I also am assuming, but want to clarify, you are not looking to make an extrusion of the silhouette of the person as this wouldn’t get you much further than the quad solution. And by an extrusion I mean something like a 3D letter
- I assume you are talking about a fully dimensional 3D character since you mention “changing the shirt and trousers” which could mean changing the material of the shirt (to a solid color instead of a plaid pattern) or it could mean changing the long-sleeve dress shirt to a short-sleeve polo shirt. Which do you mean here?
If there was an automated way to create a 3D character from a photo, we wouldn’t need any 3D artists anymore. You can buy a 3D character mesh, or hire artists to make a few generic characters to use, but no real automated way to convert an image to 3D.
I assume that your challenge lies in the fact that the folds on the shirt will deform any pattern and to make the pattern look like it is on the shirt you need to deform that pattern over a 3D surface which is why you are looking to convert to 3D.
In that case, I would take the approach of modeling out just the shirt and trousers because you don’t need a fully rigged character just to do a material swap, and you still get the photo realism of the face and hair. You can then render out the models with the new materials matched up with lens properties and camera angle and create a paper doll approach of layering the rendering of the clothes over the image of the person. The hand in the pocket will require a little more effort as the hand will need to be a layer on top of everything else.
Drawbacks here are:
- it’s expensive to change the pose of the person modelling the clothing as you will need custom meshes for every pose or a rig for the clothes so you can change the pose on a single mesh
- you could need a cloth simulation system if you want to change the pose of a single mesh so that you get realistic cloth deformations. Maybe something like Marvelous Designer, but this is another tool in the pipeline that has its own expertise requirements.
- As your library of people modeling clothing grows, you will need an increasing number of assets to fit their proportions and body shapes so the images look natural.
The only other option this leaves is to remain in 2D for the entire pipeline and create your overlays for materials through image editing and some manual deformation of pixels to match up with the underlying clothing surfaces. In any case, none of this is easy to automate so you will need artists with expertise in either 2D or 3D asset creation (depending on the path you take) for this problem space.