Positioning multiple objects from 3d worlds transformation(position, rotation, scaling) in image/surface tracking

Questions
,
1

Hello, I am working on a small drag-and-drop editor that lets the user select an image and then show it as a plane in a Babylon scene. We then allow the user to place the objects( 3d model, text, image, primitive shapes, etc) over the image plane. Each of these objects can have its own position, rotation, and scaling with respect to the world matrix. When the same transformation is applied in image tracking mode, they seem to be not positioned correctly.

What should be the right way to convert the projection matrix in image tracking mode so that they appear similar to how they have been designed? We tried doing our own transformation, looking for examples and documents to have some reference on it but had no luck moving forward. Any help would be great.

More Information On Scene :

2

I am having a hard time trying to understand what you are trying to achieve or mostly what kind of image tracking you are using. they usually requires the camera setup to be identical.

Let s add @RaananW our XR expert to the thread.

1 Like
3

Yes, it’s hard for me to understand the process as well. Code example (preferably a playground) would be very helpful.

1 Like
4

@sebavan @RaananW: thanks a lot for the quick response.Extending original image tracking example demo, I am sharing below a playground link below. Just like the scene, a user would be placing the objects over the image. Each of these objects has size and transformation properties that are defined in json. We then save the scene as a json data(example added below). What would be to correct way to load this in image tracking so they appear similar to how they appear in 3d scene.

Scene Data Json -

{
   "sceneObjects": [
      {
         "meshType": "GLTF",
         "position": "{x,y,z}",
         "rotation": "{20,30,90}",
         "scaling": "{1,1,1}"
      },
      {
         "meshType": "box",
         "size": "{1,1,1}",
         "position": "{x,y,z}",
         "rotation": "{90,45,30}",
         "scaling": "{1,1,1}"
      }
   ]
}
5

This depends very much on your objects and usecase, but this is the general way of doing this when enabling image tracking:

  1. Get the position of the tracked image
  2. get the tracked image’s reald world width (realWorldWidth on the image result). This value is in meters, which fits the unit scale of WebXR
  3. position an element in the center of the image tracked using the transformation matrix provided
  4. calculate the object’s new position based on the realWorldWidth and the ratio (with which you can calculate the height).