Plane Detection and Model Positioning

Questions
1

I am currently working on an augmented reality application that consists of placing an object based on two detected planes, one vertical and one horizontal.

Let’s say that the object is a cube and when finding a vertical and a horizontal plane, this cube is placed parallel to the floor and the wall, preserving the orientation of the vertical plane.

I’m having trouble orienting the object correctly, as sometimes the object takes a totally different rotation than the plane.

this._xrPlanes.onPlaneAddedObservable.add((plane: IWebXRPlaneWithMesh) => {
            mat = new StandardMaterial("mat", this._scene);
            mat.alpha = 0.35;
            mat.diffuseColor = Color3.Random();
            this.initPolygon(plane, mat);
            
            let transformationMatrix = plane.transformationMatrix.m;
            let planeMatrix = BABYLON.Matrix.FromArray(transformationMatrix);

            console.log("PlaneMatrix:");
            console.log(planeMatrix);

            normal = new BABYLON.Vector3(
                planeMatrix.m[8],
                planeMatrix.m[9],
                planeMatrix.m[10]
            );

            normal.normalize();

            position = BABYLON.Vector3.TransformCoordinates(
                BABYLON.Vector3.Zero(),
                planeMatrix
            );

            if(plane.xrPlane.orientation.match("Horizontal")){
                console.log("Horizontal plane");

                if (conH < 1) {
                    floorNormal = normal;
                    console.log(floorNormal);
                    model.position.y = position.y + cubeSize / 2;
      
                    conH++;
                }
                
            }else if(plane.xrPlane.orientation.match("Vertical")){
                console.log("Vertical plane");

                if (conV < 1) {
                    console.log(wallNormal);
                    model.position.x = position.x - cubeSize / 2;
                    model.position.z = position.z - cubeSize / 2;
      
                    const planeMatrix = BABYLON.Matrix.FromArray(transformationMatrix);
                    const planeNormal = new BABYLON.Vector3(planeMatrix.m[8], planeMatrix.m[9], planeMatrix.m[10]);
                    planeNormal.normalize();
      
                    let angle = Math.atan2(planeNormal.x, planeNormal.z);
                    model.rotation.y = angle + Math.PI / 2;
      
                    conV++;
                }
            }
            console.log("====================================================");
        });

It is possible that I am doing something wrong or I am missing information, since I am new to using babylonjs and also with augmented reality.

Any tips?

1 Like
2

cc @RaananW

3

Is the rotation incorrect around the normal ? Want to share a playground we can both play with? That would help me debug it together with you

4

So, I’m developing the app with React and Typescript in my local device, so I really don’t know how to adapt the project to the playground. Maybe I can share my github repo (GitHub - jlopez-02/babylon-ar-ts).

The problem is that when detecting 2 planes, the model does not acquire the desired orientation most of the time.

Anyway I will be trying to create a playground.

1 Like
5

Here you have the playground:

6

i’ll probably need an explanation as to what the expected behavior is. I scanned a section of a room with a horizontal and vertical plane, and the box just stayed on the floor. Is that the issue?