Get rotation and position from 4 points

Questions
1

Hi!

I’m working on some image tracking with QR codes for an AR BabylonReactNative project. I’ve got the scanner part and am getting both the data from the QR code and the 4 cornerpoints in screenspace.
I thought I could get position and rotation from those but I don’t seem to be able to do that.

I’ve tried

const position: Vector3 = Vector3.Unproject(
              new Vector3(corner.x, corner.y, 1),
              engine.getRenderWidth(),
              engine.getRenderHeight(),
              Matrix.Identity(),
              scene?.getViewMatrix()!,
              scene?.getProjectionMatrix()!,
            );

with the separate corner points but that gives me numbers that are in the thousands.
I’ve had a little more success with

var pickResult = scene?.pick(
    Number(corner.x),
    Number(corner.y),
);
const distance: number = 0.3;
if (pickResult != null && pickResult.ray) {
position.x =
 pickResult.ray.origin.x +
 pickResult.ray.direction.x * distance;
position.y =
 pickResult.ray.origin.y +
 pickResult.ray.direction.y * distance;
position.z =
 pickResult.ray.origin.z +
 pickResult.ray.direction.z * distance;

Which is better, the coordinates make more sense, but it’s still just the individual points. I also have to hold the camera at a set distance for this to work…

Googling doesn’t really give me any answers, but I may be using the wrong search terms. Could anyone help me figure out how I get the position and rotation from the 4 cornerpoints I have?

Thanks!

2

I think I found what I’m looking for: Plane | Babylon.js Documentation
it doesn’t seem to be entirely correct yet though, as the planes I’m generating are way too big.

It seems that my way of transforming screenspace to worldspace isn’t working out, as all my planes are sort of pointing towards me, as if on a sphere. Which makes sense with the fixed distance. I get the feeling I’m also missing something obvious there.
Also, how do I update the mesh plane with data from the math plane? I got it to transfer when creating it, but how do I update it?

2 Likes
3

You should use 0 instead of 1. 1 means the farthest point (as the range is 0…1), that’s why you get huge coordinates.

However, I think the picking solution should be the best as it will give you the true 3D coordinates of the point. You don’t need any computation to get the 3D point, just use pickResult.pickedPoint.

1 Like
4

Ah that makes sense! Thanks for clarifying that, I’ll go with the other method as you recommended.

5

In case someone stumbled across this question also trying to do a thing with QR codes. I realized it’s quite simple when you combine the screen coordinates with AR. You just cast a ray, save that with the QR code, and when you have a second ray see where they are closest!

I rewrote a function I found in this Unity math library (3d Math functions - Unify Community Wiki) to:

const ClosestPointOnTwoLines = (ray1: Ray, ray2: Ray): Vector3 => {
const lineVec1: Vector3 = ray1.direction;
const lineVec2: Vector3 = ray2.direction;
const linePoint1: Vector3 = ray1.origin;
const linePoint2: Vector3 = ray2.origin;

const a = Vector3.Dot(lineVec1, lineVec1);
const b = Vector3.Dot(lineVec1, lineVec2);
const e = Vector3.Dot(lineVec2, lineVec2);
const d = a * e - b * b;
if (d !== 0.0) {
  const r = linePoint1.subtract(linePoint2);
  const c = Vector3.Dot(lineVec1, r);
  const f = Vector3.Dot(lineVec2, r);
  const s = (b * f - c * e) / d;
  const t = (a * f - c * b) / d;
  const resultLine1 = linePoint1.add(lineVec1.scale(s));
  const resultLine2 = linePoint2.add(lineVec2.scale(t));
  const result: Vector3 = resultLine1.add(resultLine2).scale(0.5);
  return result;
} else {
  return Vector3.Zero();
}
};

And badabing badaboom got myself some cornerpoints in AR space!

1 Like