How to convert screen coordinates to world coordinates, must there be a plane in the scene?

Questions
1

https://playground.babylonjs.com/#7CBW04#812

Or is it possible to use a temporary plane, just to calculate

2

Hi!
I’m using this helper function. When converting screen position to 3d you have to define the z value (depth) of the click position yourself because there is no way to get the correct 3d z position without setting this value. So the best is to have a plane to get the correct 3d position or adjust the z value for the mouse click yourself const screenPosition = new Vector3(x, y, 1)

export const screenToWorld = function(x: number, y: number, engine: Engine, scene: Scene): Vector3 {
  const screenPosition = new Vector3(x, y, 1)
  const vector = Vector3.Unproject(
    screenPosition,
    engine.getRenderWidth(),
    engine.getRenderHeight(),
    Matrix.Identity(),
    scene.getViewMatrix(),
    scene.getProjectionMatrix()
  )

  return vector
}
4 Likes
3

Thank you for your reply, I tried this method, but the result is wrong

result:
Vector3 {_isDirty: true, _x: 4258.59808640104, _y: -9169.523426204225, _z: -652.7164912657977}
4

Try to adjust the z value of the mouse click.

5

Try this:

https://playground.babylonjs.com/#XWXBNB

You will see the sphere following the mouse.

EDIT: please note that the 3d coordinate’s z value is affected by the minZ value of your camera

4 Likes
6

You can also create an invisible plane and dispose of it when it’s not needed anymore: Drag Demo | Babylon.js Playground (babylonjs.com) (changes on getGroundPosition function)

2 Likes
7

https://playground.babylonjs.com/#7CBW04#822
When dragging, the result is wrong, maybe because of the coordinate’s z

9

Could you explain the difference between screen coordinates and world coordinates?

10

Hey @pigga. Screen coordinates are the coordinates you see on your 2D screen. For example the top left pixel of your screen would be defined as screen coordinates (0,0). World coordinates are the points in 3D space. For example (0,0,0) would be the center of your world. :slight_smile:

Here’s an article with additional information as well.
https://www.cs.uic.edu/~jbell/CourseNotes/ComputerGraphics/Coordinates.html

3 Likes
11

Can I not add mesh to the scene?

12

Ok, I can see that you didn’t get the idea behind the z value of the mouse click, because you set it to 0. Imagine, you have infinite number of invisible planes in front of your camera, all facing your camera. If you set the z value to 1, you will hit the first invisible plane in front of your camera, so basically your 3d z position will be the minZ value of the camera. By lowering the z value of the mouse click you can hit planes located farther away.

However if you are into dragging meshes, here is a PG I’ve quickly created to demonstrate, how we can use an invisible plane. The plane will show when dragging to get the concept behind this technique.

https://playground.babylonjs.com/#KZ3LSC#3

There are many possible ways of doing this and this is just one of them.

2 Likes
13

Thanks,i get it

1 Like
14

You are welcome!

15

Another great link to learn more about coordinate systems is LearnOpenGL - Coordinate Systems :grinning_face_with_smiling_eyes:

3 Likes