# Screen x,y coordinates to world coordinates

**URL:** <https://forum.babylonjs.com/t/screen-x-y-coordinates-to-world-coordinates/7402>\
**Category:** Questions\
**Created:** [December 5, 2019, 4:50pm UTC](https://forum.babylonjs.com/t/screen-x-y-coordinates-to-world-coordinates/7402 "2019-12-05T16:50:03Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![local\_dev](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/local_dev/32/5917_2.png) [@local\_dev](https://forum.babylonjs.com/u/local_dev)\
**Post date:** [December 5, 2019, 4:50pm UTC](https://forum.babylonjs.com/t/screen-x-y-coordinates-to-world-coordinates/7402/1 "2019-12-05T16:50:03Z")

</div>

Hi,

I’m using tracking.js to get the faces coordinates from an image.

```
coords = {
  x : 38.12355469963794,
  y : 96.47419166998122,
  width : 106.33406420946123,
  height : 102.4978113412857
}

```

I’m an absolute 3D and Babylon beginner, so far I managed to project the image texture on a plane with the right aspect ratio. But when it comes to draw a box over a face with the given `coords`, I just can’t position get the right position.

Here is my attempt : [https://www.babylonjs-playground.com/#0CSKL1](https://www.babylonjs-playground.com/#0CSKL1) (the black box is suppose to be over the guy with the glasses)

Tried many configurations :

```
scene.createPickingRay(
   coords.x,
   coords.y,
   BABYLON.Matrix.Identity(),
   camera
).direction;

BABYLON.Vector3.Unproject(
  new BABYLON.Vector3(coords.x, coords.y, 0),
  engine.getRenderWidth(),
  engine.getRenderHeight(),
  BABYLON.Matrix.Identity(),
  scene.getViewMatrix(),
  scene.getProjectionMatrix()
)

faceBox.setPositionWithLocalVector(...)

faceBox.position = ...

```

Thanks.

---

<div class="post-metadata">

**Author:** ![Gijs](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/gijs/32/44_2.png) [@Gijs](https://forum.babylonjs.com/u/Gijs)\
**Post date:** [December 5, 2019, 10:00pm UTC](https://forum.babylonjs.com/t/screen-x-y-coordinates-to-world-coordinates/7402/2 "2019-12-05T22:00:31Z")

</div>

Hello and welcome!

It looks like what you need is to transform the _picture’s_ local coordinates to the world coordinates, like this:

[https://www.babylonjs-playground.com/#0CSKL1#3](https://www.babylonjs-playground.com/#0CSKL1#3)

In the code on your playground, you create a picking ray, but you don’t actually use it to pick anything with! (`scene.pickWithRay`). (You can also pick with `scene.pick` or create a new `BABYLON.Ray(...)` and use `pickWithRay`).  
A ray has an origin, a direction, and optionally a length. Those are used by BJS to cast the ray into the scene and determine what gets hit, so setting the position of the box to the direction of the ray doesn’t achieve much of anything… BUT if you get a distance from a picking function, you can do `mesh.position = ray.origin.add(ray.direction.scale(distance))`

---

<div class="post-metadata">

**Author:** ![local\_dev](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/local_dev/32/5917_2.png) [@local\_dev](https://forum.babylonjs.com/u/local_dev)\
**Post date:** [December 5, 2019, 11:19pm UTC](https://forum.babylonjs.com/t/screen-x-y-coordinates-to-world-coordinates/7402/4 "2019-12-05T23:19:39Z")

</div>

Thanks a lot for the explanation and the clear code.

I guess, still have a few more weeks to master Babylon, but this is encouraging, so thanks again.