[Drag & Drop] Inconsistent pickedPoint Vector3 when camera is zoomed in vs. out

[deet_dev]

Hi Babylon.js team,

I’m currently developing a 3D editor using Babylon.js, where users can drag and drop landmarks onto meshes. During this, I encountered an issue where the pickedPoint (from scene.pick()) seems to significantly differ depending on the zoom level of the camera, even when dropping on the exact same screen position (i.e., same canvas pixel coordinate).

---

Context

• I’m using scene.pick(x, y) with coordinates derived from a DragEvent.
• The red dot at (0,0,0) is a known mesh.
• When dragging and dropping onto the center of the mesh, the pickedPoint differs significantly depending on the zoom level.

---

Example

• Zoomed in result:

pickedPoint: Vector3 {x: 0.2746..., y: 0.3..., z: 0.0074...}

• Zoomed out result:

pickedPoint: Vector3 {x: 1.7239..., y: 0.3..., z: -0.4043...}

• As you can see, the difference is over 1 unit, despite the drop happening visually at the same center point.

---

Screenshots

(Attached below — showing zoomed-in vs zoomed-out drag-drop at the same visual location)

---

Drag logic in use

Here’s the code used to perform the picking:

ts

복사편집

const mouseX = e.clientX - canvas.getBoundingClientRect().left;
const mouseY = e.clientY - canvas.getBoundingClientRect().top;

const pickResult = scene.pick(mouseX, mouseY);
if (pickResult.hit && pickResult.pickedPoint) {
    const localPos = Vector3.TransformCoordinates(
        pickResult.pickedPoint,
        Matrix.Invert(pickResult.pickedMesh.getWorldMatrix())
    );
    ...
}

---

Question

Why is there such a large difference in pickedPoint between zoom levels?
Is this a precision issue or camera-related ray generation problem?
How can I get consistent results regardless of zoom level for drag & drop?

---

Any advice or direction would be much appreciated. Thanks in advance!

[11128]

From what I’ve seen, it looks like the camera isn’t a central axis gizmo…

check the PG

1. click to black box
2. create whte box
3. check the pos

There may be some difference depending on the precision, but it seems to be a fine range that needs to be adjusted when placing.

[Tricotou]

Hello

A few comments :

---

@deet_dev

I encountered an issue where the pickedPoint (from scene.pick()) seems to significantly differ depending on the zoom level of the camera, even when dropping on the exact same screen position (i.e., same canvas pixel coordinate).

When you change the camera zoom, any 2D points of the canvas will map to a different 3D points, except only 1 point, the one which is exactly at center of the camera. Are you picking the exact center ? If not, then the result is expected, I would say

---

@deet_dev

const mouseX = e.clientX - canvas.getBoundingClientRect().left;
const mouseY = e.clientY - canvas.getBoundingClientRect().top;
const pickResult = scene.pick(mouseX, mouseY);

Better would be to use

scene.pick(scene.pointerX, scene.pointerY)

To make sure you are using the right mouse coordinates

[deet_dev]

Hello, thank you for your response.

I used the code you mentioned, and I don’t know if it’s because of the left panel part of the image below
There was a serious error in the code.

So the solution I found after asking the llm model such as gpt4o is that code.
Attached is the screen below.

[deet_dev]

hello thank you your response.

I tried to run the playground you showed me.

This error did not occur when I just mouse-clicked in scene.

Add drag-and-drop function and drag outside the scene to drop inside the scene, resulting in an error.

Why is that?

[11128]

Your words and description do not give us any idea about the environment within the project you have implemented.
So you need to repro it and make it functional in a simple way.
Can you make a PG with add drag-and-drop function

[deet_dev]

Babylon.js Playground

here i make a PG

I felt while making it, will the camera panning that moves the camera’s position have a lot of influence on the coordinates you get by clicking on the mouse pointer?

It doesn’t seem to be happening in this example pg, but I’m still having issues with my project.

Please advise if there is anything I am missing.

[Tricotou]

Hello @deet_dev

I think we would be glad to help but your description of the issue is quite incomplete, for example when you say :

@deet_dev

drag outside the scene to drop inside the scene, resulting in an error.

Why is that?

Without giving the “error”, impossible to answer to the “Why ?”

---

Also, I had a look at your PG, adding a cube target as debug on drop seems to be ok :

Babylon.js Playground

Did you try to use scene.pick(scene.pointerX, scene.pointerY) in your project ? As I said, it more reliable than any compute on the canvas object within a custom complex Dom…

---

Also, just a comment about your grid :
You could create this with a simple plane and use a Grid Material

[deet_dev]

Yes, I’ve used scene.pointerX and scene.pointerY.

But that wasn’t the solution.

I read the comments from the community and checked what’s wrong with my code,

I found the reason.

The reason is that within the dom video, canvas does not start at top and left at 0,0 but as in the image below
There’s about a top 50, a left 50, and the problem was using boundingClientRect() for the whole of the dom, not the canvas.

[Tricotou]

@deet_dev

Yes, I’ve used scene.pointerX and scene.pointerY.
But that wasn’t the solution.

@deet_dev

I found the reason.

The reason is that within the dom video, canvas does not start at top and left at 0,0 but as in the image below
There’s about a top 50, a left 50, and the problem was using boundingClientRect() for the whole of the dom, not the canvas.

I a bit confused here since, if you use scene.pointerX and scene.pointerY there is no need to use boundingClientRect() on any dom element

[deet_dev]

hmm..
When I heard what you said, I thought I could use that and applied it, but there was an error. Theoretically, it should be, but it’s weird not to.

[11128]

Congratulations on solving the problem, but as @Tricotou said, it seems like the projected scene should have the same values, but if the size of the components on the left and top are changed a bit, will an error in the position values ​​occur proportionally?

The event for drag is expected to have the same scene.pointerX and scene.pointerY values ​​regardless of whether it is triggered on DOM, canvas, or engine.

        document!.addEventListener("click", (e) => {
            e.preventDefault();
            const mouseX = getScene().pointerX;
            const mouseY = getScene().pointerY;
            const pick = getScene().pick(mouseX, mouseY);
            if (pick.hit && pick.pickedPoint) {
                const point = pick.pickedPoint;
                console.log(`x: ${point.x.toFixed(4)}`);
                console.log(`y: ${point.y.toFixed(4)}`);
                console.log(`z: ${point.z.toFixed(4)}`);
                console.log(`drop at ${mouseX}, ${mouseY}`);
            }
        });

        canvas!.addEventListener("click", (e) => {
            e.preventDefault();
            const mouseX = getScene().pointerX;
            const mouseY = getScene().pointerY;
            const pick = getScene().pick(mouseX, mouseY);
            if (pick.hit && pick.pickedPoint) {
                const point = pick.pickedPoint;
                console.log(`x: ${point.x.toFixed(4)}`);
                console.log(`y: ${point.y.toFixed(4)}`);
                console.log(`z: ${point.z.toFixed(4)}`);
                console.log(`drop at ${mouseX}, ${mouseY}`);
            }
        });

        getScene().onPointerPick = ((e) => {
            const mouseX = getScene().pointerX;
            const mouseY = getScene().pointerY;
            const pick = getScene().pick(mouseX, mouseY);
            if (pick.hit && pick.pickedPoint) {
                const point = pick.pickedPoint;
                console.log(`x: ${point.x.toFixed(4)}`);
                console.log(`y: ${point.y.toFixed(4)}`);
                console.log(`z: ${point.z.toFixed(4)}`);
                console.log(`drop at ${mouseX}, ${mouseY}`);
            }
        });