Mesh follow mousepointer in HTML-DragOver-Event

Hey guys I am struggling a bit with a function that I want to implement in my code and could need some advice:

The idea is to drag a HTML image over the renderCanvas and instead of showing the image, I want the mesh to appear and follow the mousepointer, so that you can place it in the scene like a drag and drop function. The mesh has a pointerDragBehavior, which works pretty nice when dropped. But the process of dragging-over works quite bad.

Here is my Playground example:

For the dragover-event I use a function (setPositionToPointerXZ):
(found in:

var setPositionToPointerXZ = function(evt, obj){
    var pickResult = scene.pick(evt.offsetX, evt.offsetY); //works only in the ground area!
        //if i add Math rounding like Math.round(pickResult.pickedPoint.x) it works better
        obj.position.x = pickResult.pickedPoint.x;
        obj.position.z = pickResult.pickedPoint.z;

I encounter the following problems:

  1. the positioning is imprecise and kinda wobbles (if I round the pickedPoint it gets better but depends on the mesh)
  2. the method is slow on some computers (the dragover event maybe gets fired too often)
  3. pickedPoint only has a value if I hit the ground-mesh (is there a way to get a pickedPoint withouth any ray-hits?)

And at last my evt.dataTransfer.setDragImage(new Image(), 0, 0); to hide the dragged image only seems to work in firefox…

Many questions :sweat_smile:
Any advice is appraciated!

  1. Not entirely sure if or why this might be the reason but I think it might do with the setting of the mesh y position on line 126

repositioning the new instance with the scene X and Y values seems to my eyes to prevent a problem but that might be just a coincidence

When and if I have time later today I will look to doing another version using aspects that babylon.js provides such as GUI and Gizmos.

As promised an alternative version (just hope it is close to what you need) still needs some tidying up

Drag from skull image to place new skull, then use gizmo to re - position. After creating multiple skulls click on one to add positional gizmo.

Placement by pointer from


wow many thanks that looks very promising!! :smiley:

Will this example still work with HTML-images ? I see you used the babylonGUI here,
I need a crossover later that can interact with HTML elements…:sweat_smile:
That’s why I used the jquery-stuff in the Playground.
The idea was to build a GUI in HTML and handle the 3D stuff in the babylon canvas

Don’t see why not. You already deal with the pointer down over your HTML image just adapt this. Cannot see anything that would not be available to you

Now I had the time to take a closer look and the way you solved it is really nice!
I still have problems to figure out how to do it without the use of Babylon.GUI…
here is my example (with your Gizmo in it):

The problem here seems that the scene.onPointerObservable.add((pointerInfo) that you use, does not fire/update, because the renderCanvas has no “focus” while dragging the image! The dragging of the image happens kinda outside and the scene.Pointer won’t update anymore :thinking: I haven’t tested it yet, but guess that we will have the same behavoir with pointer down…

That was btw the reason I used the evt.offsetX, evt.offsetY in the first code I provided :face_with_monocle:

Please note that I am a beginner in webtech and maybe got things wrong :sweat_smile:

Had a go using HTML Image and there seems to be a problem switching from events from canvas type to scene type. The best I have is

Click on image to add skull at origin, click on skull for gizmo, move skull with gizmo repeat. This may be too simplified for what you want.

Using again your code from Version 4 brought me a ‘breakthrough’ check: :relieved:
(need to proof it a bit more, but could not hold it back to share)

I used the code you supplied and just changed the the pointer.X to evt.offsetX!
Will now try it in my project code where the canvas has other dimensions. I hope that will not affect the offsetX values in a way that it breaks the behavoir.

Thank you very much for bringing me here :heart_eyes: