Drag & Drop: From Babylon.js to other web page or native application

[pascal]

I need to implement a Drag and Drop, for now I have working:

• Drag from a web page and drop inside Babylon.js (using the canvas)
• Drag and Drop within Babylon.js (using Babylon.js)

what I cannot resolve is Drag from Babylon.js and drop inside a web page.
I just need to transfer some payload that identify the dragged graphic.

I used engine.getRenderingCanvas(), and then register to ondrag, ondragover, ondrop to receive drop from outside of Babylon.js. and evt.dataTransfer for the payload. That works very well.
I tried to trigger the same mechanism from inside Babylon.js without success.

I even try to make draggable the full canvas of Babylon.js, but I cannot start a drag.

If I can get some hint where to look, a sample will be outstanding

Thank you,
Pascal

[labris]

Hello and welcome!

Interesting question.
Could you tell a bit more about what kind of information (or which objects) you need to be dragged from Babylon canvas?

[pascal]

Thank you Iabris,

We use Babylon.js to show railroad activities.
We show a 2D schematic representation of the railroad with tracks, switches, signals, trains, …

For example, each train is associated with a context including the Train’s UID,
When a drag start, we includes this context to the drag payload.
The drop can be anywhere on one of our web page, Babylon or not.
The drop target looks at the payload to do some contextual processing.

One usage to illustrate can be yard management, where we have a list of available trains (UI) and a yard (graphic), and move trains in or out of the yard.
This is working fine for web page to Babylon.js, we can for example take a component on a page representing a Train, associate the Train context when the drag start, drop it inside Babylon.js, and create a train graphic on the drop position.
We need now to take a train from Babylon.js and drop it outside of Babylon.js.

The only solution I see is to use the canvas drag and drop mechanism to provide a generic way to transport the context. I cannot have the drop target to depend on where the drag was initiated.

I will also need at some point to associate an image to the drag to represent the Babylon.js graphic outside of Babylon.js. But that part start to be cosmetic.

This is just a basic example of the many usage we have for drag and drop between our web pages and Babylon.js, and probably something other can use for different purpose.

Hope that clarify the context.

Pascal

[JohnK]

When a train is picked as well as storing context obtain screen pointer x and y coordinates, create an icon on the webpage and set its zIndex above that of the canvas drag and drop to where you want.

Scene | Babylon.js Documentation

Hope I haven’t misunderstood what you need.

[labris]

Would it be enough just drag it to the canvas border or you need to drag from Babylon canvas to some special place at the HTML page?

[Pryme8]

There is quite a bit to be determined here. Are your elements on the same domain (the canvas and the website you are trying to drag to)? What is the actual context you are trying to transfer?

I would imagine you would just have two separate map objects with the correct contextual values one hosted in the BJS scene and one hosted on the website. On mouse down bind whatever context to some sort of global stash, watch for a mouse up, if the target of the mouse up is different then the target of the mouse down run your process and pull up the associated map context. Rinse wash repeat.

[pascal]

Hi JohnK,

Thanks for the answer, I probably need to reformulate the title to focus more the question, I need to be able to do D&D between different web pages. It is why I’m looking at the OS level D&D, not try to re-implement my own D&D locally.

and what you describe is very near of what I was expecting:

• to set a hook on the canvas when a gesture D&D is detected,
• on dragstart to provide an icon and a payload,
• and the OS take care of the rest. The icon is used to show the dragged object, and the drag target use the payload to process anywhere, including other native application, like a real D&D.

It is basically the API I get 27 years ago on the graphic library I was using, was native but multi-platform (Windows, Unix, Linux).

Thanks,
Pascal

[pascal]

Hi Labris,

no, I see a lot of example on how to D&D within Babylon.js, I use that temporary to provide a D&D within Babylon.js,

But the main usage is outside of the canvas, and in fact outside of the web page. Our application is multi-window, each with a different web page, we have one for our diagram, that contain only the graphic (Babylon.js), and multiple other windows for the rest of the application.

I really need to access the HTML D&D, that I understand is directly linked to the OS D&D.

Thanks
Pascal

[pascal]

Hi Pryme8,

I was really hopping I do not need to implement my own “system” D&D…
I do UI for Train Control since 1991, in beginning we did our own implementation for mostly everything from network, database, and graphic, but we are in 2022…

I’m few months old in Web technology, and started by re-writing our graphic engine using Babylon.js, with my previous experiences with X Server, OpenGL, OpenGL ES, SceneKit, and JavaFX 3D, that part was smooths, my problem is mainly with the rest of the web technology.

Our graphic engine is a common framework use by multiple projects, I have very limited dependency outside graphic, I cannot and should not need to bring a custom web server design with it.

We still have native application, we can expect someday to do D&D between Babylon.js and a native application, I was hoping to have a payload like a little JSON Object, with all the context needed to process the drop and use the HTML D&D mechanism. Was a simple and universal design, and it is currently working beautifully when the drag came from anything on web page but Babylon.js.

I try:

    <div draggable="true" ondragstart="drag(event)">123</div>
    <br>
    <img id="image" draggable="true" ondragstart="drag(event)" src="./texture/Logo.png">
    <br>
    <canvas id="canvas" draggable="true" ondragstart="drag(event)" width=400 height=300 ></canvas>
    <br>
    <canvas id="renderCanvas" draggable="true" ondragstart="drag(event)" width=400 height=300></canvas>

    <script>
      function drag(ev) {
        console.log("drag");
        ev.dataTransfer.setData("text", ev.target.id);
        ev.dataTransfer.effectAllowed = 'copy';
      }
    </script>

I can:

• drag the label “123”,
• the image “Logo”,
• the empty canvas with just one background color,
• but the canvas "renderCanvas " associated with Babylon.js is not draggable. Unexpectedly, a very small border at the extremity of the canvas is draggable, probably the border that indicate the focus.

I understand Babylon.js is probably capturing mouse events that may block the standard HTML D&D mechanism, and I will be fine to handle mouse events internally, recognize the Drag gesture, but after I do not see how to “manually” start a canvas Drag.

Hope that clarify,

Thanks,
Pascal

[jeremy-coleman]

I think johns solution is very practical.

On mouse down:
Raycast to get metadata of babylon train model or return early
Create dom node under mouse
Insert train data into dom node
Drag dom node
Change scene state on sucessful drop

Also, Another option is to use the clipboard to do the data transfer instead of event data

[pascal]

Hi Jeremy-coleman,

I din’t read johns answer this way, I miss the idea of “dom node” outside of Babylon.js,
Very interesting idea indeed…

Thanks you JohnK and Jeremy-coleman, I will try this promising approach.

Pascal

[jeremy-coleman]

I suspected so:) good luck

[pascal]

Hi @JohnK,

This idea is working, the only change I did was to create a dom node before the pointer down event, else all future D&D events goes to the node that initially received the point down, eg Babylon.js.
I also dispatch all events I needs from this node to the Babylon.js canvas.

So far look good and I didn’t see yet side effect.

Thank you all for your answer
Pascal

[VerandaLouis]

@pascal

Drag from a web page and drop inside Babylon.js

Hi,

i’m interested in this part, i would like to reproduce this on my project, can you send/show me your code for this ?

thanks by advance

[wulf11]

hi @VerandaLouis
you may check this post:

Mesh follow mousepointer in HTML-DragOver-Event

Using again your code from Version 4 brought me a ‘breakthrough’ check: https://www.babylonjs-playground.com/#F7ZZJF#6 (need to proof it a bit more, but could not hold it back to share) I used the code you supplied and just changed the the pointer.X to evt.offsetX! Will now try it in my project code where the canvas has other dimensions. I hope that will not affect the offsetX values in a way that it breaks the behavoir. Thank you very much for bringing me here

[pascal]

Hi VerandaLouis,
For this part I used standard HTML 5 Drag & Drop.
To make a component draggable:

<body>
  <div draggable="true" ondragstart="drag(event)">123</div>
  <br>
  <img id="image" draggable="true" ondragstart="drag(event)" src="./texture/WabtecLogo.png">
  <br>
  <canvas id="canvas" draggable="true" ondragstart="drag(event)" width=400 height=300></canvas>

  <script>
    function drag(ev) {
        console.log("drag");
        ev.dataTransfer.setData("text", ev.target.id);
        ev.dataTransfer.effectAllowed = 'copy';
    }
  </script>
</body>

and to get the drop, I used the rendering Canvas method:

  const canvas = this.camera.getScene().getEngine().getRenderingCanvas();
  canvas.ondrop = evt => {
    const data = evt.dataTransfer?.getData('text');
  }

For data it is what you want to identify the correct processing in Babylon.js to apply upon drop.
Low level, but working very well.

Pascal