Scene fails to observe `pointerup` when pointer captured from external html

Bugs
1

Hi.

My intention:

Implement feature of dragging stuff from page’s plain html into scene. (actually, from shadow dom)

My plan:

  • in onpointerdown handler of html buttons or something initialize initial picking rays and simulate startDrag or whatever needed for gizmos
  • call the_rendering_canvas.setPointerCapture(pointerId)
  • hope for the engine and scene to handle all further motion and droppping

My fail:

  • scene’s onPointerObservable doesn’t see pointerup event and doesn’t trigger anything related like stopping drag behaviors

My playground:

(The playground code manipulates page html and adds a button)

Test behaviour:

  • grab the button
  • drag into scene
  • release in the scene

Expected logs:

  • button pointerdown
  • (moving skipped)
  • canvas pointerup
  • scene pointerup

Actual result: the canvas pointerup is logged, but scene’s pointerup isn’t.

Maybe that’s actually how it should work, maybe because pointerup automatically releases pointercapture somewhere between canvas and whatever input managers attached.

The question is how to acheve the desired feature (without redrawing all GUI in babylon)?
Are there any workarounds?
Do I need to simulate pointerUp with recreating all picking rays again?

2

Ok, here’s kinda workaround with simulating both pointerdown and pointerup

Not sure if it would work in production though.

3

cc @amoebachant

4

Hi @qwiglydee, I’m glad you found an approach that works in the playground.

To know if it’s going to work for your scenario, I’d need to better understand what you’re trying to do. When the user starts a drag from the HTML page outside of the canvas, then moves the mouse over the canvas, what do you want to have happen in the scene? Should it be dragging an object already in the scene but off-camera under where the mouse was, should it be adding an object t the scene under the mouse pointer, or something else?

Thanks!

5

Hi!

My use case in general is: drag’n’drop items (of furniture) from html menus to the scene (of the room being designed).

I’m actually performing lots of overcomplicated stuff: activating my custom super-gizmo that contains several nested gizmos and creates a ghost-mesh of picked item, and then switching to “moving” sub-gizmo and simulate startDrag of it’s handle. The process involves calculating picking ray, finding initial position in the scene, and passing all that to my gizmos.
Apparently, that’s the most natural way of working with furniture :smirking_face:

But the key issue is that pointerup event doesn’t happen in scene, and the gizmos stuck in their dragging state.

Given the complexity of the process, simulating pointerup event with another picking ray isn’t really a big deal for me.
But the situation in general seems somewhat inconsistent: html canvas receieves an event, but input manager ignores it.

On another hand, neither the canvas, nor the scene don’t receive initial pointerdown event (handled completely in external html), and that seems how it’s designed to work