I have a project where I am recording all of the users mouse interactions and pointer events for later playback.
Everything works great except to store the PointerInfo, I basically store my own info for the coordinates and what interaction it is, then do a scene.pick() with that data to recreate the pointerInfo to pass to the input manager to simulate the interactions.
I would like to be able to do it without having to rebuild the entire pointerInfo by running a pick.
Any ideas?
cc @amoebachant
My coworker @FrameVRWill , suggested that instead of doing the scene pick and running the simulated pointerEvents on the inputManager that I should emit synthetic mouse events and let the DOM just handle everything so that way we are really running through the same code conditions as the recording.
Would that work?
I think you could create a simulated ‘PointerEvent’ set bubbles: true and call document.dispatchEvent(mySimualtedPointerEvent). It should work.
Hi @Pryme8,
Yes, I agree with @roland and @FrameVRWill and recommend playing back the events at the DOM level to ensure all of the internal state in Babylon is the same when you play it back. Here’s a quick playground demonstrating how you could do it:
Hope that helps!
That is close to what I am doing already. Ill take a look, the only things I have stored though are the pointerXY, the type, pickedMeshId and the timestamp. So I don’t think that solves the reconstruction of the pointerEvent (but it might).
I ended up with this:
It captures the DOM events and then replays them as synthetic DOM events. Try to use the gizmos and replay the events. This is just a basic example capturing the 3 main events (up, down, move) but works quite well:
@roland You’re always doing cool stuff, aren’t’ you? 
On a side note Ive noticed the pickingInfo event is set as a IMouseEvent instead of an IPointerEvent which is causing code hinting to be wrong for the properties that are under it.
When you console log it out, its for sure a IPointerEvent.
Haha, this is actually working too well now and it created camera movement conflicts between my recording and inputs. But that is a great problem to have. This will get me exactly where I needed to be thank you. Got it all working now without having to recreate the pickingInfo.
So if you don’t need further assistance, maybe you could mark it as a solution and close the topic 
