Simulating virtual Babylon JS input into an iFrame? Custom controllers?

Questions
1

20190729183416_1
20190729183416_1.jpg1151×1152 170 KB

I am trying to send fake keyboard input into a iFrame running BabylonJS. The reason is I want to create custom controllers in VR. I don’t have a keyboard in VR so I need to fake one.

I am trying to play Babylon.JS Minecraft so I can’t edit the source code.

Normally faking keyboard input to an iFrame is impossible because of security reasons. But I wonder for Babylon.js if there might be other ways do to input through javascript.

Is it possible to create virtual input into a Babylon JS app through an iframe?

2

This is not entirely true. You can pass messages between your window and the iframe, and react to this input. A quick search brought me this - https://gist.github.com/pbojinov/8965299 , but i guess there are better tutorials for that. It is only a security issue when the iframe doesn’t know it is being “manipulated”.

3

Well there are 2 kinds of iframes I know of, cross-origin iFrames and same-origin iFrames.

With same-origin, you can do just about anything without security concerns. That github example is same-origin. They show a good technique to make event listeners too. The Babylon.js Minecraft site is cross-origin.

Does Babylon.js have default event listeners that could possibly receive input messages relating to controls over a cross-origin iFrame?

4

The post message function doesn’t care about cross origin (first sentence here : Window.postMessage() - Web APIs | MDN)

Babylon itself does not have this functionality, but it can be implemented by you and added to your scene’s logic

5

Dang I really just want to play Minecraft with this in multiplayer VR with my friends in multiplayer Minecraft. I don’t want to go down the road of a hack to inject code into the Babylon.JS scene. I feel like hacking babylon is probably frowned on here.

I really love the idea of cross-origin input lettings babylon sites communicate ontside their ‘box’. I really hope one day more websites are designed with this use case in mind. Thanks for the help!

6

Sorry for a late reply :slight_smile:

Dont forget that there IS a reason for blocking cross-origin content. It is nice to have, but easily exploitable.
I don’t see a reason why Babylon should integrate such a feature, mainly due to the security problem it presents when implemented incorrectly. You dont have to “hack” babylon. If you are the creator of the scene, you can very simply integrate the feature into your code. If you are not - oh, then you will need to communicate with the main developer and ask for this to be implemented.