Some variants of the current google cardboard designs as well as the merge vr goggles use simple touch interactions (without any additional controller). How would it be possible to use the babylon gui with these kind of goggles, concering following issues:
gaze interaction: a press on a touch-button (seperate from the gaze target) needs to trigger a babylon gui element. pickWithRay is not an option, as this doesnt work with gui elements (as far is i know).
the non stereo buttons, needed for the touch interaction, probably prevent the use of the webvr helper (as the webvr button only renders the canvas element fullscreen and therefore “removes” the html buttons). Also touch interactions dont seem to work on some phones during webvr-mode.
The webvr helper probably holds the solution to this problem, as the gazecontrollers somehow simulates pointer events.
Also see: https://www.interactive-art.at/experiment/webX/kh.html
Simple Scene which shows the setup: the blue buttons represent buttons, which could be used for google cardboard / merge vr, the fullscreen button calls a custom fullscreen function, which uses the document as fullscreen target (to keep the single rendered html buttons)
Scenario: the gaze target is over a babylon gui element → the user presses one of the blue buttons →
a gui event should be triggered (emulating a regular touch / click on the gui element)
Thanks for the fast reply, but like I mentioned in the initial post I probably can’t use the webvr helper (due to the needed html buttons and the missing of touch events in vr-mode) and therefore I don’t have access to the gaze controller.
I’m currently trying to to create my own emulated click event (fired by one of the buttons, and with the location in the middle of the half-canvas), but I’m not sure if that is even possible for the canvas object - no luck so far.
Any hint to a simpler solution would be great, at his is a somewhat complex hack for such a (seemingly) simple requirement.
Thank you so much @Deltakosh, I think im close now with your help.
Could you take a look at this playground: Babylon.js Playground
It seems as if the gui events arent fired (neither for 2d nor 3d buttons).
Also the regular 2d gui hover / hittest area seems off in stereo view, or are only 3d gui elements working properly in this view mode?
Notice that mouse events no longer works, just sensors. the buttons can be pressed using the f button, hover works, and all other goodies of the UI work as well.
This is not an optimal solution, and should be cleaned a bit, but this is a wonderful starting point!
