Creating an "Idle State" for a VR project

Hi, I have a VR project and want to create an ‘idle’ state; that is, to stop rendering the scene while it is not in use, and then allow the user to reenter (so I can save battery life).

The issue Im having is that, after stopping rendering, I can’t register any pointer down (or other) events on the screen, so cannot retrigger the render loop.

This only happens when in the XR session; doing this in normal/canvas mode I am still able to click on a GUI button despite the rendering being stopped. Im also able to restart rendering from XR using a timeout, so the missing piece is being able to do this manually with a button press.

Im stopping rendering using the stopRenderLoop() method on the Engine.

While in VR I’m using the sessionManager.session.onselect method to handle other click events, but this are not triggered when rendering is stopped…

Does anyone have any ideas how to get a button-like input to a stopped XR session, or another way to fix this issue?

Many thanks :pray:

Let me add @PolygonalSun to the thread to see if he knows VR inputs :slight_smile:

1 Like

Sadly, I’m not too familiar with VR/XR inputs and their respective events. @RaananW might know a bit more here but I do wonder if there’s something that’s causing the onselect event callback to be overridden or if it’s just not calling it. It might be worth trying to see if adding an addEventListener call would react the same.
Example:

sessionManager.session.addEventListener("select", (evt) => {
    // Do something like console.log("event");
});
1 Like

Thank, I have actually already tried that and still no luck.
I was wondering if it was related to the ‘gaze’ XRInput, as that is the only one that seesm to be on the session when its idle, but I cant attach event listeners to that it seems…?

A personal opinion in that matter - there is not a single point that the device is idle, so there is never a need to stop rendering. I’ll explain - XR sessions are usually rendered on a wearable/handheld device. no hand or head is that stable that the device’s rendering loop can be skipped. There are very small changes, but those changes are super important. If you do it in AR (on a mobile device) it might not be so noticable, but in VR it will probably cause some serious headaches.

But to the matter at hand - how do you stop rendering?
I wouldn’t use the session select directly, as babylon provides an abstraction layer for you that should take care of everything. We simulate the pointer system so you can use babylon’s pointer observables and callbacks just as you would on a desktop. Have you tried that?

1 Like

I know what you mean about having a constantly moving headset, but I got around that by putting in a view matrix comparison every 3 - 4 seconds, but manually reducing the ‘resolution’ of the view matrix; ie, simplifying the numbers to 2-3 decimal places, and then comparing them, until I got a reasonable result where a headset left on a table for x amount of seconds would turn idle. So yes you’re right it was certainly an issue but the workaround seems ok for now…

Im not sure what you mean by pointer system, but you’re saying this way would be effectively the same as stopping rendering when it comes to processing/battery life of a device?

sorry just realised about the pointer system you’re talking about.

That is another issue where Im using just a phone in a headset with mechanical buttons, no controllers for pointers, and to get the input (for teleportation etc) I’m doing it manually with onselect, which is working well, but the issue is still that this input doesnt work when the rendering is stopped, which is what i need to save battery life (i think).

That would require overwriting the entire XR render loop (and, again, TBH, would lead to serious headaches after a short period of time). Not sure how you do it, but it will be hard for us to support something like that.
We support our abstractions and support the default render loop. You have full access to the XR objects, but if you change something there you will probably need to debug it yourself, because it’s very hard for us to check that. Having said that - A reproduction would bew great, with the code that is not working.

I think theres some misunderstanding, that part is working (and am happy to explain it, however hacky), but its kind of beside the point, ill rephrase whats not working:

Inside a VR scene, displayed on a phone inside a headset, a custom ‘idle state’ is triggered.
This stops rendering via stopRenderLoop().
Before doing that, it displays a full-screen GUI with a message and button, which has a callback attached.
However, this button (or any event event triggered via onselect, which is whats being used inside the VR session to interact) does not register anything when the screen in pressed, like it would when inside VR, probably because the stopRender() has been called.

I want to be able to register a touch event on the screen, inside a webXR session that has had rendering stopped.

I have been able to leave webXR (at the time the idle state is triggered) and then can get click events on the screen even through the rendering is stopped, but wanted to skip this step

(in terms of triggering the idle state, in pseudocode the steps are:

  • get the viewMatrix as an array of numbers (ie [0.3423423423, 1.4234342, … etc])
  • map the array converting each number to 2 decimal places ([0.34,1.42,…])
  • store this
  • wait 3 seconds (or whatever idle time you want)
  • get and map the current viewMatrix
  • compare these 2 number arrays
  • if different, reset the stored value and timer
  • if the same, trigger idle state

sorry don’t have proper js code with me atm, hope its helpful)

I am so sorry, I really don’t understand. And I think that a reproduction would be very helpful.

The thing I don’t understand is what “stopRenderLoop” (or stopRender) does. Does it just stop babylon’s render loop?
When you register an onselect (or “select”) event directly on the session, babylon has nothing to do with it. Which means - you are doing something in thie stop render function that prevents the native webxr scene from passing the event. Maybe you are exiting the session? Or maybe (and this very WebXR-related) events don’t pass to scenes that are not being rendered (though I doubt it).

yes, heres the link to it, its a method that runs on the Engine:
https://doc.babylonjs.com/typedoc/classes/BABYLON.Engine#stopRenderLoop

So as its not a custom function I’m not doing anything to stop this.

No, I have actually tried exiting the session, and in this case GUI functions, but I don’t want to exit the session

So could it be this, and if so how to test/overcome? :thinking:

hi @RaananW (or anyone else), I’ve made a playground that shows the issue.

It turns ‘idle’ after a setTimeout of 3 seconds, if you press anywhere on the screen before then it will register, but after that point it will not (but will not exit the XR session)

Hope that makes sense?

Again thanks for the help :slight_smile:

hi all, is there any ideas on whats happening here in the above playground?

It seems like select events in xr are not triggered when the render loop is not running.
A click will Queue tasks:

I can only assume those tasks will not execute if the render loop is not running. And it seems like the playground is a great proof for that.
If you disable the active camera it continues working. The render loop runs, but doesn’t render to the XR camera:

Clcik events after stop rendering in XR | Babylon.js Playground (babylonjs.com)