Occulus Browser with hand tracking change has hit

Questions
1

I just tried my headset, after a week of finishing up putting syllable delimiters in my ARPABET dictionary, and lived. Because I have not run the headset in that time, I do not know when it started failing.

I have endless other stuff, so I can adjust my activities, but letting you know it now fails. I get an exception in _attachHand(). I am not sure if this is everyone, or because the hand meshes which I modified need changing now.

I also get an exception in HTMLCanvasElement._pointerBlurEvent. Not sure if this is related.

2

Well, I checked the asset repo. The hand meshes have not changed, so it must be everyone, all 3 of us.

3

The Hand-tracking API has slightly changed, this PR addresses it - initial Hand-API update by RaananW ¡ Pull Request #9855 ¡ BabylonJS/Babylon.js (github.com)

1 Like
4

I notice that this pr was merged 18 hours ago, and so was nightly. Has anyone seen it working by calling preview? I have not, but I:

  • Deleted my tab, and loaded in a new tab
  • Deleted my browsing data
  • Restarted
5

That is because nightly didn’t include this change for some reason. Next nightly, in a few hours, will.

6

Ok. Though I am not sure where it was night 18 hours ago. If this is a “cron” thing, maybe a name ‘daily 16 gmt’ would be better.

7

Not a cron thing, just a definition of some form of daily builds. It’s mostly manual :slight_smile:

And it’s always night somewhere!

8

A couple of things. I checked right after Thursday’s Nightly, and it was not changed. I just figured that it was not on the web server yet. I wanted updated local source anyway, so I did a pull, then just referenced a local version of the preview build, and kept on going.

Made some more progress today, but noticed when I left immersive mode & tried to go back, I just got black. Turns out I was getting an exception. This is the stack. I am referencing my server by IP address. You will also see eventually my hand tracking sub-class at the bottom of the stack, but I do not think this is me.

Uncaught DOMException: Failed to execute 'getJointPose' on 'XRFrame': XRSpace and XRFrame sessions do not match.
    at https://192.168.50.238/babylon.js:16:3396139
    at Array.forEach (<anonymous>)
    at e.updateFromXRFrame (https://192.168.50.238/babylon.js:16:3396083)
    at https://192.168.50.238/babylon.js:16:3403108
    at Array.forEach (<anonymous>)
    at HandTracking.t._onXRFrame (https://192.168.50.238/babylon.js:16:3403058)
    at HandTracking._onXRFrame (https://192.168.50.238/Palmer-JC.github.io/lib/XR-UIPortal.2.0.js:1809:19)
    at r.callback (https://192.168.50.238/babylon.js:16:2097536)
    at e.notifyObservers (https://192.168.50.238/babylon.js:16:37003)
    at XRSession.renderFunction (https://192.168.50.238/babylon.js:16:1341566)

Second thing is I tried to check it was not me by running a PG, but I got the error from the API change. I also quickly switched back to preview in my scene. It does not look like the preview is being updated.

Edit: up @RaananW, Posted this late Friday, not sure you have seen.

9

I will have to check that, as this should not happen. Seems like the hands are not disposed correctly, and that the XRSpace stays in cache. Haven’t experienced that yet, but it should be easy to reproduce. Want to submit a ticket on github? Please reference me and I will look into it.

10

see Exception in XR Hand tracking when leaving immersive mode & returning ¡ Issue #9882 ¡ BabylonJS/Babylon.js ¡ GitHub

1 Like
11

Almost off topic, but I have continued working on my UI, I just cannot leave. I have just started looking at the FPS, un-optimized, and it is pretty bad. My stuff I can work on, but a big drag by hands much less so.

Bootstrapping the UI in the development of itself, I wanted to see what is the best that can be done using hand tracking. There are very few meshes in the test other than the hands, the 2 arm control panels, & the 2 digit “Nexie Tube” display. Had a little trouble taking a picture, but this is what I got:

handtracking FPS
handtracking FPS1440×1440 56.3 KB

I know you have use an overlay tool. I could never get that to work. Do you see a mid 40’s print with hands in an empty scene? I can even take out the display & write 2k lines to console. Nixie tubes use a Highlight layer, which might be expensive.

12

The UI is one thing that might damage performance. The other one is the actual hand tracking implementation (in babylon). Until the last update there was no way to avoid getting the position of each joing (25 per hand) per frame. That means - 50 calls on each frame, with quite a few immutable objects that were created and cleared. And that 72 times per second. Not an ideal situation.

I still didn’t find the time to change the hand implementation, but I hope to do it very soon. This will surely help performance.

13

That is 90 times per second with my device. While think the majority of profiling should be done on desktop, this part cannot. Going to see remote profiling works, and what it might show.

14

Well I found that setting the value of the display was really way too heavy, and actually performed for as many digits displayed, 2 in this case.

Using it to be a FPS display requires it be set EVERY frame, and actually just demonstrated that problem. Went to a much lighter set function, and now in between 87 - 90 FPS.

1 Like