Babylonjs Native xr issue on iOS

Questions
, ,
1

I’m working on a project where we would like to use BabylonJS native to run our WebXR application on iOS devices.

I have the current project running and able to start the playground application. But when I try to start the experience the application crashes. Not sure if this is a bug or something else I experience?

I’m testing on a iPhone Se running iOS 17.4.1. I am able to run WebXR in a app clip from Variant https://demos.launchar.app. Not sure how they solved it, or if they are using BabylonJS Native at all or create their own interface.

xr.baseExperience.enterXRAsync("immersive-ar", "unbounded", xr.renderTarget).then((xrSessionManager) => {
   console.log("XR started");
});

I get XR started message but the screen starts to blink and I get following error:

BJS - [10:59:03]: session.enabledFeatures is not available on this device. It is possible that this feature is not supported.
XR started
interruptionHandler is called. -[FontServicesDaemonManager connection]_block_invoke
Thread Performance Checker: Thread running at User-interactive quality-of-service class waiting on a thread without a QoS class specified (base priority 31). Investigate ways to avoid priority inversions
PID: 1083, TID: 176361
Backtrace
=================================================================
3   AppleNeuralEngine                   0x00000001deff0f34 7A5A59FE-13B5-31AB-B69D-6BB75B5B7CC8 + 28468
4   AppleNeuralEngine                   0x00000001df00ea78 7A5A59FE-13B5-31AB-B69D-6BB75B5B7CC8 + 150136
5   Espresso                            0x000000019f276d20 C576584B-5370-37C8-B9F2-4D02604F22E0 + 10960160
6   Espresso                            0x000000019ed86108 _ZN8Espresso26net_compiler_segment_based9__forwardERKNSt3__110shared_ptrINS_14abstract_batchEEE + 1248
7   Espresso                            0x000000019e8affd0 _ZN8Espresso3net9__forwardERKNSt3__110shared_ptrINS_14abstract_batchEEEii + 388
8   Espresso                            0x000000019e8af14c C576584B-5370-37C8-B9F2-4D02604F22E0 + 704844
9   Espresso                            0x000000019e8ad814 C576584B-5370-37C8-B9F2-4D02604F22E0 + 698388
10  Espresso                            0x000000019e8ad5f0 espresso_plan_execute_sync + 132
11  SceneIntelligence                   0x00000002099caf18 656443F8-FAF6-3ECA-B452-2108F9B50A2A + 36632
12  SceneIntelligence                   0x00000002099d5784 656443F8-FAF6-3ECA-B452-2108F9B50A2A + 79748
13  ARKitCore                           0x00000001bf51d7ac F0271512-8F53-3E91-8AE3-A6C6AD3CBF74 + 804780
14  ARKitCore                           0x00000001bf5c75cc F0271512-8F53-3E91-8AE3-A6C6AD3CBF74 + 1500620
15  ARKitCore                           0x00000001bf5c74bc F0271512-8F53-3E91-8AE3-A6C6AD3CBF74 + 1500348
16  libdispatch.dylib                   0x0000000105054b98 _dispatch_call_block_and_release + 32
17  libdispatch.dylib                   0x00000001050567bc _dispatch_client_callout + 20
18  libdispatch.dylib                   0x000000010505e66c _dispatch_lane_serial_drain + 832
19  libdispatch.dylib                   0x000000010505f43c _dispatch_lane_invoke + 460
20  libdispatch.dylib                   0x0000000105060abc _dispatch_workloop_invoke + 2336
21  libdispatch.dylib                   0x000000010506c404 _dispatch_root_queue_drain_deferred_wlh + 328
22  libdispatch.dylib                   0x000000010506ba38 _dispatch_workloop_worker_thread + 444
23  libsystem_pthread.dylib             0x00000001eb5f8f20 _pthread_wqthread + 288
24  libsystem_pthread.dylib             0x00000001eb5f8fc0 start_wqthread + 8
2

cc @BabylonNative

3

Can you set up a repro in a GitHub repo?

4

Thanks for reaching out!
And sorry for the delay, I had some other issues I need to address too.

I just cloned the Babylon Native repository and build it for iOS. I did have some troubles. I managed to run the Native project with help from Cedric:

Build and run errors in the native playground targeting iOS

If I set ar = true and run the application on my iPhone Se I get the mention error.

6

Sorry for the delay. There are a lot of issues recently. For tracking purposes, can you file an issue on GitHub? It is likely we won’t be able to look at this one for a bit. Are you blocked by this?

7

Thanks for returning :slight_smile:

I will file an issue on GitHub.

I was able to have something running in Babylon React Native. I am able to start the xr experience and able to run the hittest, where I show a marker on the floor. Unfortunate my luck runs out after that. When I try to use the hittest to add an anchor I’m able to get the first sometimes, but after that I keep getting an error in CreateAnchor method:

Anchor System::Session::Frame::CreateAnchor(Pose pose, NativeTrackablePtr) const {
    return m_impl->sessionImpl.CreateAnchor(pose);
}

The exception:

com.facebook.react.JavaScript (17): EXC_BAD_ACCESS (code=1, address=0x0)

I will gladly do what I am able to do, getting this to work :slight_smile:

1 Like
8

I am not expecting Babylon React Native to have issues. Please make sure to mention this in the GitHub issue(s).

10

Should I file a new issue on BabylonReactNatives’s GitHub regarding the problem with CreateAnchor ?

11

@bghgary I just tried to run a simple AR Kit app and I got the same exception! I suspect this could be a issue outside Babylon!

I’ll try dig some more into it…

1 Like
12

After upgrading xcode to 15.4 I’m able to run the AR Kit sample project.

I tried to rebuild BabylonNative again. I do not get the exception anymore when entering XR, but the camera feed is not shown, and I’m not able to manipulate the cube either. There is still coming logs from the fps counter though…

I also tried to run the React Native app but still get the same exception as mention above.

13

I’ll take a closer look this week.

1 Like
14

Let me know if I can assist with anything.

15

I can repro with latest xcode/iOS on iPhone 11. I’m working on a fix now :slight_smile:

1 Like
16

problematic commit is fix: cleanup view controller by okwasniewski · Pull Request #1368 · BabylonJS/BabylonNative · GitHub

17
2 Likes
19

I just pulled the newest version of BabylonNative and was able to run the WebXR experience on my iPhone.

Great work!

3 Likes
20

Hello @dnrn, I am working in babylonjs and I need some clarification on the compatibility across IOS and android?

Firstly, BabylonJS can work in IOS platform using WebXR?
Secondly, If yes, Did you build two projects for IOS and android separately?

@Cedric and others, Please don’t mind to throw in your suggestions also.

Thanks