Using WebXRImageTracking in a bundled React Native app

Hi!

I’m trying to use WebXRImageTracking in a bundled React Native app and running into an issue with getting an image source.
I’m using Image.resolveAssetSource to transform a number representation of an image to the uri needed for WebXRImageTracking. This is working fine in an unbundled app, I’m getting something like http://10.0.70.18:8083/assets/img/etcetcetc. However, in a bundled app, this same code is getting a img_pathtotheimageetcetc instead, which the WebXRImageTracking doesn’t seem to want.

It’s throwing a Error loading images for tracking, WebXRImageTracking disabled for this session.

I’m not getting a specific error, but I changed some stuff and got this more specific error: Error loading image with exception: Error: Image with provided source returned empty response.

Any idea what I can do to make this work?

Seems like the error is coming from here: BabylonNative/Image.cpp at 08cec4cacf4d187eb839b6318a7414860f8fde77 · BabylonJS/BabylonNative · GitHub

So it seems that it’s trying to parse the image in there, but it’s empty. Hardcoding a url does work in a bundled app, but how do I provide a local image (I do see it in the android project, so it is there) so that it doesn’t get seen as empty?

Maybe take a look at this issue which has information about how to load any asset from the bundle.

Hi! Thanks for the answer. I’ve looked through the link you sent, and some other threads on this topic. I’m not entirely sure what the exact solution is though? :sweat_smile:

I have a location of the bundled image, and when I put this in a normal image component it renders it. Is there another way to get the image uri in such a way that the image tracking will accept it? From what I’ve read a common solution is to convert the image to base64? But I don’t think that would work for this feature.

A require was also suggested, but that also doesn’t seem to get accepted by the image tracking as a src.

I’ve also tried adding the prefix file://, but that also gave an error.

Unfortunately, I don’t know since I haven’t investigated this myself. @Cedric/@ryantrem maybe can help.

I guess this happens on Android? Did you try it with iOS? It looks like the same issue with resources loading on Android in release build.

Hi! I just checked, and I don’t seem to have the same issue with iOS. Do you maybe know of a way to make this work on Android as well? Any way to convert the bundled image so that it is accepted by the image tracking?

You may get some tricks here : importing a .glb file fails if only local path is provided and not served through HTTP · Issue #165 · BabylonJS/BabylonReactNative · GitHub
For some projects, we used base64 encoded images. Not great but if you only have a few, it’s fine.

Hi! I saw that, but I wasn’t sure if base64 would work as src for the ImageTracking, I thought it had to be a url specifically. Did I misunderstand that?

No, you are right! base64 will not work with that. I’m adding an issue for that.

Thanks so much :smiley: if there’s any other way to do this I’d also be very open to that.

1 Like

Have you tried expo-asset as mentioned in the linked issue? It seems like other folks were having success using it for resolving uris for embedded/bundled assets.

1 Like

Hi! Yeah I saw that as well, but unfortunately we don’t have expo in the project so using expo-asset isn’t an option. I’ll take a look if there’s another way to transform the link.
What does expo-asset output that babylon does understand?

I haven’t tried it myself, but I believe expo-asset can be used as a standalone npm package in an existing react-native app: expo-asset - npm

Also, my understanding is that it correctly produces a valid local file path in all cases (debug/release for both iOS/Android).

I would suggest trying to experiment with it a bit and then share findings here.

Ah thanks for the tip, I’ll give that a try then!