Multiple client world synchronization in Web AR

This is a high level discussion as I am having trouble deciding what the best way to approach this would be.

I am investigating options that would allow multiple users to “share” a Babylon AR experience - so one user positions a virtual objects in the real world and everyone sees it in the exact same spot in the real-world. This could be inside or outside, but inside buildings (no GPS) is first up. Eventually, my application will need both. Target devices are mobile phones. The placed objects would be persistent across sessions. This will probably need multiple method for rough position ID and fine grain position correction. But, I don’t know, which is why I am dumping a few options I am thinking about, and asking for any / all thoughts.

One option is that an april tag, qr code, unique spots on a wall, or even random found object positioned in the real-world can be used as a calibration object each time a user starts a session. This seems like it should work, I can use a QR code to locate a user. I can also use that same code to correct the users position compared to others, by measuring the code’s skew, rotation and that of the user. This is the option I am leaning toward, as it gives me both a location and orientation. But, it could be off putting / annoying for users to need to calibrate against an object every time they launch a session.

I then got to thinking about use WIFI / Cell strength to rough position / orient as then you would not need a calibration object. So, triangulating a position by ambient WIFI. But, I do worry users won’t go for allowing a website to access signal strengths - if possible - as it sounds like a security issue. Still, there seems like potential here.

Then I got to thinking about Babylon’s AR plane detection. That maybe comparing AR plane sizes and positions to locate a client in the real-world could work. So, use plane sizes and positions in relation to each other as a measure of where a user is and how they are oriented in the real-world. But, in looking into it, plane detection’s accuracy seems wildly variable even between two sessions on the same device, say noting of two clients with different phones / cameras.

That led me to think about maybe ignoring Babylon’s internal tools, and to integrate a javascript CV library to do some sort of edge detection (track edge size / orientation) to triangulate. This is what I am currently leaning toward, but it seems like a lot of work and given how battery intensive Babylon is for phones when in AR mode, its a bit of a concern adding on top another battery intensive operation. I also still would have the issue of different cameras on different devices measuring slightly differently.

So, none of these options are great, and I feel like there must be a simpler way to approach this than continuing down the rabbit hole I am heading down.

Thoughts on things would be appreciated.

Hmmm that’s an interesting discussion :slight_smile:

That maybe comparing AR plane sizes and positions to locate a client in the real-world could work

When you say “locate” in this context, you mean just to have a rough idea of the space surrounding the user, or to get precise lat lon coordinates? Because I can’t see how this would give the second option, but maybe I’m missing something :thinking:

But, I do worry users won’t go for allowing a website to access signal strengths - if possible - as it sounds like a security issue. Still, there seems like potential here.

I do agree that most users would think giving signal strength perm on a website would be overkill, but maybe they would feel more inclined to in an app? So you could package your experience as a webview app and get this “trust”, for lack of a better word.