Hi friends,
I have used babylon.js in a couple of projects with great success, love it!
now I have to make a web based project that should do AR with image tracking rather than marker tracking
I tried to use ar.js (GitHub - AR-js-org/AR.js: Image tracking, Location Based AR, Marker tracking. All on the Web.) as I had used them years ago for marker based AR, but their NFT image tracking AR is super buggy and unstable, unusable really
so the question is, is such thing possible with babylon.js?
to build a web app in which i point the camera of mobile phone to a natural image, and that triggers the appearance of a 3d mesh that appears on top of the image and stays in the same place always, in a stable way, as I move my mobile camera around?
thank you for any tips 
@RaananW
Of course. Babylon has an image tracking feature, based on the webcr marker tracking capability. It’s not yet officially available in all browsers, so it will require seeing the WebXR incubation flag in your chrome://flags page. Here is a demo. Notice you will need the image found in the url to get it to work, but you can provide any other image(s) you want:
4 Likes
thank you very much @RaananW , oh my god this works SO incredible WELL, amazing!
it works 10000 times better than ar.js !!!
So now questions:
-
I had to yes activate the experimental flag about webxr incubations, any forecast as to when this will become officially standard in chrome so that we dont have to activate the flag? obviously it is not usable as long as people have to activate the flag
-
and I guess the only other question is about Safari and iOS, obviously this will only work in Chrome, and nobody knows when iOS will support this, but anyway what is your personal opinion, do you think they may support it next year?
thank you very much again, the demo works amazing!!!
Both are very good questions, to which I have, sadly, no answer…
The marker tracking is stable and very much needed, so I can only assume chrome will very soon make it a stable(r) feature of the browser. I am personally waiting for this as well, because until then we can’t really push towards the usage of this feature.
Now about apple… my favorite tech company… They already have a webxr flag in safari. It doesn’t work at all, but the flag to enable webxr is there. which is a very positive hint from them. I assume we will see webxr in iOS when apple release their own XR headset (rumored to be in the works). When that happens, we will have wider adoption of webxr and we can finally start using webxr cross-os as wll.
1 Like
thank you @RaananW ,
Im crossing all my fingers and beyond so that at least we get the webxr stable by default in Chrome very soon, and later hopefully on iOS hopefully in some months crossing fingers thank you
I’ve played around with the image target in this project before and it works pretty well:
Have not used it in production just yet though.
4 Likes
@Gerard_Baldsing thank you very much for sharing this! just tested it and it works! some questions if I may:
- is this cross platform? does it work also on iOS? I tested the example on my android pixel 4a
- so this doesnt need to activate any flag on chrome? so it is not using webxr? is it then using similar stuff to ar.js?
it is not as stable as the webxr examples I tested, if I stay still, the triggered 3d stuff kind of wobbles around, but it is still much much better than ar.js (which is a disaster for image tracking)
1 Like
@javismiles no problems. Been waiting for iOS to finally implement WebXR Device API like yourself so have had to come up with some interim solutions.
-
I’ve tested on iOS and works pretty well in Safari. It didnt work on iOS Chrome because it uses WebRTC to deal with the video. I’ve got an iPhone 7 to test on and performance was OK. My samsung S21 handles it a lot better - but I assume thats due to power / being a new phone as opposed to the OS
-
Doesnt need any extra chrome flags (i dont think) and works similar to ar.js and GitHub - webarkit/ARnft: A small javascript library for WebAR with NFT
What i’m unclear about is if this is free to use or if there is a pricing model at some stage.
I know their software ZapWorks Pricing: Choose The Right Plan For You has a subscripton fee so not sure if this universal AR project is free for commercial use.
1 Like
all great points @Gerard_Baldsing ,
for me as long as it works on chrome android and safari ios that would be more than enough so that part sounds good
yes, it is interesting that it is the same system than ar.js and yet ar.js issues area is full of people complaning about how bad image tracking works on ar.js
licence info, yes, I see no licence info at all there, well, that is sort of important to find out, is it open source for commercial use, etc
@Gerard_Baldsing I contacted the creators of zappar-xr, and basically its not good news, the code is not open source and it cannot be used commercially without paying for their subscription licence etc with the rest of their conditions which make you be tied to their platform continuously,
so im afraid that this is definitely not a viable solution in my view
I really hope for webxr to become an open stable standard in 2023
Shame it’s not open source.
In terms of pricing, I’ll be keen to how it compares to 8th wall now:
For a commercial project that’s not too bad depending on your client.
In response to your comment:
“I really hope for webxr to become an open stable standard in 2023”
You sound like me in 2021!
Would love to see some of your work.
1 Like
@Gerard_Baldsing In 2017 I did some wearable AR projects using ar.js and geometric markers that worked really well, what amazes me is that 5 years later (which is such a long time in tech time), we still don´t have a decent open source image tracking web based solution (ar.js offers image tracking now but it works sooooo poorly), that is truly astonishing to me
I would like to understand why the work on these standards, like webxr proceeds so slowly, I’m sure the answer is that there is no other way, that this stuff has to be done meticulously and carefully and methodically, etc, etc and requires work on the designers of the standard plus the browser companies etc
I get it, but compare that to the evolution of generative AI these days, night and day; you have thousands of people (I mean talented developers and researchers) pushing generative AI forwards, and it kind of seems like there are 15 people pushing web AR tech forwards or something like that?
1 Like
Hi @javismiles, some time ago I had to develop some AR experiences for the web and instead of using ar.js I used mindAR.js it’s a good image tracking solution using JavaScript until we have webXR implemented in most browsers.
2 Likes
@Gustavo_Font thank you very much, but:
- is it open source like ar.js?
- is it better than ar.js? because ar.js is great for geometric markers but a disaster for NFT image tracking, so unstable
thank you
Hello, yes it is open source just like ar.js. I leave you the link of the project MindAR | mind-ar-js
The difference with ar.ja is that mindAR allows tracking of images not only of geometric markers. It is very interesting, it deserves at least one test.
On the other hand, it works with aframe for the 3D part just like ar.js but there is the possibility of using the api with other 3D engines like three.js or even babylon.
3 Likes
thank you @Gustavo_Font , just to clarify ar.js also allows tracking of images, the problem is that it works very poorly, it is very unstable, hopefully mindAR will be better!
@Gustavo_Font sadly I just tested their basic example at:
and its just not good enough. Like ar.js, it jumps around, dissappears, responds very bad to the angles, etc, etc webXR is 1000 times better,
not usable in my view, a real pity, but thank you for suggesting it, it was worth a try,
im afraid that both ar.js and mindAR.js are just not usable for serious projects in regards to image tracking
webXR however gives really fantastic stability and angle response, speed up dear browser developers!
Hello javismiles, what you say is totally true but webXR is not yet supported in most browsers. Hopefully it will be available soon.
On the other hand, the experience in mindAR and also in AR.js improves a bit considering the quality of the images that are used as a trigger. I copy some links to an article where they explain the things to take into account and also has a tool that allows you to measure how trackable an image is. I hope it helps you to improve even a little until webXR is available.
1 Like
@Gustavo_Font thank you Gustavo, I wish it was as simple as choosing the right image, well I will check your links, thank you
1 Like