Awesome to hear about people trying this stuff! We have a project called BabylonAR (blog post about it here) that’s beginning to tackle issues like these. However, the project is very young (alpha), so there are still a lot of features we have yet to add to it.
Can you tell me a little bit more about your use case? Depending on the constraints on your video – for example, if you can include markers, at least in the short term – it might be possible to do what you’re hoping for with just the current capabilities. If what you need is more general case, that may veer into an extremely difficult problem called SLAM, which BabylonAR doesn’t have the capability to solve. (Not yet, anyway. )
So, in short, there may be several possibilities both inside and outside Babylon, depending on what exactly you’re trying to do. If you can share a bit more about the use case you have in mind, I’ll be happy to discuss what options might work for you. Thanks!