I’m Alexa Carpentier, I was recently featured on Babylon Chronicles Ep. 19, I wanted to share a few examples of how we at CodeBaby use Babylon.js in our work!
We’re a live-rendering animation company that combines AI with real-time avatars to enable interactive conversations. Here’s one of our concierge demos in action: https://concierge.codebaby.com/concierge/hotel/
Our avatars have also been featured on Big Brother Season 26 (as mentioned in the podcast). This was a great showcase of how powerful live rendering with Babylon.js can be in entertainment by eliminating long wait times for pre-rendered animations! We created the characters Ainsley and Jankie (check out a clip of Jankie and ‘evil’ Ainsley here: https://www.youtube.com/watch?v=dqyXINxJA0o)
If you’d like to create your own avatar, you can sign up at portal.codebaby.com for a free 14-day trial! Explore characters from our gallery, customize voices, build unique prompts, and more!
I’m happy to answer any questions you might have about CodeBaby, Babylon.js, or avatar creation, just drop them here!
To answer your first question, we do offer a variety of different LLM’s available, the default is ChatGPT but you can change this at any time in the conversation tab of our portal under advanced settings!
To answer your second question, our animations are hard coded we believe in order to give our avatars a human touch its important for us to use animations created by humans! However we have a library of 30+ animations to choose from so you can fully customize your avatar experience and make them as stoic or as active as you would like!
This demo is amazing! Is this using exclusively morph targets or is it using bones as well?
My thought was not about putting humans out of the loop of course ! The same way VideoGame industry is going toward deepbased character animation trained on animation datasets, you would have a dataset of “speaking while moving”, and after training and generalizing, a Neural Network would take control on the mesh with the LLM output as its only reference !
It’s using both ! morphtargets for face deform, and bones for body, neck, head, eyeballs
Ah I see what you’re saying now, its so amazing what improvements the game industry has been able to make combining animation with AI to make things even more immersive! We are currently working out some bugs on giving our avatars and their LLMs the ability to select appropriate animations to play from their libraries! Here’s a quick demo playing Simon Says with an avatar;
But this is so far the closest we have gotten to anything similar to deepbased character animation. That said, we are very interested in how we can keep moving the direction the games industry is going and keep improving users experiences with our avatars!
Amazing!! I noticed on the thread that you are waiting for more popularity before releasing the full demo, and I just want to add my interest to the list! I would love the chance to explore this further
