Lipsync Blink and Animations(Dynamic) based on words

Hello, Can someone help me in achieving lipsync and dynamic animation based on text, i tried but the lipsync is automatically getting triggered, i.e not speaking based on words.
Here is the bot you can do it either in blender or babylon.js.

you can try any of the bot in both repository.

I recommend taking a look at previous posts in the forum about lipsyncing Using observables for lip syncing - Questions - Babylon.js (babylonjs.com)

@carolhmj i’ve already gone through the article i guess that won’t help me in achieving dynamic lipsync based on text, that’s why i posted an issue here
Thank you

What you’re asking for is a pretty complex problem, so there probably won’t be easy simple answers for that. Facial animations are usually based on Morph Targets Morph Targets | Babylon.js Documentation (babylonjs.com), but you’ll have to prepare the mouth positions and then mix them based on what letter the character is reading now.

1 Like

@carolhmj I’ve prepared the mouth positions and mixed them with the letter too, but when you load the character in babylon sandbox the bot lips are automatically moving without playing, by default when you load the 3d character to sandbox it’s moving her lips

I believe that’s because first thing you have to stop the animationGroup.
And then a PG would certainly help (if by any means possible?).
Thanks and have a great day,

@mawa I didn’t get what is this PG?, And before exporting the bot from blender, i’ve stopped all the animations, but still when you load it in babylon it starts moving her lips

In BJS, imported animations are referenced in animationGroups. I’m not a specialist of that (I never do imported animations) but it seems like without the instruction to stop it, the animation runs by default.
You can check that with the ‘Inspector’.

A PG is a ‘playground’. One of the key features of Babylon.js and probably the best tool for debugging and sharing code and content with the team and the community.
Paste your code (extract) and load your model in a playground such as described here
Best way to get quick and accurate answers to your questions.

1 Like

The link that is included in all questions template: Make the Best Use of the Forum | Babylon.js Documentation (babylonjs.com) explains what the Playground is and gives general tips on how to productively ask questions :slight_smile:

@carolhmj I know about playground, my issue is lipsync based on words which , i’m not able to achieve it, i’ve tried but i failed, Thats why i asked for on this platform

Explaining your issue with the Playground greatly helps everyone on the forum understand what’s going on and help you.