I’m trying to animate the loaded from the .glb file model by changing data.
The main idea - I want to animate visemes that I’ll receive from AWS Transcribe. After some actions, var animData will be changed and I need to start animations a soon as it’s updated - kind of useEffect hook.
If I have some kind of getAnimationGroupByName(“someAnimation”) so how I can play() with changing data trigger? Can I use some options of the ActionManager for that?
Here is my code example:
import React from 'react'
import { Engine, Scene } from 'react-babylonjs'
import { Vector3, Color3, ArcRotateCamera, HemisphericLight, AssetsManager } from '@babylonjs/core';
import '@babylonjs/loaders';
const WithModel = () => {
let animData = {"time":6,"type":"viseme","value":"k"}; // that data will be updated and it's should trigger the animation
const onSceneMount = (e) => {
const { canvas, scene } = e
const camera = new ArcRotateCamera("Camera", Math.PI / 2, Math.PI / 4, 10, new Vector3(0, 6, 0));
camera.setPosition(new Vector3(0, 7, -4));
camera.setTarget(new Vector3(0, 6.5, 0));
scene.clearColor = new Color3(0.3,0.3,0.3)
camera.speed = 0.01;
camera.wheelPrecision = 70;
camera.minZ = 0.001;
scene.activeCameras.push(camera);
camera.attachControl(canvas, true);
var light = new HemisphericLight("light1", new Vector3(0, 1, 0), scene);
light.intensity = 0.7;
var assetsManager = new AssetsManager(scene);
var meshTask = assetsManager.addMeshTask("tory task", "", "./assets/", "Tory_idle.glb"); // loaded model
meshTask.onSuccess = function(task) {
console.log("loaded Tory");
task.loadedMeshes[0].position = Vector3.Zero();
task.loadedMeshes[0].scaling.copyFromFloats(5, 5, 5);
}
assetsManager.onFinish = () => {
scene.getEngine().runRenderLoop(function() {
scene.getAnimationGroupByName("idle").play(); // standing animation in a loop
scene.render();
});
};
assetsManager.load();
}
return (
<Engine antialias={true} adaptToDeviceRatio={true} canvasId="renderCanvas">
<Scene onSceneMount={(e) => onSceneMount(e)} />
</Engine>
)
}
export default WithModel
Welcome to the forum, you could use a codeAction in actionManager to play the anim as you wish but the main issue is you would not have a trigger for the data received from AWS.
In your case I would simply as soon as the model load store all the animation groups you need in variables.
And then play pause stop at will when you receive your AWS notification ?
The idea is the animation of the visemes from the speech marks file from AWS TTS (Polly). Each time will be received new array with timecodes and visemes value, for example:
Later I’ll need to sync it with an audio file that also will be received from AWS Polly, but this is not the main target for now. Each new data array from AWS should be animated, but ExecuteCodeAction is not accepting updated data as a trigger…
The point of stored animation groups is an excellent idea - that’s works just fine for me if I need to animate separate viseme with GUI buttons - check the picture. The trigger there - are buttons with .start() function.
And the main thing is - AnimationGroups for visemes will be the same and can be stored in vars, but the order of visemes in received AWS data each time will be different.
The component with BabylonJS inside React will not have UI with inputs or buttons (that’s an important point) - other components will handle and process data to feed Babylon.
A soon as React will receive data from AWS, it’s should put it to BabylonJS component for animations group start() - there are no manual calls from BabylonJS such as keyboard input or mouse click.
I’m looking for the solution to start BabylonJS animation from the trigger outside - data update or a separate function.
You could use a command pattern for this or an observable that you would pass to the babylon component through props, you do not need a special Babylon Trigger for this.
Adding @brianzinn who uses Babylon with React a lot
I think the original idea @JoeDev77 has with useEffect sounds right. You can combine that with observers and triggers. Have a look at this hook, which subscribes to Observer<T> events (of a state machine) and then the external updates are handled by state (or you can change animations, etc.). xmachina-semaforo/useMachina.ts at main · brianzinn/xmachina-semaforo (github.com)
You can see the full scene animation here - the lights are changed only from events occuring outside of React, but observed by React through that hook above via state: xmachina semáforo (brianzinn.github.io)
Awesome example - exactly what I am looking for!
I’ll try to play with different hooks similar to your example to check how it works in my situation. I share something here a soon as I’ll get some results.
Thanks, @brianzinn, and @sebavan
[by the way, it’s good to have something like xmachina-semaforo example in Babylonjs docs, cause it is hard to find anything about external hooks/triggers]
Thanks Joe. Welcome to the community. If I should add as well (if that example is close to your real application) that you can get away with just babylonjs-hook. The react-babylonjs is a more powerful library that also takes care of declarative rendering of babylonjs objects as host elements and other goodies. babylonjs-hook is a small library, but still has convenience hooks like useScene, so you can compose your scene without prop drilling to get access to babylon: babylonjs-hook - npm (npmjs.com)
but may I ask about the main reasons for the switch so that we could improve our engine as well ?