I’ve recently been thinking more about serializing scenes created with Babylon for reuse in other rendering contexts, however I used interactivity quite heavily mainly through the action system. It seems like the best way to serialize interactivity of scenes is through the gltf interactive spec, which if understand correctly is implemented in Babylon as the flow graph system.
I am wondering if people have any thoughts on how feasible it would be to automatically convert interactions implemented with the action system into a flow graph?
So far we have been working on enabling Babylon.js to import glTF model that use KHR_interactivity extension. During import time that will be converted to flowgraph when generating the Babylon Scene.
We could modify the Babylon exporter to convert the flowgraph used in the scene into a KHR_interactivity but we don’t have a roadmap for that yet.
If other engines allow both to import models that have KHR_interactivity and convert it to their own action system as well as export their action system to KHR_interactivity when exporting GLB models I think this could be a really at some point. However, this would probably only be a subset of what is possible with the engines natives action system, since KHR_interactivity would be like a common denominator.
There could possible be some overlap here with how it was done with animations, which are serialized when scenes are exported. Though, I understand these are different concepts. Also yeah I imagine the scope of interactions that could be serialized would be limited compared to what is possible in engine, but even having basic stuff like property change/animation on click, hover, etc would be enough for many use cases.
I think we would need a couple of things to be able to serialize the action system to glTF.
A way to convert from action manager to flow graph (as said in the OP).
A distillation of flow graph to the subset of features supported by glTF interactivity.
I think 1 should be doable as flow graph is much more capable than what the action system can do for the most part, but we will have to try it to be sure.
I would expect 2 should be doable with the features converted from the action manager but is a far more complex topic if we wanted to support everything that flow graph will be able to do as flow graph will be a superset of glTF interactivity.