Hello everyone,
I have experience working with bones and animations in programs like Blender, but I’m wondering if there is a more visual way to do this in Babylon. From the examples, its seems this can be accomplished by writing code directly, but being new to programming, I’m struggling to visualize where the bones are, what objects they’re connect to, and how to animate them. Does the desktop editor do this maybe?
Thanks in advance!
-Dustan
Oh you’ll love this!
Pop open this playground: https://www.babylonjs-playground.com/#BCU1XR#0
Click the “Play Run” button.
Open the inspector (located in the gear menu of the playground).
Select the skeleton in the selection list
Then in the properties window you can turn on the “Debug Mode” property.
And presto! you get to visualize your skeletons without writing a single piece of code!
P.S. - Blender rocks!
Incredible, perfect! That helped me get past my first big hurdle.
Blender does rock 
I started this whole process by making game characters with Blender, but I need to figure out the code for how they move first. The game I’m trying to make is more of an evolution simulation for my students (I’m a bio teacher). Here is the ambitious first goal:
-Virtual creatures spawn with random elements (limbs, body segments, textures, etc.). I suppose this is called procedural generation?
-These elements are all connected, but can move independently. For instance, limbs can pull the body segments forward and vice versa.
-Have all of this work in a physics environment.
-The creatures navigate the world independently with no animations added (just movement parameters and what the physics engine will allow).
I definitely have a lot to learn before I get this to work the way I envision, but it’s fun figuring it out.
Thanks again @PirateJC!
-Dustan
Lots of folks around here that LOVE procedural generation! @syntheticmagus is the king of this stuff!
This sounds like a lot of fun!
Not exactly related to biology per say, but on the topic of procedural generation and textures…I just put together this demo and video last week on this very subject. Check it out!
This is an entirely fun world! Can’t wait to see what you create!
Awesome, I will definitely check that out.
Thanks!
Hi Dustan,
Sounds like a really cool idea! Procedural generation is a lot of fun. 
In terms of animation and locomotion, however, I just want to forewarn you that this might not be a very easy problem to tackle. Are you envisioning something that’s more or less (for lack of a better word) faked such that animation and locomotion are solved independently and work together only nominally under pretty constrained circumstances, or are you envisioning a system where the actual physics of locomotion are solved out and articulated through animation? (In other words, are you going for illusion or simulation?) If the former, the creators of Spore wrote a SIGGRAPH paper about how they did something similar back in 2008. It’s a bit dense, but it might be good starting point for your exploration of how to make arbitrary skeletons appear like they’re walking.
However, if you’re going for something more like the second thing I described (i.e. simulation, not illusion), while I definitely don’t want to discourage you from jumping into it, just go in knowing that it’s an extremely difficult open and active research problem at the crossed bleeding edges of machine learning and robotics. I’m not an expert in either of those fields, but I speculate that one of the reasons this is so difficult is because simulation approaches have to deal directly with a lot of problems illusory solutions get to hand-wave: center of mass, inertia, stumbling, getting up after a fall, jumping, slippery surfaces… As linked above, the Bullet Physics open-source community is the most accessible nexus of this kind of research that I can think of, so that might be a place to start if you want to go down that road. However, even if simulation is your ultimate objective, I’d probably recommend starting with illusion and the Spore research paper. Unless you’re already familiar with the topics in that paper, you’ll probably need to learn about them at some point in this workstream anyway, so it might be an easier (and more fun 
) place to start. Best of luck, and keep us posted!
Hello @syntheticmagus!
First let me say that I really appreciate your feedback and the information. I mentioned to @PirateJC that I’ve been incredibly impressed by this community and the whole Babylon.js experience. Everything seems designed to encourage people to learn and create 
I’ve had a lot of fun doing just that the past couple of weeks, and the support from the forum members has been a big part of that!
I’ll try to answer your questions to see if what I’m trying is a little too ambitious for my current skill set. You definitely seem like the expert on this topic!
-With respect to inspiration, I’ve been completely fixated over the years on procedural generation, neural networks, and machine learning. These programming advancements are so exciting because they simulate what actually happens in nature–natural selection, and for some species, learning from past experiences. Countless variations selected for or against based on the parameters of the environment (or the programmer’s set goals). Just amazing stuff! Years ago a programmer named Jeffrey Ventrella created a simulation called Gene Pool (here). These “swimbots” spawn with random traits (body segments, segment movement, etc.), and these traits determine their ability to move through the environment. These traits determine whether or not they can successfully find food and mates. If they do mate, they pass down information about their character’s traits to their offspring. The swimbots are completely self-animated and move very realistically. Over time (you can let it run through a browser for hours, days, weeks), the swimbots literally evolve into new characters without any user input. I’ve never seen a more realistic depiction of natural selection in a game or simulation. I mentioned before that I’m a biology teacher (and an avid gamer!), so that’s why I’m so interested in creating something like this, but with my own artistic and game play variations.
-I thought it would be fun to use 3d models and a physics engine for my digital creatures. Obviously, babylon.js supports those two things, but here’s what I don’t know yet:
Can I create a creature made of multiple meshes (body, limbs, head, etc.), have those meshes use the same skeleton, and have those body segments each be able to spawn a random mesh? This is basically what Gene Pool does, but with 2D objects and no physics. I could be completely wrong, but this seems like the biggest technical hurdle to the whole concept.
For instance, when a creature spawns:
*A default skeleton and physics impostors are loaded.
*The head, limbs, and body are randomly assigned from a set of premade meshes.
*The world will be populated with hundreds of these self-animated creatures, which will then move
around and attempt to achieve their objectives with no user input. I know there will be some
seriously complicated math to make them move realistically!
I’ve eliminated the potential challenge of trying to have the creatures have one continuous mesh (like in Spore or No Man’s Sky). My characters will have different segments that overlap, but will not actually be part of the same mesh. This won’t look as good as having continuous meshes, but I didn’t want to add another tremendous challenge to the mix.
So, this is what I envision, but I’m realistic about my current programming limitations. I certainly don’t want to spend months trying to program something that even the experts have trouble creating!
Any input or advice you have would be most appreciated!
Thanks again,
-Dustan
Hi Dustan,
Sounds like you have a lot of really awesome enthusiasm! Like I said above, I definitely don’t want to discourage you from jumping in, and I wouldn’t recommend worrying too much about whether or not something is beyond your current skill set. Tackling things like that is a great way to build new skillsets. However, if you’re enthused about a project like this, I think it’s best to go in knowing (which it sounds like you already do) that it’s not going to be a short or simple journey.
In the details you outlined, I see two mitigating factors that might make your particular project a little easier. Firstly, the fact that you’re interested in an evolutionary simulation rather than in a direct application means that your creatures don’t actually have to move well; as long as they move at all such that they can compete with each other, that’s enough for success, which lowers the bar. Secondly, you mentioned having a default skeleton; if the parameters on that skeleton are fairly uniform, that can dramatically reduce the complexity of the problem (though it also reduces the variability of your simulation) by constraining the number of ways it’s possible/sensible to move. In sum, what it looks like you’re describing is kind of a 3D visualization of an old-school genetic algorithm, which is a really cool idea!
My main recommendation would just be to break down the challenge into components and tackle them one at a time. For example, you might consider foregoing the physics-based aspect of the idea to start with, instead having your creatures’s movements dictated by randomly-assigned parameters (walk speed/cost, tendency to wander, etc.), switching to a physics-based approach later in development so you don’t have as many unknowns to deal with at the same time. You might even consider trying to stand up the simulation by itself in a purely text-based environment, with no graphics at all, because that should be a comparatively quick process that might help to guide the other steps that follow.
Other than that—being aware that this will be a lengthy and complicated project, and doing what you can to mitigate the complexity you have to deal with at once—I’d say you’re in a pretty good position to start on a very exciting project. Best of luck, Dustan!
Great suggestions, thank you. I’ll keep everything simple for now while I’m learning the basics, and worry about more advanced features later. Just getting simple things to work is enough of a challenge for now!
Thanks again
