Babylonjs + project malmo

Off topic
1

Hi There,

So, I’ve been thinking a lot about the unity ml agents package. I’m playing around with it. And, my main conclusion, is that it would be super cool to have an agents package for babylon. At first, I was looking at tensorflowjs, but tensorflow agents is not written in javascript. I guess you could port it…? Then, I was looking at project malmo and the project that grew out of it http://minerl.io/ (agents in minecraft). I’m beginning to explore this, with the idea of trying to figure out its compatibility with babylon. I know that there was a ‘minecraft classic’ edition of minecraft written entirely in babylonjs, so I think I’d start with trying to figure out how to run an agent in that world. Is anyone else interested in this? (is this a crazy thing to be interested in)?

3 Likes
2

apparently, you can do deep reinforcement learning in the browser with tfagents
https://livebook.manning.com/book/deep-learning-with-javascript/welcome/v-6/

3

maybe that’s a better approach to getting agents into babylon

4

I’m now talking to the tfagents team and might create a discord for “deep reinforcement learning in JS” bc they say that there is some interest. I will keep plugging babylonjs as the visualization engine of choice.

1 Like
5

Just in case, have you seen this doc:

https://doc.babylonjs.com/extensions/crowd

6

Yes, it is dope. I want to extend it and make them neuroagents that can learn!