Agent Based World Creation and Destruction + Human Swarm Interaction

Hi y’all,

First off, I just want to say: <3 . Being part of this community has been really awesome, and I’m really excited about the community and the tech is heading.

I’ve posted about my VR project on the questions side of the website, but I also wanted to post it on this side too.

I’m making a VR game (that has some AI components) to explore all sorts of artificial life concepts.
(1.) Here is an example of world creation/destruction with architect agents:
(2.) Here is an example of looking through the eyes of the swarm each player is assigned when they log into the VR world:

I’m currently working with some friends trying to put together a 3D gan for shape generation and some reinforcement learning for weird agent behaviors (it’s actually a javascript reinforcement learning library, please check it out: GitHub - polyzer/rllib.js: Reinforcement learning library with JavaScript.). Here is an example of a project that uses 3D gans (not mine): .



Rlib.js is really interesting!
Is it difficult to use it with Babylon.js instead of three.js?
(or maybe there are some examples already?)

I am working on create the first babylon example, which also happens to be a vr example.

Any interest in contributing. :stuck_out_tongue:

It’s pretty neat, right?

Have you gotten anywhere with this? I am currently using Babylon to create a playground to train RL agents… no link for babylon quite yet but you can see a canvas-based version of RL.js learning to play Snake here - - well - it’s sort-of snake - in my version though, like in real-life, the food runs away :slight_smile:

I’ll post updates on my work here.


This post was from a while ago.

I backed up.

I built a human-swarm interaction + swarm creating architecture webxr world: 2020+ VR Worlds — △Ark

I also started building a… currently unannounced… library for artificial life/agents in webxr: 2020+ VR Worlds — △Ark

In the future (how far, idk), I’m definitely going to add three things to this library: 1.) agent-based data visualization, 2.) genetics for alife and 3.) learning for agents/alife (reinforcement learning)

I would love to see any work you produce and/or are thinking about along this trajectory; I’m obsessed with this stuff…

1 Like

GitHub - polyzer/rllib.js: Reinforcement learning library with JavaScript. maybe this library will be helpful?

1 Like

Links - Awesome! Thank you. I will take a look as soon as I get off work. I haven’t used the tensorflow-based RL lib yet, I’ve been using this one - it’s much smaller (as in - no dependencies) and works great:

I am obsessed as well - anything to do with artificial life and emergent behavior, both in the fundamental (geometric) domain of things as well as biology. The reason I’m learning Babylon JS right now is specifically to create environments for agents I want to train. On the serious tip, we need easy-to-set up environments for training agents, and on the more whimsical side - I have this recurring vision for a Battlebots-type training environment slash competition that just seems awesome in my head and won’t go away.

1 Like

Did you see this?

Generally capable agents emerge from open-ended play | DeepMind Seriously interesting stuff.

In my webxr agents library, I basically have a boids example and a steering agents (on the ground example). It’s the basics or almost “the shell” that people can shove their “brains” into.

I also have a bunch of server-side, graphics and interactions goodies in the library; take a look at it.

I know that library; it’s a little old, but it’s karpathy, lol. Do you know if hardmaru has some more contemporary javascript examples?

Totally hear you on the battlebots. !

1 Like

Right? I would watch that over anything on TV any day. I started working on a library of components a while back that allow me to easily assemble interactive environments for agent training - currently I’m working on creating my own version of the Amazon racer challenge. I have the vehicle more or less done for my needs and today I get the racetrack placed and the vehicle instrumentation put in place. I don’t have a public host yet but you can see my work at GitHub - sschepis/wheelbots: babylon JS playground for autonomous agents - you’ll need to

npm i -g zero

to see it run and the vehicle is in vehicle.ts if you wanna look at the source

Do you have some kind of rapid-visual demo and/or youtube video I could check out?

For my next piece I was thinking, oh, maybe I’ll build a beautiful agent-based data visualization, but now lol I think I just want to go up against a reinforcement-learning tank in a VR environment (like the last scene in ghost in the shell), haha; so, maybe our interests are colliding.