The hitchhiker's guide (I mean idiot's guide) to reinforcement learning (dumb coding problem)

Questions
1

I am working on a VR reinforcement learning environment for a friend of mine, and I am having an embarassing coding 101 problem (I think) that I don’t understand.

When I log into the console and put in agent.moveNorth(), my agent moves properly. But in my update loop in my code when I run agent.moveNorth() nothing happens; and, I am honestly just having a wtf moment.

The code is here: Glitch :･ﾟ✧ . The agent code is on lines 47-169, but the move methods are on lines 64-81. I try to activate them in the draw loop on line 413.

He has implemented reinforcement learning (PPO) in javascript. I’m tasked with building the environments. We are trying to put this together to build “mlagents” like in unity, but instead for babylon (and three.js).

Anyway, help and/or thoughts would be appreciated.

1 Like
2

If you do agent.moveNorth(); sceneToRender.render(); instead of just agent.moveNorth(); in the console, it does not work. So, it’s something done by the scene that resets the agent position… I don’t know what, maybe try to disable the physic engine and see if that helps.

3

OK, this is weird… when I comment out this:

this.rayHelper.attachToMesh(
    this.body,            
    forward,
    this.body.localMeshOrigin,
    length
  );

but still do this this.rayHelper.show(arg)

it works just fine; do you have any idea why? I’m… quite confused…

4

The ray helper attachToMesh method overwrites ray.origin as part of its processing, and as you pass agent.body.position as the ray origin that may explain why the value is lost. Try to pass agent.body.position.clone() instead (or even new BABYLON.Vector3()).

However, it’s a side effect of using attachToMesh, if you don’t use it you don’t need to do that, as you discovered.

1 Like