Constraining a UniversalCamera on a Navigation Mesh

Moving this conversation from the Navigation Mesh PR.

@tibotiber:
[Navigation Mesh] looks really cool @Cedric. Just tried it, an it’s super easy to setup and compute the navigation mesh. Is there any way this could be used to constraint the walkable area for a UniversalCamera?

@Cedric:
Hi Thibaut

It should be possible to attach a camera to a transform agent. Something like:

camera.parent = agent.transform;

But you’ll have to create an agent as it’s the only object constrained by the navigation automatically. Or, every frame, you can do it manually with a getClosestPoint call and set the camera position relative to that point.

@tibotiber:
Hi! So I’ve tested what you suggested Cédric and some things are worth reporting for whoever attempts something similar.

Setting the camera’s parent to the agent’s transform node:
This is pretty neat. If you implement the picking to move agents to a location demonstrated in your PG demo, you get a pick-to-move-camera à la Google Street View. It doesn’t mix well with keyboard inputs however since the FreeCameraKeyboardMoveInput will update the camera position, while the agent will update the parent’s position, resulting in a hard to control global position which almost always gets out of the navmesh. As long as you stay away from the keyboard or remove that input, it’s pretty interesting though. To be combined with a camera rotation input using keyboard or swipe (I’ll test that next).

using getClosestPoint at every frame:
This clearly is the simplest way to make sure the camera remains on the navmesh, and it’s compatible out of the box with the FreeCameraKeyboardMoveInput . There are some weird behaviours though. For example, when you walk under some stairs which are part of the navmesh, the camera can sometimes jump on top of the stairs. I have seen other jump behaviours near edges of the navmesh. I haven’t got a full multi-storey map to test further unfortunately. From the first tests, it’s as if getClosestPoint ignored the height (y value) when picking the closest point. Is that possible?

Quick question also:
Why are we disconnecting the camera in your PG demo when picking an agents’ location?

Pinging @Cedric

Hi @tibotiber

Welcome to the forum :slight_smile:

When you compute the reference point to call getClosestPoint with, can you try to set it slightly below. Y component should be used for getting the closest point so setting it a bit below might give you better result. If you have a playground with multilevel geometry, I’d be curious to give it a try.

No particular reason for disconnection the camera. I just found it easier to show the agents and do the picking.

Hi @Cedric

Thank you, glad to be here :slight_smile:.

I don’t have an easy to share geometry because my app allows me to generate the geometry (floorplan) I’m testing so I can’t just extract it and put it in a PG. I could give you access to the app if you want to check it out from a “user” perspective but no easy code access. However, I searched a bit the PG and found a multilevel house which I modified to generate the navmesh and add a navmesh-bound camera. You can test it here: https://www.babylonjs-playground.com/#RNCYVM#64.

Notes:

  1. you can use AWDS keys to rotate the camera. (bindings may be weird, I’m not a gamer :stuck_out_tongue:)
  2. in this example, the stairs don’t get picked up in the navmesh. I can’t understand why, and that’s not a problem I have in my app. See screenshot below.

For the y-component, I have tried lowering it but it doesn’t seem to help even with big values. So in the screenshot above, if I walk behind the stairs (there is a navmesh path) it’s ok, but as soon as I step out of the navmesh, the calculated closest point is on top of the stairs, not the ground floor navmesh right beside it :thinking:.

One more thing, the camera sometimes manages to walk through walls. I could increase the cs value but it would get harder to get through doors then. Any idea on this?

To fix both of these, I wonder if we could somehow skew the getClosestPoint result towards the last known point on the navmesh, so these “jumps” would not happen.

(ahah, ok for the disconnected camera, no magic potion there then)

Yes, if you lose details from your mesh, the only solution is to increase cs/ch until the navmesh corresponds to what you want.

For the stairs not picked up by the system, increase the walkableClimb value ( I used 3 in my test). Then, because of the floor, there is a zone that might not be reachable in the middle of the stairs. decrease walkableHeight. Any height (roof-floor) smaller than that will be discarded.

image

PG : https://www.babylonjs-playground.com/#RNCYVM#65

For the closest point thing, I don’t think that the logic you expose should be part of the navigation. Can you try to make a couple calls to getClosestPoint (with different heights) and select the one that suits you well?

One more thing. If you let camera move freely then change its position thanks to getClosestPoint, the camera might move way beyond the navmesh. getclosestpoint will find the best candidate but way forward the navmesh. Recast provides a function to ‘cast’ a segment along the navmesh (not exposed yet in the API).
With that function, you would compute the camera destination, substract that position with the current camera position. And limit that vector using Recast. Then add that constraint vector to the current camera position. I’m sure this eliminate the jumps between floors.

I’ll add that functionality in the coming days.

I did the PR for the moveAlong function.

This function casts a segment on the navmesh and returns the furthest position along that segment that is on the navmesh. Call this with (previousCameraPosition, nextCameraPosition)
I also found on your test scene that setting maxSimplificationError to 0 makes a more blocky navmesh but better result with the stairs. So, more tweaking for you is expected :slight_smile:

Oh yes, good catch. I did increase walkableClimb previously but I didn’t realise the roof getting closer when you get up the stairs. Tall people problems haha.

Now that you’ve fixed this, I guess you are able to observe the “jump” behaviour. In your PG, I’m getting it easily when walking near the top of the stairs. That’s also what I think, the issue is that the camera moves too far out of the navmesh before getClosestPoint is executed for correction. Great idea for moveAlong, and thanks for adding it so fast :rocket:. I’ll try that and update the PG (once the preview release is published), thank you so much for the great help :pray:.

Noted for maxSimplificationError, will tweak off once I get to fine tuning.

Hi @Cedric

I took a while to test this and revert as I missed that the preview release was updated. I’ve just tried it. A few things.

  1. moveAlong seems to work well to avoid the jumps (through wall or near stairs) but it seems to be a 2D method only, height is not taken into account so there is no issue changing the Y coordinate of the camera in my case (like a helicopter take off), and I walk through stairs instead of up onto them. I’ve checked Recast and I guess your implementation relies on dtNavMeshQuery::moveAlongSurface (docs). There doesn’t seem to be a 3D equivalent. Quoting the docs, we may need to add getPolyHeight as well to get build a 3D version ourselves.

    resultPos is not projected onto the surface of the navigation mesh. Use getPolyHeight if this is needed.

    I have tried to combine moveAlong for the XZ position and then getClosestPoint for the Y position, but I’m back to stairs jumps again. Do you think things would be better with getPolyHeight?

  2. Looking at the 2D moveAlong issue, I wonder if that’s not something similar we’re hitting with getClosestPoint. Which Recast method is that using? I’ve found dtNavMeshQuery::closestPointOnPoly (docs) and dtNavMeshQuery::closestPointOnPolyBoundary (docs), the 2nd one stating that “the height detail is not used”.

  3. I’ve an updated PG but the latest recast preview has apparently not been published so we’re getting TypeError: this.navMesh.moveAlong is not a function.

  4. Something new and unrelated: I’ve consistently experienced a weird bug where the world origin Vector3.Zero() is considered part of the navmesh whether it belong to it or not. See screenshot below. Any idea why?

Once again, thanks a lot for your help!

Bonus question: is the source for the recast.js available anywhere?

Hi @tibotiber

1-2 Let me check for the height value. I’m sure I can improve that.
3 That’s weird! I’m checking that too
4 (0,0,0) is the value returned when recast fails at finding a result
5 yes, the source is available in extensions GitHub - BabylonJS/Extensions: Extensions for Babylon.js
and recastjs is accessible here Extensions/recastjs at master · BabylonJS/Extensions · GitHub

  1. It’s the first function that’s used
  2. I know what I did wrong. I’ll update the recast.js very soon.

I’ve added a new function in this PR


That allows to set the extent box query. Basically, set a small value for finer result. bigger value for broader result query.
So, setting a small value on the navmesh (like 0.1,0.1,0.1) should make your navigation much better.

Hi @Cedric

Thank you for your answers and adding the glue for extent params, looks like it’ll be a good help. I’m guessing the getClosestPoint failure could be solved by increasing the extent(?)(was it (1,1,1) by default until now?). And yes, looking forward to try the navigation with a smaller extent, especially vertically.

I’ll update the PG once the PR is merge and preview release updated.

Thanks also for indicating that (0,0,0) is a failure by Recast to perform a spatial query. This silent failure seems a bit unnatural, I’m guessing this is something internal to Recast, and not addressable by your wrapper. Maybe we could document this? I don’t mind doing it if you point me to the best place, here maybe?

To get a closer point you have to use a smaller extent. Imagine a bounding box around the parameter point that Recast uses to compute a valid solution.

I’m updating the documentation. The extensions doc is on GitHub at this address : Documentation/content/extensions/Navigation at master · BabylonJS/Documentation · GitHub

Doc update is merged precision on queries and query extent by CedricGuillemet · Pull Request #1666 · BabylonJS/Documentation · GitHub