This is all normal and expected as moveWithCollision is meant to be only representing ellipsoid shapes: Camera Collisions | Babylon.js Documentation
This playground from our friend @Wingnut might be helpfull:
All based on the mesh intersections so you can have fine control: Mesh Intersections | Babylon.js Documentation