As I wrote before, OimoJS does not support the MeshImpostor. It uses a box instead. So either use a BoxImpostor, or switch to a physics engine supporting the correct impostor-type, like CannonJS or AmmoJS: Use a Physics Engine - Babylon.js Documentation
Edit: Check this PG: Babylon.js Playground
Lines 21 - 23. Try OimoJS, and you’ll see the mesh isn’t used. Instead it uses the boundingBox of the model. CannonJS and AmmoJS both use the geometry.
Can be used with heightmap terrain as well: https://playground.babylonjs.com/#C0ILX9#1
(HeightmapImpostor only supported by CannonJS, OimoJS again uses boxes, and the AmmoJS plugin requires a MeshImpostor instead)