This magnification is not because of the IOR but because you are using a probe, which by default is supposed to represent the geometry at infinite distance. You should use a local probe (cube map) instead, if you want a more localized render.
To convert a cube map to a local cube map, you must set the boundingBoxPosition and boundingBoxSize properties. For example: