For some cases it would work with a box. I tried an invisible Box below the Ground mesh, which also helped somewhat. Thanks for the tip.
I found a solution with using forces instead of rotation. This doesn’t look as cool on the desktop though, but you cannot catapult the sphere like that, so I am not sure if I like this approach.
Another solution might be to reset the position of the sphere onBeforeRender or something.
I think I will either let the ball fall through sometimes or restrict it altogether. Was a nice idea.