Thanks for the hint!
It just does not seem to be easy to control the zOffset properly. Though zOffsetUnit might be better suitable if you look at the following playground. But both options change when camera is moving away from object. I guess that is because of the depth render precision?
More on zOffset: ZOffset explanation
https://registry.khronos.org/OpenGL-Refpages/gl4/html/glPolygonOffset.xhtml
Calculation is: zOffset x DZ + r x zOffsetUnits “where DZ is a measurement of the change in depth relative to the screen area of the polygon, and r is the smallest value that is guaranteed to produce a resolvable offset for a given implementation.”