Forgive me if this is a silly question but I just found there is something wrong with what I think of this.

In the playground, I think the black part should be yellow but they are not.
The left point and the center point have different distance to camera but they have same depth value, it is strange for me.

itâ€™s prolly coz depth map measures distance from the near plane and not the camera position exactly.

we can rectify this taking the camera fov into account.

Since if we know the fov, then we know how much the angle of view changes from left of screen to right of screen and we can correct the depth we read by calculating the falloff.

Itâ€™s the normal behavior of OpenGL based rendering, and by the way thatâ€™s why itâ€™s called â€śDepthâ€ť. Best way to understand this is taking the camera as a boat over the see :

The â€śDepthâ€ť is indeed the same for points A and B, even if the distance is not the same. Depth map is a depth map, not a distance map.

Now, if you need the distance you can easily create a function depth_to_distance which compute distance from depth map. Itâ€™s just a matter of trigonometric cosinus computation, using the FOV of the camera, the image size, and the 2D coordinates of the considered point