It seems to be just as I guessed.
I also read the article on GPU pickup schemes.
However, since detailed coordinates and normal direction need to be obtained at that time, ray picking can be optimized using Octree. I gave up delving into GPU pickup schemes.
Read this article：Using forceSharedVertices() to enable ray picking breaks my morph targets，You mentioned that you can use the depth buffer to obtain 3D coordinates, so is there a way to calculate the normal.