This is an example of using depth textures to pick up world coordinates and normals：
GPU picking point and normal | Babylon.js Playground (babylonjs.com)
Combined with GPU object pickup, the capability of near ray pickup can be realized：
GPU picking demo - Demos and projects - Babylon.js (babylonjs.com)
GPU pickup can save CPU performance in complex scenarios.
But compared with ray picking, there is a problem of accuracy error.
In order to reduce the impact of accuracy errors on the normal calculation, I had to increase the sampling pixel distance of the normal calculation, but this amplified the error of the normal calculation in boundary cases (such as cube corners).
Do you have any suggestions?
New Edit:
The error issue has been fixed。
4 Likes
That sounds great, but when mouse is moved near the border of canvas, ths normal seems incorrent.
Well, reducing offsetBase can alleviate this problem. However, due to precision error, there will be new problems. I think this method is feasible, but my algorithm is not perfect. 
I’m looking for a solution.
Could using webgpu compute shader instead of rendertargettexture help?
I can’t figure out what to do with it, this scheme is based on depth textures.
This is awesome even with the error.
I mean reproducing the original cpu-based algorithm in gpu compute shader, using GPUBuffer from mesh as input, would this suffer from the same issue here?
See How to obtain the correct picking result after morph application? - #8 by Evgeni_Popov for the discussion about precision errors.
1 Like
GPU picking point and normal | Babylon.js Playground (babylonjs.com)
I followed the solution in the article to handle the boundary case, and now it works a lot better.
1 Like
GPU picking point and normal | Babylon.js Playground (babylonjs.com)
Version that addresses accuracy error issues.
1 Like
This is super useful, I can think of a ton of use cases for it. Thanks for the pg.
I found two more examples in the playground made with fragment shader.
Disc on multiple meshes with slider | Babylon.js Playground (babylonjs.com)
2 Likes
This is pretty cool 
I wonder how we could make it a Babylon feature and when exactly the perf tips from cpu to gpu picking
Their coordinate picking is still based on scene.pick This method, the underlying implementation also seems to use rays.
First, create a dedicated depth map for the pickup list.
Add parameters in the pick method?
Or add a property to the scene?
I think that ray picking and GPU picking have their own advantages and disadvantages, and giving users the freedom to choose seems to be the optimal solution.
1 Like
I’m sorry to hear that. I haven’t looked at the pick method in detail. But if your applications are time-critical, I would suggest running time-intensive functions with gpu.js.
Also, I asked GPT what fast methods are out there.
GPT
The fastest methods for ray picking, also known as ray casting or ray tracing, depend on the specific context and requirements of the application. Here are a few commonly used techniques:
- Bounding Volume Hierarchy (BVH): Construct a hierarchical data structure, such as a BVH tree, to accelerate ray-object intersection tests. The tree organizes objects based on their spatial relationships, allowing for efficient pruning of large portions of the scene during ray traversal.
- Spatial Partitioning: Divide the scene into spatial partitions, such as grids or octrees, to quickly identify potential intersections with objects in the ray’s path. This approach reduces the number of intersection tests required by excluding objects in empty or irrelevant partitions.
- GPU Acceleration: Utilize the parallel processing power of modern graphics processing units (GPUs) to perform ray-object intersection tests efficiently. Techniques like GPU ray tracing or shader-based algorithms (e.g., ray marching) can leverage the GPU’s computational capabilities for real-time ray picking.
- Ray-Sphere or Ray-AABB Intersection Tests: For simple shapes like spheres or axis-aligned bounding boxes (AABBs), use specialized intersection tests that can be computed quickly without complex computations.
- Culling Techniques: Implement early exit strategies, such as back-face culling, to discard objects facing away from the ray’s origin. This can help reduce the number of intersection tests performed.
It’s important to note that the choice of method depends on factors like scene complexity, object types, available hardware, and performance requirements. A combination of these techniques or additional optimization strategies may be necessary to achieve the desired level of performance in ray picking applications.
Last but not least for your interesst this is demo shows the cost of raycasting per pixel or ray:
Ray-Casting Algorithm Internal Stuff - YouTube
Have nice day!
1 Like
Ray pickup speed can be accelerated at Babylon using octree.
I chose to study GPU pickup because there were other factors besides speed. For more information, see: How to obtain the correct picking result after morph application? - #11 by xiehangyun
Of course, using the GPU .js is also an important recommendation that I will learn and try to use in my programs.
Thanks for your reply!
GPU picking point and normal | Babylon.js Playground (babylonjs.com)
I’m trying to integrate GPUDepthPicking with GPUColorPicking.
This enables simple GPU picking.
Can I ask for your help and submit a PR when the feature is complete enough.
GPU picking point and normal | Babylon.js Playground (babylonjs.com)
This is the biggest performance optimization I can do, and it’s probably still not as fast as spatially demarcated ray pickup.
The advantage is that vertex changes that only exist on the GPU side, such as warps and skinned meshes, can be picked up correctly.
If you can ensure that the camera is stationary and the picking list is unchanged, using cached readPixels data may greatly improve speed.
GPUDepthPicking is WebGPU compatible.