# GPU picking point and normal example **URL:** \ **Category:** Demos and projects\ **Tags:** picking\ **Created:** [May 19, 2023, 6:08am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965 "2023-05-19T06:08:27Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![xiehangyun](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/xiehangyun/32/30777_2.png) [@xiehangyun](https://forum.babylonjs.com/u/xiehangyun)\ **Post date:** [May 19, 2023, 6:08am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/1 "2023-05-19T06:08:27Z")
This is an example of using depth textures to pick up world coordinates and normals： [GPU picking point and normal | Babylon.js Playground (babylonjs.com)](https://playground.babylonjs.com/#01U4D7#155) Combined with GPU object pickup, the capability of near ray pickup can be realized： [GPU picking demo - Demos and projects - Babylon.js (babylonjs.com)](https://forum.babylonjs.com/t/gpu-picking-demo/30507) GPU pickup can save CPU performance in complex scenarios. But compared with ray picking, there is a problem of accuracy error. In order to reduce the impact of accuracy errors on the normal calculation, I had to increase the sampling pixel distance of the normal calculation, but this amplified the error of the normal calculation in boundary cases (such as cube corners). Do you have any suggestions? New Edit: The error issue has been fixed。 ---
**Author:** ![kzhsw](https://avatars.discourse-cdn.com/v4/letter/k/848f3c/32.png) [@kzhsw](https://forum.babylonjs.com/u/kzhsw)\ **Post date:** [May 19, 2023, 7:08am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/2 "2023-05-19T07:08:13Z")
That sounds great, but when mouse is moved near the border of canvas, ths normal seems incorrent. ![screenshot](https://us1.discourse-cdn.com/flex024/uploads/babylonjs/original/3X/a/5/a542ba01093d97e007e50abd01eb75b22e478784.gif) ---
**Author:** ![xiehangyun](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/xiehangyun/32/30777_2.png) [@xiehangyun](https://forum.babylonjs.com/u/xiehangyun)\ **Post date:** [May 19, 2023, 7:15am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/4 "2023-05-19T07:15:13Z")
Well, reducing offsetBase can alleviate this problem. However, due to precision error, there will be new problems. I think this method is feasible, but my algorithm is not perfect. 🥲 I’m looking for a solution. ---
**Author:** ![kzhsw](https://avatars.discourse-cdn.com/v4/letter/k/848f3c/32.png) [@kzhsw](https://forum.babylonjs.com/u/kzhsw)\ **Post date:** [May 19, 2023, 7:16am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/5 "2023-05-19T07:16:03Z")
Could using webgpu compute shader instead of rendertargettexture help? ---
**Author:** ![xiehangyun](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/xiehangyun/32/30777_2.png) [@xiehangyun](https://forum.babylonjs.com/u/xiehangyun)\ **Post date:** [May 19, 2023, 7:17am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/6 "2023-05-19T07:17:55Z")
I can’t figure out what to do with it, this scheme is based on depth textures. ---
**Author:** ![Pryme8](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/pryme8/32/7212_2.png) [@Pryme8](https://forum.babylonjs.com/u/Pryme8)\ **Post date:** [May 19, 2023, 7:26am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/7 "2023-05-19T07:26:06Z")
This is awesome even with the error. ---
**Author:** ![kzhsw](https://avatars.discourse-cdn.com/v4/letter/k/848f3c/32.png) [@kzhsw](https://forum.babylonjs.com/u/kzhsw)\ **Post date:** [May 19, 2023, 7:28am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/8 "2023-05-19T07:28:57Z")
I mean reproducing the original cpu-based algorithm in gpu compute shader, using GPUBuffer from mesh as input, would this suffer from the same issue here? ---
**Author:** ![Evgeni\_Popov](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/evgeni_popov/32/4432_2.png) [@Evgeni\_Popov](https://forum.babylonjs.com/u/Evgeni_Popov)\ **Post date:** [May 19, 2023, 8:39am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/10 "2023-05-19T08:39:23Z")
See [How to obtain the correct picking result after morph application? - #8 by Evgeni\_Popov](https://forum.babylonjs.com/t/how-to-obtain-the-correct-picking-result-after-morph-application/40901/8) for the discussion about precision errors. ---
**Author:** ![xiehangyun](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/xiehangyun/32/30777_2.png) [@xiehangyun](https://forum.babylonjs.com/u/xiehangyun)\ **Post date:** [May 19, 2023, 12:40pm UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/11 "2023-05-19T12:40:00Z")
[GPU picking point and normal | Babylon.js Playground (babylonjs.com)](https://playground.babylonjs.com/#01U4D7#146) I followed the solution in the article to handle the boundary case, and now it works a lot better. ---
**Author:** ![xiehangyun](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/xiehangyun/32/30777_2.png) [@xiehangyun](https://forum.babylonjs.com/u/xiehangyun)\ **Post date:** [May 19, 2023, 1:22pm UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/13 "2023-05-19T13:22:47Z")
[GPU picking point and normal | Babylon.js Playground (babylonjs.com)](https://playground.babylonjs.com/#01U4D7#146) Version that addresses accuracy error issues. ---
**Author:** ![Pryme8](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/pryme8/32/7212_2.png) [@Pryme8](https://forum.babylonjs.com/u/Pryme8)\ **Post date:** [May 19, 2023, 4:38pm UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/14 "2023-05-19T16:38:29Z")
This is super useful, I can think of a ton of use cases for it. Thanks for the pg. ---
**Author:** ![CodingCrusader](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/codingcrusader/32/29857_2.png) [@CodingCrusader](https://forum.babylonjs.com/u/CodingCrusader)\ **Post date:** [May 19, 2023, 5:26pm UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/15 "2023-05-19T17:26:16Z")
I found two more examples in the playground made with fragment shader. [Disc on multiple meshes with slider | Babylon.js Playground (babylonjs.com)](https://playground.babylonjs.com/#8ZQVW1#4) [Colorize facet mesh | Babylon.js Playground (babylonjs.com)](https://playground.babylonjs.com/#UIVCZ3#6) ---
**Author:** ![sebavan](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/sebavan/32/57_2.png) [@sebavan](https://forum.babylonjs.com/u/sebavan)\ **Post date:** [May 19, 2023, 9:10pm UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/16 "2023-05-19T21:10:06Z")
This is pretty cool 🙂 I wonder how we could make it a Babylon feature and when exactly the perf tips from cpu to gpu picking ---
**Author:** ![xiehangyun](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/xiehangyun/32/30777_2.png) [@xiehangyun](https://forum.babylonjs.com/u/xiehangyun)\ **Post date:** [May 20, 2023, 1:09am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/17 "2023-05-20T01:09:51Z")
Their coordinate picking is still based on scene.pick This method, the underlying implementation also seems to use rays. ---
**Author:** ![xiehangyun](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/xiehangyun/32/30777_2.png) [@xiehangyun](https://forum.babylonjs.com/u/xiehangyun)\ **Post date:** [May 20, 2023, 1:13am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/18 "2023-05-20T01:13:26Z")
First, create a dedicated depth map for the pickup list. Add parameters in the pick method? Or add a property to the scene? I think that ray picking and GPU picking have their own advantages and disadvantages, and giving users the freedom to choose seems to be the optimal solution. ---
**Author:** ![CodingCrusader](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/codingcrusader/32/29857_2.png) [@CodingCrusader](https://forum.babylonjs.com/u/CodingCrusader)\ **Post date:** [May 20, 2023, 8:06am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/19 "2023-05-20T08:06:51Z")
I’m sorry to hear that. I haven’t looked at the pick method in detail. But if your applications are time-critical, I would suggest running time-intensive functions with [gpu.js](https://gpu.rocks/#/). Also, I asked GPT what fast methods are out there. GPT The fastest methods for ray picking, also known as ray casting or ray tracing, depend on the specific context and requirements of the application. Here are a few commonly used techniques: 1. Bounding Volume Hierarchy (BVH): Construct a hierarchical data structure, such as a BVH tree, to accelerate ray-object intersection tests. The tree organizes objects based on their spatial relationships, allowing for efficient pruning of large portions of the scene during ray traversal. 2. Spatial Partitioning: Divide the scene into spatial partitions, such as grids or octrees, to quickly identify potential intersections with objects in the ray’s path. This approach reduces the number of intersection tests required by excluding objects in empty or irrelevant partitions. 3. GPU Acceleration: Utilize the parallel processing power of modern graphics processing units (GPUs) to perform ray-object intersection tests efficiently. Techniques like GPU ray tracing or shader-based algorithms (e.g., ray marching) can leverage the GPU’s computational capabilities for real-time ray picking. 4. Ray-Sphere or Ray-AABB Intersection Tests: For simple shapes like spheres or axis-aligned bounding boxes (AABBs), use specialized intersection tests that can be computed quickly without complex computations. 5. Culling Techniques: Implement early exit strategies, such as back-face culling, to discard objects facing away from the ray’s origin. This can help reduce the number of intersection tests performed. It’s important to note that the choice of method depends on factors like scene complexity, object types, available hardware, and performance requirements. A combination of these techniques or additional optimization strategies may be necessary to achieve the desired level of performance in ray picking applications. * * * Last but not least for your interesst this is demo shows the cost of raycasting per pixel or ray: [Ray-Casting Algorithm Internal Stuff - YouTube](https://www.youtube.com/watch?v=R9CA8i2Of_g&ab_channel=AtomontageInc.) Have nice day! 🙂 ---
**Author:** ![xiehangyun](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/xiehangyun/32/30777_2.png) [@xiehangyun](https://forum.babylonjs.com/u/xiehangyun)\ **Post date:** [May 20, 2023, 10:07am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/20 "2023-05-20T10:07:10Z")
Ray pickup speed can be accelerated at Babylon using octree. I chose to study GPU pickup because there were other factors besides speed. For more information, see: [How to obtain the correct picking result after morph application? - #11 by xiehangyun](https://forum.babylonjs.com/t/how-to-obtain-the-correct-picking-result-after-morph-application/40901/11) Of course, using the GPU .js is also an important recommendation that I will learn and try to use in my programs. Thanks for your reply! ---
**Author:** ![xiehangyun](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/xiehangyun/32/30777_2.png) [@xiehangyun](https://forum.babylonjs.com/u/xiehangyun)\ **Post date:** [June 25, 2023, 9:07am UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/21 "2023-06-25T09:07:37Z")
[GPU picking point and normal | Babylon.js Playground (babylonjs.com)](https://playground.babylonjs.com/#01U4D7#269) I’m trying to integrate GPUDepthPicking with GPUColorPicking. This enables simple GPU picking. Can I ask for your help and submit a PR when the feature is complete enough. ---
**Author:** ![xiehangyun](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/xiehangyun/32/30777_2.png) [@xiehangyun](https://forum.babylonjs.com/u/xiehangyun)\ **Post date:** [June 26, 2023, 1:26pm UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/22 "2023-06-26T13:26:15Z")
[GPU picking point and normal | Babylon.js Playground (babylonjs.com)](https://playground.babylonjs.com/#01U4D7#283) This is the biggest performance optimization I can do, and it’s probably still not as fast as spatially demarcated ray pickup. The advantage is that vertex changes that only exist on the GPU side, such as warps and skinned meshes, can be picked up correctly. If you can ensure that the camera is stationary and the picking list is unchanged, using cached readPixels data may greatly improve speed. GPUDepthPicking is WebGPU compatible. ---
**Author:** ![sebavan](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/sebavan/32/57_2.png) [@sebavan](https://forum.babylonjs.com/u/sebavan)\ **Post date:** [June 26, 2023, 3:08pm UTC](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965/23 "2023-06-26T15:08:10Z")
cc @Evgeni_Popov [Next page](https://forum.babylonjs.com/t/gpu-picking-point-and-normal-example/40965.md?page=2)