Ray picking in SkinnedMesh

Questions
1

How to achieve in babylonjs that SkinnedMesh can be picked up by ray like threejs?
Threejs seems to have updated the SkinnedMesh transformation at the CPU level （guess）

I need to pick up the mesh faceId and normal and update my annotation position

Here is compare of video


2

Try this

// Create your skinned mesh
const skinnedMesh = /* your mesh with skeleton */;

// Generate normals
skinnedMesh.createNormals(true); // true for updatable

// Then in your pick handler
const pickResult = scene.pick(scene.pointerX, scene.pointerY, (mesh) => mesh === skinnedMesh);
if (pickResult.hit) {
    const faceId = pickResult.faceId;
    const normal = pickResult.getNormal(true); // true to useWorldCoordinates
    console.log("Face ID:", faceId, "Normal:", normal);
}
4

If I load a skeleton mesh, do I need to do this for every skinnedMesh?

5

Logically speaking - yes, considering that skeleton is not a mesh :slight_smile:

Here is the example with Hat picking - https://playground.babylonjs.com/#W2D1C2#3
And another example with Ray picking - Babylon.js Playground

1 Like
6

Someone has a good time :smiley:

1 Like
7

I tried the PG you provided. When the character moves horizontally, I click on the hat and he can’t pick it up. I also tried it in my project. Since mine is a robotic arm, I can’t pick it up when the robotic arm is raised.

8

Ray picking should be much more precise

9

i use threejs make a demo for your reference ，when the robot arm moves, it shoots a camera to the mouse position and it can pick up the

Is this possible in Babylon?

https://codesandbox.io/p/devbox/cy2pyk

10 
(skinnedMesh as BABYLON.Mesh).refreshBoundingInfo(true);

Now it seems one is always able to click the Hat :slight_smile: - https://playground.babylonjs.com/#W2D1C2#4

1 Like
11

cool that’s great, but I have a question. If the model is uploaded by the user, can I determine whether it belongs to skinnedMesh like threejs, and then update the bounding box information in the pointermve event to reduce unnecessary performance consumption?

12 
        for (const m of allModelMeshes) {
            console.log((m as BABYLON.Mesh).validateSkinning())
        }

Docs - validateSkinning
Example - https://playground.babylonjs.com/#W2D1C2#5

1 Like
13

Thanks for your answer, this is very useful :laughing:

1 Like
14

How to find the corresponding bone through the picked up mesh and attach the sprite to this bone?

15

One of possible ways - https://playground.babylonjs.com/#W2D1C2#6
(again with the Hat :slight_smile: )

1 Like
16

Thanks, it is really useful, but what if I want the sprite to be at the position where I click? It just follows the bone to make the same movement. Now it seems to be at the position of the root node of the bone that I found.

PG for robot：https://playground.babylonjs.com/#W2D1C2#10

17

One needs to calculate the local offset from the bone to the picked point.

image
image692×429 35 KB

1 Like
18

Great ! I use `TransformNode’ to help me ,it automatically calculate the location for me :grinning_face:

This is PG using TransformNode. If someone encounters this problem in the future, I hope he can find another solution.

https://playground.babylonjs.com/#W2D1C2#13

19

I believe something is possible as well with Hotspots queries - Babylon.js docs

3 Likes