Pick animated mesh

Questions
1

Hello everyone,

I am working on a project where I need to pick animated meshes. I read that for performance reasons the animation is done on gpu side while pick uses cpu, wherefore the pick can’t get the animated data. Is there any workaround? For the moment of picking the mesh, it is not animated. So it can animate to different poses and then the user should be able to pick. Since the mesh should be animatable after the pick I would like to avoid something like baking. Is there any way to do that?

Best

2

I’m assuming this is the thread you were taking a look at? :wink:

@Evgeni_Popov any insight? I’d love to learn more as well. :slight_smile:

1 Like
3

Thanks for you reply! Yes, this is one of the threads I read. I digged further and found a solution in another thread.

Before picking the mesh the bounding info hast to be refreshed:

for (let i = 0; i < this.scene.meshes.length; i++) this.scene.meshes[i].refreshBoundingInfo(true)

4

Okay, too early! I encountered a new problem. The normals are still not updated when using “pointerInfo.pickInfo.getNormal(true, true)”.

5

Pinging @Deltakosh, but I think it would hit perf too much if the normals were also recomputed on the CPU. Currently, only positions are recomputed to take into account the skeleton/morph targets that could exist.

You can try to get the 3 vertices of the face that was picked (pickInfo.faceId) and compute the normal from there (the updated vertex positions can be retrieved from mesh._positions and the face indices from mesh.getIndices()).

6

That should work for me. It’s a good idea to use the updated vertices to recalculate the normal.

My approach is not working unfortunately. The vertex positions are definetely not correct but I am not sure why:

const mesh = pointerInfo.pickInfo.pickedMesh

        const positions = mesh.getVerticesData(VertexBuffer.PositionKind)

        const faceId = pointerInfo.pickInfo.faceId

        const indices = mesh.getIndices()

        const v1 = new Vector3(

            positions[indices[faceId*3]],

            positions[indices[faceId*3]+1],

            positions[indices[faceId*3]+2]

        )

        const v2 = new Vector3(

            positions[indices[faceId*3+1]],

            positions[indices[faceId*3+1]+1],

            positions[indices[faceId*3+1]+2]

        )

        const v3 = new Vector3(

            positions[indices[faceId*3+2]],

            positions[indices[faceId*3+2]+1],

            positions[indices[faceId*3+2]+2]

        )

        const a = v2.subtract(v1)

        const b = v3.subtract(v1)

        const dir = a.cross(b)

        const norm = dir.normalize()
7

The updated positions are in mesh._positions, not in mesh.getVerticesData("position") (which are the initial positions, without the skeleton/morph modifications applied).

8

Sure, I noticed that you mentioned that. But I tried to get it done in initial position first. So the mesh isn’t modificated by the skeleton but the vertex positions are not correct anyway. Is this the right method to get them?

const v1 = new Vector3(
positions[indices[faceId*3]],
positions[indices[faceId*3+1]],
positions[indices[faceId*3+2]]
)
9

That would be more along the line:

const v1 = new BABYLON.Vector3(
    positions[indices[faceId * 3 + 0] * 3 + 0],
    positions[indices[faceId * 3 + 0] * 3 + 1],
    positions[indices[faceId * 3 + 0] * 3 + 2],
);

const v2 = new BABYLON.Vector3(
    positions[indices[faceId * 3 + 1] * 3 + 0],
    positions[indices[faceId * 3 + 1] * 3 + 1],
    positions[indices[faceId * 3 + 1] * 3 + 2],
);

const v3 = new BABYLON.Vector3(
    positions[indices[faceId * 3 + 2] * 3 + 0],
    positions[indices[faceId * 3 + 2] * 3 + 1],
    positions[indices[faceId * 3 + 2] * 3 + 2],
);
1 Like
10

That’s very helpful. Thanks!

So in initial position …

mesh._positions[indices[faceId*3]]

… equals …

const positions = mesh.getVerticesData(VertexBuffer.PositionKind)
new Vector3(
    positions[indices[faceId*3]*3],
    positions[indices[faceId*3]*3+1],
    positions[indices[faceId*3]*3+2]
)

That works perfectly. Is it save to use mesh._positions?

11

I think so @Deltakosh? Maybe we can make this accessor public and remove the leading underscore?

12

It is not always populated
It is generated when we initiate a picking (or a collisions)

but I guess we could expose a positions property that will do the check and generate it on demand