Get accurate FacetData from mesh that is animated by a Skeleton

So right now I have a situation where I need to get picking facetData from a mesh that has been manipulated by a skeleton.

I have controls to allow the user to move the bones and do transforms, which then update the mesh. The problem now is when the mesh has been moved by these methods, the picking methods stop working.

The only way to get it close to working was to turn of GPU skinning, and after every transform do a

    this.finalMesh.updateFacetData()
    this.finalMesh.refreshBoundingInfo(true)

Just wondering if there is a better way to do this, and if there is a way to keep the Shader Skeleton computation intact with still getting my facet picks.

Not sure to understand. Moving the bones should be enough to see it right? Why do you need to pick facetData?

In a project for a client here I am making a situation where they can click on a mesh with a skeleton, it checks the facet data for weights and returns the index of the bone with the most weight at that facet location. Which I then use to select the bone the user then has the option to rotate the selected bone and this is where it all falls apart.

ok so your solution is the right one :slight_smile:

Is there any way to make the picking more accurate then other then what I am doing?

unfortunately no there is no back channel from the GPU

You can try to draw your mesh in an offscreen texture by using a different color for each facet depending on the bone with the max weight (you will need something like convertToFlatShadedMesh to get a uniform color accross each facet). Then, read the pixel color from this offscreen texture to infer the bone.

You will need to get the texture data from GPU to CPU, which can be slow if done too often / continuously. Given your use case, it seems you can do the transfer only if 1/ the camera view has changed and 2/ the user has clicked down.

2 Likes

Dude you are right on point! I was talking about doing a GPU picking method for that and was trying to see if there was a hacky way to do it like that. You suggesting that give me lots of hope. The other method I was going to try to work out was to make a shader that calculates a signed distance function that represents the mesh parts then have it ping-pong the results back to the CPU but that seems like it would take some wizardry. This texture based pick method might be the ticket!

So I am going to end up having to do that GPU method I think. Any pointersm on how to start that? Cause right now no matter how I subdivide the octree and refine the facet picking there is still situations where the ray misses the facet data but I guess hits the bounding box and returns data but not the right data.

it seems to always be tword the very tips of the geometry and only at certain angles.

Not specifically, but “gpu picking” in google should get some pointers.

1 Like