The latter preferably. And it would be ideal if it could fire with respect to the mesh as it is animating, although I would also be ok with firing when the mouse is over the entire meshes bounding box as well.
Sorry for the delay, this one was pretty tricky to think through. A lot of credit goes to @Deltakosh for helping me wrap my head around this.
Ok so here’s the problem. Babylon optimizes skeletal animations, by passing the work of the animation to the GPU. This means that when a mesh is animated by a bone, the CPU doesn’t actually inherently know that the mesh’s position has changed. So you have to update the CPU with the mesh’s new position. Why? Because in the case of the observables, which you’re using here, the CPU is what’s doing the work to “listen” for pointer/mesh interactions.
SO what happens…well as far as the CPU is concerned, the mesh is lying flat on the ground, not animating at all.
The animation is technically there, but I’ve disabled skeletons in the scene to represent what the CPU is seeing. So this now makes sense as to why the action manager appears to fire events at the “feet” of the robot. It thinks the robot is lying down.
SOOOOO what we have to do is update the CPU on the mesh’s new position each frame, as you point out in your second playground. However, this is a HUGE red flag, because telling the CPU where each vertex is of every mesh in the scene is CRAZY expensive.
So this means we actually need some sort of proxy for the CPU…bounding boxes is probably the best way to go about this.
So the good news is that we can get 75% of the way there by getting the CPU to have updated bounding info for every mesh in the scene.
So the final 25% would be using something like a "scene.pickBoundingInfo() to get the mesh that was selected based on the bounding box under the cursor when a click occurred. The bad news is that such a helper function doesn’t exist in Babylon…YET.
I’ve added this issue to add this helper to the core engine!
Sorry for the long winded response, but it’s been a tricky one to wrap my head around.
I am by no means an expert on this subject, but I also wonder if there are easier ways to go about this. Perhaps exploring raycasting for example?
Thanks for the great response! I came to a lot of the same conclusions based on all the docs I read. Before your response I did actually go with a very crude solution of wrapping the mesh in a bounding box, and just using box for the hovering. This makes it so the CPU doesn’t need to be updated every time.
Very glad to see how the CPU sees the mesh, I could not figure out how to figure out how it was doing it’s scene picking.
I tried messing with ray picking and I think it would be basically the same. I did notice that there is a fastCheck for scene.pick that supposedly uses boundingInfo, but I couldn’t get it to work as expected with your example either.
Once that new helper is in, I will update to try it out, but for now looks like the current simple solution is my best bet w/o spending a lot more time on the problem.