New feature: Mesh Hotspots

Announcements
1

Mesh hotspot is a new feature aiming at computing a world coordinate from a surface info of an animated and deformed mesh. Basically, from a triangle and barycentric infos, it computes the world coordinates of that point, for skinned, morphed or static meshes.

Soon to be merged documentation, with code is here : Mesh hotspot by CedricGuillemet · Pull Request #1215 · BabylonJS/Documentation · GitHub

And 2 examples with animated mesh are visible here :

image
image887×954 57.6 KB

image
image721×690 38 KB

24 Likes
2

I love it ! :grin:
I remember helping @alexandremottet when he was trying exactly this, manually, in this topic
Compared to such manual solution (double click on surface) :

const indices = pickedMesh.getIndices();
let bw = 1 - bu - bv;
// run the following on each render :
let pointPosition = BABYLON.Vector3.Zero();
pointPosition.addInPlace(facePositions[0].scale(bu));
pointPosition.addInPlace(facePositions[1].scale(bv));
pointPosition.addInPlace(facePositions[2].scale(bw));
// etc, etc...

We will save a lot of code ! :grin:

Is it supposed to work as well at picking time or not ? Because what I can notice, is that on my side, even tough it works after picking (tracking of the point along animated surface), the picking itself seems to happen on the static mesh (picking multiple times at the same 2D spot won’t change the picked point, while the animation is running. Also, sometimes picking outside in the background actually picks the mesh, etc…)

2 Likes
3

Yes there is a little more work needed to make sure picking is accurate while skeletal or morph target animations are running. Specifically, refreshBoundingInfo must be called at the end of a frame right before picking. Here is an updated PG: https://playground.babylonjs.com/#BQOFIX#16

3 Likes
4

Neat!

Something I noticed, when you open the inspector, the hotspot seems to be off.

I also had this issue with my own version of this functionality, just never bothered fixing it since it’s a niche scenario. But curious on how one would fix this.

5

This is due to the viewport not being updated with new rendering sizes.
Once the update is done everyframe, it works fine.

5 Likes
6

@ryantrem I see you added a interpolateTo method to the ArcRotateCamera (to make use of with the hotspots), however when testing it out it doesn’t seem to… interpolate, and rather just instantly jumps to the given position. Is this intentional? And if so may I ask why you chose to name it interpolateTo rather than e.g. moveTo or jumpTo even though it doesn’t interpolate? It’s slightly confusing :sweat_smile:

camera.interpolateTo(angle[0], angle[1], angle[2] + (window.innerWidth < 756 ? (angle[6] != undefined ? angle[6] : 4) : 0), new Vector3(angle[3], angle[4], angle[5]));

Thanks for your great work as always :slight_smile:

7

Hey @leo_h - thanks for the feedback, much appreciated! ArcRotateCamera already had behavior to interpolate its pose when resetting to the previously saved state (e.g. storeState and restoreState), so when I introduced the interpolateTo function it was a bit of refactoring of the existing code (so that you could specify the new camera pose to interpolate to rather than only being able to interpolate to the single stored state). For storeState and restoreState, the interpolation relies on the restoreStateInterpolationFactor property, which is 0 by default (for back compat). The new interpolateTo function also relies on this property, and you can see an example here: Babylon.js Playground

That said, I realize now that this is confusing, and I’m trying to think of the best way to improve the API while maintaining back compat. I think maybe what would be best would be to add one more (optional) parameter to interpolateTo for the interpolationFactor. I think the behavior that would make sense would be:

  1. If an argument is explicitly passed in for the interpolationFactor parameter, that would be the factor used.
  2. If an argument is not explicitly passed in for the interpolationFactor parameter, then it would default to the restoreStateInterpolationFactor if it is non-zero, otherwise it would default to some “reasonable value” (~0.1).

Let me know if you have thoughts on this!

1 Like
8

That does sound like a good solution, yes! That way it leaves both options open - and allows for the functionality of directly using interpolateTo to interpolate as the name suggests rather than relying on other factors being set beforehand.

Also - I know it’s super new so no rush, but it would be great to have it added into the documentation as well at some point! Having an easy-to-use camera interpolation function I think is great - it’s something which I imagine is quite common to want and until now relied on either “misusing” state restoration (at least from a naming - and therefore readability - perspective) or having to implement a bunch of animation and easing stuff. So I think it should definitely be included in the documentation so people know about it!

Thanks as always for being open to feedback - I was chatting with some PhD students + lecturer the other day about babylon.js and the activeness of you and the other devs on this forum was a very strong point in its favor :smile:

3 Likes
9

PR for this change is here: Add param for interpolation factor to ArcRotateCamera.interpolateTo by ryantrem · Pull Request #15923 · BabylonJS/Babylon.js · GitHub

2 Likes