Thin instance picking issues & picking with world matrix issues

Questions
1

I’m facing issues both with thin instance picking and when passing a world matrix to Ray.intersectsMesh.

Here is my repro for the latter: https://playground.babylonjs.com/#L1TSN0#8

There are number of visual issues with my repro (but the console output is sufficient):
- Cant see my rayLine visualization
- My mesh faces are the wrong way round, and specifying object.sideOrientation hasn’t helped

In this repro I don’t use thin instances. When TEST_BOOL is false I try to call Ray.intersectsMesh with a custom world matrix, and it fails. However if TEST_BOOL is true then I set my object mesh’s world matrix and then pick without specifying a world matrix, and it succeeds.

In my real project I use thin instances with the same mesh verts, indices, and world matrix, and they display properly. Similar to the simplified example of passing a world matrix to intersectsMesh, I get very unreliable results (false positives and false negatives) when using scene.multiPickWithRay when some of the meshes have thin instances. I’ve tried syncing bounding info, enabling thin instance picking, freezing / unfreezing world matrices etc etc. My current workaround is to pick thin instances individually by transforming the ray myself, calling Ray.intersectsMesh, and transforming the result back, and this isn’t very fast.

My regular meshes work properly with scene.multiPickWithRay. I’ve verified the bounding box encompassing all thin instances is shaped and positioned correctly. My only issues lie with picking. The base mesh (and its ancestors) all have identity world matrices.

Has anyone faced similar issues with either of the problems I’ve described?

2

About the line, you can not sum and multiply vector like this in js :slight_smile: you need to use the appropriate functions https://playground.babylonjs.com/#L1TSN0#5

sideOrientation should be set with the correct constants: https://playground.babylonjs.com/#L1TSN0#6

You can not set _worldMatrix this way as it will be overriden and _x properties are private and should not be manually set. I am not sure of the issue but a playground reproducing it would be great.

2 Likes
3

Thank you for helping me fix the visual errors with my picking repro. I’ve updated the link in my post.

Is setting _worldMatrix in this way only bad if the position/rotation/scale of the mesh changes? If I do .freezeWorldMatrix() afterwards then is it fine even if the transform changes?

However this isn’t relevant to the issue I hope to have reproduced here. The line is green, indicating picking success, when I do set _worldMatrix. The line is red, indicating that the object has not been picked, when I pass the worldMatrix as the worldToUse argument of Ray.intersectsMesh.

Is it possible I am misunderstanding what worldToUse does, and it is only used to transform the result and not the mesh used in the intersection test?

4

This is an internal property you should normally not change manually that said if it is frozen it could be ok-ish.

The line is green when TEST_BOOL is true independently of the rest of the setup as you can see here: https://playground.babylonjs.com/#L1TSN0#9 cause the rendering matrix is the same as the one used in the intersection.

When TEST_BOOL is false you where changing the rendering after the intersection test. Changing that back would show you why it is red: https://playground.babylonjs.com/#L1TSN0#10