Hello,
I’m trying to achieve picking using a scene that has clipping planes.
I need to access the actual 3D location of the clicked point visible in the viewport, the picking will not take clipping planes into account.
Any suggestion would be most welcomed.
Did a small playground to illustrate by problem
https://playground.babylonjs.com/#Y6W087#28
Thanks
so clipping planes are a GPU only operation but I totally get your issue. This is not something we support at the moment. Would you be willing to contribute it ?
Well, I have been using babylonJS for a while now, but I never took the time to look into the source code. I really do need this feature though, so I’m going to try to find a way.
If it can benefit to other, that even better.
But your reply implies that picking is not done on GPU as well. I wasn’t expecting that.
I’m very familliar with OpenGL (I have my own engine but it’s not web compatible) and I implemented picking on the GPU. Is there a WebGL limitation that forces the use of CPU ray/triangle collision to implement picking ?
not sure there was any restriction, but just the way it has been done 
If you can not do it and so as I consider it more as a bug I can volunteer @PirateJC to have a look into it as he is extremely keen on contributing code more and more
Thanks.
I tried to use the optional parameter of scene.pick
“trianglePredicate?: BABYLON.TrianglePickingPredicate”
But the documentation is not quite clear on what that does.
My idea was to reject triangles that is at least partiarly clipped. But I can get if to work.
I’m cloning Babylon git right now, i’ll try to figure out something.
About trianglePredicate you could then depending on the 3 vertices of each triangle force reject rays which would intersects by returning false inside of it like any triangles outside the clip planes would return false in your case and true for the others (except the edge case where vertices are half in half out where the intersection points should decide of the rejection)
Got it from reading the source code. The points and ray in the predicate are not provided in world space but in mesh space, while clipping planes are in world space.
But we have no idea which object is beeing tested, so it’s not possible in the current state to perform any matrix computation to have them in the same space.
Yes, that’s what i’m trying to do.
OK, I found a “hack/solution”. Fell free to use it to find a more permanent solution in BabalonJS
I overrided Ray.TransformToRef to store the matrix of the ray space and it’s inverse (four last lines)
BABYLON.Ray.TransformToRef = (ray: BABYLON.DeepImmutable<BABYLON.Ray>, matrix:
BABYLON.DeepImmutable<BABYLON.Matrix>, result: BABYLON.Ray) => {
BABYLON.Vector3.TransformCoordinatesToRef(ray.origin, matrix, result.origin);
BABYLON.Vector3.TransformNormalToRef(ray.direction, matrix, result.direction);
result.length = ray.length;
var dir = result.direction;
var len = dir.length();
if (!(len === 0 || len === 1)) {
var num = 1.0 / len;
dir.x *= num;
dir.y *= num;
dir.z *= num;
result.length *= len;
}
if (!ray["matrix"]) { ray["matrix"] = BABYLON.Matrix.Identity(); ray["inv_matrix"] = BABYLON.Matrix.Identity(); }
result["matrix"] = ray["matrix"].multiply(matrix);
result["inv_matrix"] = BABYLON.Matrix.Identity();
result["matrix"].invertToRef(result["inv_matrix"]);
}
Then I used
let pickingRay = scene.createPickingRay( X, Y, null, camera);
let picking = scene.pickWithRay(pickingRay, null, false, (p0, p1, p2, ray) => {
let m: BABYLON.Matrix = ray["inv_matrix"];
if (m) {
let wp0 = BABYLON.Vector3.TransformCoordinates(p0, m);
let wp1 = BABYLON.Vector3.TransformCoordinates(p1, m);
let wp2 = BABYLON.Vector3.TransformCoordinates(p2, m);
let not_clipped = true;
if (scene.clipPlane) {
not_clipped = not_clipped
&& scene.clipPlane.signedDistanceTo(wp0) < 0
&& scene.clipPlane.signedDistanceTo(wp1) < 0
&& scene.clipPlane.signedDistanceTo(wp2) < 0;
}
return not_clipped;
}
else
return true;
});
Note : it does not work with a simple “scene.pick”, the TransformToRef is never called, I guess the ray managment is done diffently for some unknown reason.
Note 2 : I don’t manage “on the fence” triangles. I can only collide with triangles fully visible. That’s ok in my current application, but it’s a limitation.
As far as I’m concerned, it’s ok for now.
Maybe I’ll remove this “hack” if babylon fix this bug.
M.
What do you call a bug here?
That clipping is ignored by picking, I’m not the one calling a bug in the first place 
Sebavan said : “I consider it more as a bug”
lol no worry:) I wanted to make sure there were not a bigger one
I would not call it bug per se but more a potential improvement ;D
OK
Another efficient approach would be to provide the mesh in the trianglePredicate callback. Not a big modification I think, but I would work for me.
I found another approach which does not require to override any methods
I use the meshPredicate to store the current mesh matrix in a local variable, the use it in the trianglePredicate.
I think it’s a bit cleaner.
let currentMeshMatrix: BABYLON.Matrix = null;
let pickingRay = scene.createPickingRay(event.offsetX, event.offsetY, null, camera);
let picking = scene.pickWithRay(pickingRay,
// MESH PREDICATE
(mesh) => {
if (mesh.isPickable) {
currentMeshMatrix = mesh.computeWorldMatrix();
}
return mesh.isPickable;
},
false,
// TRIANGLE PREDICATE
(p0, p1, p2, ray) => {
let m: BABYLON.Matrix = currentMeshMatrix;
if (m) {
let wp0 = BABYLON.Vector3.TransformCoordinates(p0, m);
let wp1 = BABYLON.Vector3.TransformCoordinates(p1, m);
let wp2 = BABYLON.Vector3.TransformCoordinates(p2, m);
let not_clipped = true;
if (scene.clipPlane) {
not_clipped = not_clipped
&& scene.clipPlane.signedDistanceTo(wp0) < 0
&& scene.clipPlane.signedDistanceTo(wp1) < 0
&& scene.clipPlane.signedDistanceTo(wp2) < 0;
}
return not_clipped;
}
else
return p0.y < currentYClipping && p1.y < currentYClipping && p2.y < currentYClipping;
});I like that one! good job!
code worked!
let planes: Plane[] = [];
if (this.scene.clipPlane) planes.push(this.scene.clipPlane);
if (this.scene.clipPlane2) planes.push(this.scene.clipPlane2);
if (this.scene.clipPlane3) planes.push(this.scene.clipPlane3);
if (this.scene.clipPlane4) planes.push(this.scene.clipPlane4);
if (this.scene.clipPlane5) planes.push(this.scene.clipPlane5);
if (this.scene.clipPlane6) planes.push(this.scene.clipPlane6);
let world: Matrix = null;
var pickResult = this.scene.pick(this.scene.pointerX, this.scene.pointerY,
(mesh) => {
if (mesh.isPickable) {
world = mesh.computeWorldMatrix();
}
return mesh.isPickable;
},
true,
null,
// TRIANGLE PREDICATE
(p0, p1, p2, ray, colors) => {
// fully transparent
if (colors[0] < 0.00001)
return false;
if (planes.length == 0)
return true;
var intersectInfo = ray.intersectsTriangle(p0, p1, p2);
if (!intersectInfo)
return false;
// Get picked point
const worldOrigin = new Vector3;
const direction = new Vector3;
Vector3.TransformCoordinatesToRef(ray.origin, world, worldOrigin);
ray.direction.scaleToRef(intersectInfo.distance, direction);
const worldDirection = Vector3.TransformNormal(direction, world);
const pickedPoint = worldDirection.addInPlace(worldOrigin);
for (var plane of planes) {
if (plane.signedDistanceTo(pickedPoint) > 0)
return false;
}
return true;
}
);