@gbz i am thinking that mmm maybe
not sure though, lets see if some experts can confirm this
@javismiles you may know some of this but lets take things step by step anyway.
The gradient of a line in 2D space is a single scalar number. A gradient in 3D space is a 2D vector.
To find the gradient of a 3D surface at a point you take the plane tangent to the surface at that point, find the line of greatest slope on the tangent plane, along which lie 3D vectors and you find the gradient for that vector.
You need to remember that in Babylon.js the y axis is vertical and the x and z axes horizontal.
Take vector G to be on the line of greatest slope on a plane P, then there exists a vector R = (xR, yR, 0) and a vector S = (0, yS, zS) such that G = R + S
Now the 2D vector Rxy = (xR, yR) is in the XY plane and has a gradient gR in that plane. The 2D vector Syz = (yS, zS) is in the YZ plane and has a gradient gS in that plane.
The gradient of G is given by the vector (gR, gS)
For a surface given by the function y = f(x, z) you find the gradient at a point using partial derivatives.
As you said in Babylon.js the surface of a mesh is represented by a network of triangles. Take one of these triangles, once you have the equation for the plane the triangle lies in you can find the gradient of the plane. Since it is a plane the gradient at any point within the triangle will be a constant 2D vector.
The equation of any plane has the form ax + by + cz + d = 0; its gradient is given by (∂y / ∂x , ∂y / ∂z ) as y is vertical
Taking partial derivative w.r.t x gives a + b * ∂y / ∂x = 0 and so
∂y / ∂x = - a / b
Taking partial derivative w.r.t z gives b * ∂y / ∂z + c = 0 and so
∂y / ∂z = - c / b
Given the equation of a plane as ax + by + cz + d = 0 its gradient is (-a / b, -c / b)
Now all you need to do is find a, b, c, and d.
Let P be the position vector of a point on the plane, and N the normal vector of the plane then
const plane = BABYLON.Plane.FromPositionAndNormal(P, N)
will construct the plane and from this you can use
plane.asArray(); // to give you [a, b, c, d]
and the equation of the plane is ax + by +cz + d = 0.
and you can now find the gradient of the plane which is the gradient at point P.
As of yet I have not tested this out in practice.
John this is a fantastic explanation, thank you very much
for your help and really again this is the best community of
this kind that i have ever seen, stunning
so the theory seems quite clear now, lets see if i understand
how to actually implement this
user clicks on a point in the model
I can then get that point:
var pickInfo = scene.pick(scene.pointerX, scene.pointerY);
and the normal there:
const normal=pickInfo.getNormal();
If I understand well, because that point is of course part of the tangent plane that includes it, I could calculate that plane doing:
const plane = BABYLON.Plane.FromPositionAndNormal(P, N)
then doing plane.asArray() I can get the a,b,c,d from there, which provides the equation for the plane
in theory then the gradient at that point would be given as you calculated by:
(-a / b, -c / b), which would be a 2d direction within the plane set in the 3d space
Now, if I wanted to provide the user with 2 things:
a) the actual direction of steepest ascent or descent within the 3d space, then I would need to actually provide a 3d vector right? how would I get that?
b) most importantly, and the final objective, if I wanted to actually visualize for the user an arrow that pointed in the direction of steepest ascent or descent, how would I do it, given that I am getting again a 2d vector but to represent that arrow in 3d space i need a 3d vector right?
thank you again for your help 
@JohnK
so yes I can get that data following your advice and doing:
const normal=pickInfo.getNormal();
const plane = Plane.FromPositionAndNormal(pickInfo.pickedPoint, normal);
const Pdata= plane.asArray();
const Gradient = new Vector2(Pdata[0]/Pdata[1],-Pdata[2]/Pdata[3]);
console.log(Gradient);
so how could I now take that and if I wanted to actually visualize for the user an arrow that pointed in the direction of steepest ascent or descent, how would I do it, given that I am getting that 2d vector but to represent that arrow in 3d space i would need i guess to do something else extra
I guess i need to basically put an arrow from the selected point in the direction of the gradient, which means taking the point in 3D and adding to it a 3D vector that points in the direction of steepest ascent (or descent, whichever I need), and then using something like this arrow helper of three.js three.js docs , not sure if babylon.js has something similar, or how would you do it and how do I go from that 2d direction in the 2d plane in 3d space to the actual 3d direction in 3d space? thank you
In 2D a vector along a line of gradient say 2/3 will be a scalar multiple of (1, 2/3) or of (3, 2).
In 3D let a vector lie along a line of gradient (p, q). Form a cuboid with one corner at (0, 0, 0) and a diagonal parallel to the line with its length along the x axis of 1. Let the height of the cuboid be y and depth z. Then
y / 1 = p so y = p and y / z = q so p / z = q so z = p / q
Hence a vector along the line is (1, p, p / q) or multiplying by q (q, pq, p) this will be a vector you can then draw at a point P.
EDIT Maths incorrect
@JohnK thank you very much again and thank you for having patience with me
let’s see, so do you mean that
a line that pointed in the direction of the gradient, in the direction of steepest ascent, would be a line
that would start from the initial point, from “pickInfo.pickedPoint”
and given that my gradient on the plane is
const Gradient = new Vector2(Pdata[0]/Pdata[1],-Pdata[2]/Pdata[3]);
with
p=Pdata[0]/Pdata[1],
q=-Pdata[2]/Pdata[3]
then a vector in 3d space that was the direction in which the line should point towards would be composed of (1,p,p/q) or (q, p*q, p) ?
so a line going from “pickInfo.pickedPoint”
to (1,p,p/q) or (q, p*q, p)
being
p=Pdata[0]/Pdata[1],
q=-Pdata[2]/Pdata[3]
would be the correct line to draw for a line that went from the initial point in the direction of steepest ascent?
and if so, what if I wanted the direction of steepest descent instead? which I also need it
I wonder if this is going in the right direction:
const normal=pickInfo.getNormal();
const plane = Plane.FromPositionAndNormal(pickInfo.pickedPoint, normal);
const Pdata= plane.asArray();
const gp=Pdata[0]/Pdata[1];
const gq=-Pdata[2]/Pdata[3];
const Gradient = new Vector2(gp,gq);
const direction = new Vector3(1,gp, gp/gq);
const rx = Math.acos(direction[0]);
const ry = Math.acos(direction[1]);
const rz = Math.acos(direction[2]);
myArrow.position = pickInfo.pickedPoint;
myArrow.rotation = new Vector3(rx, ry, rz);
this doesnt work as the direction values are not between -1 and 1 so i cannot do the acos for example, there must be something wrong still
@JohnK
normalizing better, but still something missing
const normal=pickInfo.getNormal();
const plane = Plane.FromPositionAndNormal(pickInfo.pickedPoint, normal);
const Pdata= plane.asArray();
const gp=Pdata[0]/Pdata[1];
const gq=-Pdata[2]/Pdata[3];
const Gradient = new Vector2(gp,gq);
const dir = new Vector3(1,gp, gp/gq);
//to (1,p,p/q) or (q, p*q, p)
const dirmag= Math.sqrt(dir._x * dir._x + dir._y * dir._y+ dir._z * dir._z);
const rx = Math.acos(dir._x/dirmag);
const ry = Math.acos(dir._y/dirmag);
const rz = Math.acos(dir._z/dirmag);
myArrow.position = pickInfo.pickedPoint;
myArrow.rotation = new Vector3(rx, ry, rz);
Sorry do not have time to have a detailed look until tomorrow or Sunday. Did note that your gp has no negative sign and both gp and gq should have the same denominator.
@JohnK oops you are right just corrected that part:
const normal=pickInfo.getNormal();
const plane = Plane.FromPositionAndNormal(pickInfo.pickedPoint, normal);
const Pdata= plane.asArray();
// (-a / b, -c / b)
const gp=-Pdata[0]/Pdata[1];
const gq=-Pdata[2]/Pdata[1];
const Gradient = new Vector2(gp,gq);
const dir = new Vector3(1,gp, gp/gq);
//to (1,p,p/q) or (q, p*q, p)
const dirmag= Math.sqrt(dir._x * dir._x + dir._y * dir._y+ dir._z * dir._z);
const rx = Math.acos(dir._x/dirmag);
const ry = Math.acos(dir._y/dirmag);
const rz = Math.acos(dir._z/dirmag);
myArrow.position = pickInfo.pickedPoint;
myArrow.rotation = new Vector3(rx, ry, rz);
I think this is very close now, its behaving better i think, probably still something missing though mmm yes whenever you have time when you have it I appreciate any help, thank you
@JohnK interesting and confusing, im testing it and many times it seems to behave quite good, interestingly it points in the direction of steepest descent (which is actually what i want) instead of steepest ascent, thats curious given that i think you gave the calculations to point to the steepest ascent, not descent, so that confuses me
sometimes the arrow gets too inside the mesh so i dont know if this is a good idea but basically i push the position of the arrow a bit in the direction of the normal of the point where it is:
myArrow.position._x+=normal._x *5;
myArrow.position._y+=normal._y *5;
myArrow.position._z+=normal._z *5;
to get it further from the mesh so that it can be seen, not sure if this is the best idea in regards to that
and then there are times when the arrow just doesnt seem to point towards the right direction
but in general much better than before
I am also calculating the magnitude of the gradient with:
const Grad = new Vector2(gp,gq);
const gradmag= Math.sqrt(Grad.x * Grad.x + Grad.y * Grad.y);
and yes I can see it get much smaller when things are flatter and much bigger then steepness increases a lot which is a great sign
lets see when you have time if you can still find problems issues in the equations above maybe there are still issues to solve , thank you again this is amazing help
Looks like I have made an error in either doing the maths or understanding the maths. Will look into it and get it right.
@JohnK thank you very much John, looking forward to your new calculations I think it’s close to the correct thing, as the arrow is behaving quite close to the objective although not perfectly yet
Given gradient (p, q) a vector along line of steepest ascent is (p, p2 + q2, q )
PG for plane with random normal Babylon.js Playground you may have to move camera around to see the plane. For a plane parallel to xy, xz, xy zero gradient so no gradient line produced as there is no gradient line.
PG for icosphere https://www.babylonjs-playground.com/#S2IDIY#1
PGs created by cutting and pasting from existing PGs you may want to tidy up code and format.
@JohnK thank you very much John
,so if (p, p2 + q2, q ) is steepest ascent, which would be the steepest descent (which is the one i need), would it be just the opposite I guess: (-p, -(p2 + q2), -q )
and so, if I have:
const gp=-Pdata[0]/Pdata[1];
const gq=-Pdata[2]/Pdata[1];
const Gradient = new Vector2(gp,gq);
then the direction of steepest descent would be:
(-gp, -(gp^2 + gq^2), -gq)
correct?
if I want to set an arrow pointing in that direction, would this be correct to transform that 3d direction gradient vector to rotation angles of arrow?
const dir = new Vector3(-gp, -(gp x gp + gq x gq), -gq) // setting multiplication with ‘x’ doesnt let me put ‘*’
const magnitudeGradient= Math.sqrt(dir._x * dir._x + dir._y * dir._y+ dir._z * dir._z);
const rx = Math.acos(dir._x/magnitudeGradient);
const ry = Math.acos(dir._y/magnitudeGradient);
const rz = Math.acos(dir._z/magnitudeGradient);
myArrow.position = pickInfo.pickedPoint;
myArrow.rotation = new Vector3(rx, ry, rz);
thank you again
@JohnK thank you very much again John, your change in equation definitely seems to make things behave better yeah,
and about rotation I’m going to investigate the " [Target Axes Alignment", although it seems that the
const rx = Math.acos(dir._x/magnitudeGradient);
const ry = Math.acos(dir._y/magnitudeGradient);
const rz = Math.acos(dir._z/magnitudeGradient);
at least it seems that it orients the arrow quite nicely
what it is really puzzling to me is that using
const dir = new Vector3(gp,(gpxgp+gqxgq), gq);
and
const dir = new Vector3(-gp,-(gpxgp+gqxgq), -gq);
both in combination with arrow rotation make the arrow point nicely towards the descent, so odd, both seem to produce similar result, but in any case it is what i need
the other thing i wanted to ask you, if I want to display a number with the magnitude of the gradient strength, which one of these would be the correct one to use?
const gp=-Pdata[0]/Pdata[1];
const gq=-Pdata[2]/Pdata[1];
const Grad = new Vector2(gp,gq);
const dir = new Vector3(-gp,-(gpxgp+gqxgq), -gq);
const gradmag1= Math.sqrt(Grad.x * Grad.x + Grad.y * Grad.y);
const gradmag2= Math.sqrt(dir.x * dir.x + dir.y * dir.y+ dir.z * dir.z);
is the correct one gradmag1 or gradmag2 to understand the strength, magnitude of the gradient?
thank you
@JohnK I tried to use
const orientation = Vector3.RotationFromAxis(-gp,-(gpgp+gqgq), -gq);
myArrow.rotation = orientation;
but I get error apparently because the parameters of RotationFromAxis should be: " where axis1 , axis2 and axis3 are three left-handed orthogonal vectors and the mesh will be aligned with"
how could I transform that gradient vector to 3 left-handed orthogonal vectors?
thank you
I would go with
const gradmag1= Math.sqrt(Grad.x * Grad.x + Grad.y * Grad.y);
This is equivalent to
const gradmag1= Grad.length();
The normal, N, the gradient vector G = (p, p2 + q2, q ) and (N cross G) form 3 orthogonal vectors. To get these to match you may have to play around with using (G cross N) rather than (N cross G) direction of one in in opposite to the other and setting y = N then play with x = G or z = G
cross product Babylon.js docs
A way of drawing an arrow in right place and direction plus scalable
Sry for disturbance. May I just say that I’m now eager to see the end result of all this. Will you share it? 
You certainly raised my interest with your posts…