I used Math.floor() on the X and Z coordinates and the numbers are whole when I draw the lines vertically or horizontally. If I draw the lines diagonally the numbers are not whole.
const x = pickResult.pickedPoint.x,
z = pickResult.pickedPoint.z,
pickedPoint = new BABYLON.Vector3(
Math.floor(x),
0,
Math.floor(z)
);
I ended up rounding down the output of the distance between the 2 vectors.
Its not the most accurate solution but for now it will work. Because the distance between the two vectors is still a float on the actual world coordinates.
It seems like Math.round() is a better solution, but it is not! In some cases it will NOT round correctly. Also, toFixed() will NOT round correctly in some cases.
To correct the rounding problem with the previous Math.round() and toFixed(), you can define a custom JavaScript round function that performs a “nearly equal” test to determine whether a fractional value is sufficiently close to a midpoint value to be subject to midpoint rounding. The following function return the value of the given number rounded to the nearest integer accurately.