I am working on a gesture-based AR toolkit for mobile and have been trouble figuring out the best way of selecting everything within a drawn on screen circle(ish) shape between the terrain, and camera.
Ideally I would love to draw a custom shape that connects terrain to camera view plane and hit-test everything inside on the ground and in the air.
I have gotten as far as to raycast from the traced points (about 8-20 after a simplification algorithm which works on the 2D points to avoid reducing raycast precision). Each returned PickPoint returns a Vec3 for colliding with the terrain.
Points are returning but, I am having a hard time getting any shape to extrude from my points. If I see anything, its polygon soup thrown all over the walls of my room (literally), and 99% of the time nothing seems to want to render. If I make a line object out of the points, some are seemingly returning in air. Notice the faint red line of points here @ 7 seconds when I draw the shape. It is not touching the terrain (strange) but does confirm the hit points are in the generally correct area.
What do I need to be looking out for as to why my rays are mid-air, is there a precion setting or something? More importantly for the big picture, extruding up the Y sucks as it will only allow selection of very near ground units. What is the best strategy to not simply extrude “up”, but extrude out toward the camera to capture just what is in view of that 2D shape when it is drawn?