How to create an Image(from URL) on a HUD / Screen / Holographic effect

[HirangaG]

Hi guys, @PirateJC , I’m trying to create what is probably a very simple webapp that feels like a spherical/helmet HUD, that looks at a holographic screen.
*not really a game, but has some interactiveness

Detailed Intent:
1.0 Holographic Screen is:
1.1) flat (or slightly spherical?) plane in a fixed position;
1.2) with an updateable Static Image/Jpeg loaded from an External URL;
1.3) might have holographic “floating” objects offset from the plane that help create depth/parallax effect;
1.3.1) holographic HUD objects that might move
1.3.2) holographic HUD Text that might be updated from an external source;
1.3.2.1) Text might be Clickable for interacting with it???

2.0 Camera:
will sit close to the screen, pointing at it;
but needs to be constrained to:
2.1) pan, parallel to the plane;
2.2) roll/pitch to look around the screen, but:
2.3) not look past the extents of the plane (the screen and image should always fill the view)
2.4) limited zoom in/out (to prevent getting too close, or too far out/ obey constraint 2.3

I’m fairly new to all of this, but have setup a camera and Plain in Babylon haha and using the above as my plane map for constructing this thing.

Going to eat my elephant one bite at a time so, greatly appreciate any pointers along the way!
…my first question:
I’m up to step 1.2: how do you apply an image from URL onto a plane?
1.1: Is it possible to make a 3d curved (spherical) plane?

[HirangaG]

Progress: dont know if this is the right way of doing things. But I found Decals! Can I give the decal luminosity/illuminate like a computer screen, rather than shining a light onto the surface?
*I want to avoid glare
**it would also be cool to give it a texture like looking at RGB pixels when zoomed in #scopecreep

I also need to slow the camera down…

[carolhmj]

You can set any material for the decals, so you can set an emissiveTexture, for example
Drawing HTMLCanvasElement as Decal | Babylon.js Playground (babylonjs.com)

Most cameras have a speed parameter: TargetCamera | Babylon.js Documentation (babylonjs.com)

[HirangaG]

Amazing! The emission is what I’m after!! And the cat example is kinda useful! I’m trying to do another thing, that intersects with the plane of the screen…

@carolhmj is there a way to get the camera to follow what you’re looking at, intersecting with the screen?
I’m trying to get the effect of standing close to the front of a wide rectangular screen, such that all of it is not visible, but when rotating the camera to “look”, instead of camera pivoting on one origin, it follows the focus point (that intersects with the screen) and the camera effectively pans to center on that new location on the screen, while not being perfectly perpendicular to it. ie. maintains the perpendicular offset from the screen but strafes/pans with a slight tilt/angle off the focus point (not perfectly perpendicular). I’m not sure I’ve explained that very well, but I’m trying to simulate a natural effect, say for example, of looking around on a very large map.

Plain explanation(?): camera, points while pivoting on origin > to new focus position intersecting with screen plane;
camera then moves, with slight delay to get closer/almost perpendicular to new focus position (that is sitting on the plane). Any ideas/PGs i can reference?

Also is Universal Camera the wrong one to use in this scenario? Replace with Target Camera?

[brunobg]

If you need something fancier than decals, look into the material plugins: Material Plugins | Babylon.js Documentation

I wrote a caustic effect using them, by multiplying the shader result by the caustic texture. Used the same thing to do an underwater fog effect.

[JohnK]

@HirangaG from reading your current posts you have gone way beyond these questions, however a few days ago I started some answers to your questions and then the internet went down in our area and when it came back on I got distracted with other issues.

Playground (PG) Examples

This first PG demonstrates the answers described below with the camera set at a distance from a small sphere slice

Babylon.js Playground

This second PG enlarges the sphere slice and moves the camera relatively closer.

Babylon.js Playground

Answers

The following answers neither fully answer your questions nor answer all the questions but hopefully will give you some insight into ways you might progress and might be helpful to others reading your post.

@HirangaG

1.1) flat (or slightly spherical?) plane in a fixed position;

You can achieve slightly spherical by creating a slice of a sphere (an option using CreateSphere but only horizontal available) flattening the sphere by scaling in vertical direction (perpendicular to slice) and rotating the sphere to a vertical.

Creating A Sphere | Babylon.js Documentation

@HirangaG

1.2) with an updateable Static Image/Jpeg loaded from an External URL;

OK provided image is on a CORS compliant site and apply as material to mesh.

Introduction To Materials | Babylon.js Documentation

@HirangaG

2.1) pan, parallel to the plane;
2.2) roll/pitch to look around the screen, but:

ArcRotateCamera can roll/pitch (well rotate around a target) using CTRL + mouse can pan

@HirangaG

2.3) not look past the extents of the plane (the screen and image should always fill the view)

Can set limits on the rotation values alpha and beta of the ArcRotateCamera

@HirangaG

2.3) not look past the extents of the plane (the screen and image should always fill the view)

Set limits on the panning distance

https://doc.babylonjs.com/typedoc/classes/babylon.arcrotatecamera

Map image across Slice

The other thing you might want to know about is this bit

//map uvs as onto a plane not a sphere
const positions = sphere.getVerticesData(BABYLON.VertexBuffer.PositionKind);
const uvs = [];

const numberOfVertices = positions.length / 3;	
   
for(var i = 0; i < numberOfVertices; i++) {
    uvs[2 * i] = (positions[3 * i] - radius) / (2 * radius);
    uvs[2 * i + 1] = 2 * radius - (positions[3 * i + 2] - radius) / (2 * radius);
}
	
sphere.updateVerticesData(BABYLON.VertexBuffer.UVKind, uvs);

Each vertex of a mesh has uv coordinates that indicate where on the applied image that vertex picks up the image data. The uv range is (0, 0) bottom left corner to (1, 1) upper right corner.

When the sphere is created there is a particular mapping for the uvs that wraps the rectangular image around the sphere, top to one pole, bottom to the other pole. The above code overwrites that mapping and replaces it with one that maps the rectangular image as a rectangle across the sphere splice.

More Materials | Babylon.js Documentation

Updating Vertices | Babylon.js Documentation

[HirangaG]

John! Thank you so much for this detailed answer! After a bit more time in the forum, I’ve realised I might need to eat my elephant in even smaller bites given my current skillset haha!
Thus parked these questions with a simpler camera setup, until completing my current chunk here: Quest to create a Hologram with a JPEG! (how to? using Thin Instances, Matrix, Arrays) - #67 by HirangaG

I think I might keep the screen flat for now, but need to return back to the camera setup! On the above thread, I’ve managed to create to cool pixels, but now suffering the moire effect of such tiny pieces haha I’m about to attack that part now!

I’m wondering if adding some bloom might help, or if I should just bring the pixels closer together…

MOIRE EFFECT ISSUE CONTINUED HERE: Quest to create a Hologram with a JPEG! (how to? using Thin Instances, Matrix, Arrays) - #71 by HirangaG