Quest to create a Hologram with a JPEG! (how to? using Thin Instances, Matrix, Arrays)

I think there’s def a terminology difference that’s causing us to misunderstand each other. :slight_smile: I would describe that as 1 mesh with 1920x1920 thin instances and 1 NME project creating its shader… And a second mesh with 1920x1920 thin instances with a second NME project creating its shader.

And instead you can create 1 mesh with all 1920x1920x2 thin instances using just one shader created by one NME project, but in this case a new attribute(s) would be needed to control the animation so that the shader could animate one way for 1920x1920 of the thin instances and another way for the other 1920x1920 thin instances. At least, that’s the only practical way I can think of to do everything with 1 mesh and its thin instances…

I think the way to do that was how you had it before, where the color was stored directly in the color attribute rather than computing the UV’s and sampling the texture. But even then you still had to do that, it was just done in js instead of the shader…

I think if you have as many thin instances as texture pixels and you’re calculating the UVs correctly then the NEAREST sampling mode should give you the proper color value from the texture…

Oh okay! …so that’s what we’ve been doing with the structure of it…

Now the color mapping: Nearest fixed half fixed the issue!
I did it in here in NME:

But some of the cubes are half one colour, and half another color…
I’m wondering if I have not compensate a Image scale correctly somewhere in the NME…
Original method made this far simpler :cry:
These purple cubes for example:
image

Also Load time is a horrible wait, and Frame rate is still only 21 FPS (on my machine). I don’t know if that is good or bad for what we’re trying to achieve (and I still have more things to add to this :S )

Update PG: https://playground.babylonjs.com/#DNMA77#9

Update Color now working! woohoo.
PG: https://playground.babylonjs.com/#DNMA77#11

Load times/efficiency are still of concern. Would appreciate some experienced eyes over what/where i might improve!

1 Like

Aah, now that I’ve been able to look at the code more, I see the new PG is creating 25 times as many thin instances as the old one, making the FPS plunge (I didn’t realize you were using so many more now). :open_mouth:

console.log(PXLinstanceCount);
Old PG: 147,456 thin instances
New PG: 3,686,400 thin instances

I tried switching the new PG to use only 147,456 thin instances like the old one and the FPS is back up to 60, although it still takes awhile to load. I’m not sure if you’ll be able to get good FPS with 7,372,800 thin instances (total for both meshes) even by refining the NME shader (I don’t have experience with that kind of limit pushing).

Also I noticed that the NME pages for your new shaders take a long time to load as well, so I guess the problem is there with the shader material (with the texture loading?), not with the PG code, but IDK what the problem would be…

PS it was the “thin instances vars” section that I copied over from your old PG to test your new NME shaders with the thin instance count back down to 147,456 per mesh like it was with your old shaders.

But really you’re going farther than I’ve been, so I can’t help much here. :slight_smile:

1 Like

Aha yes - that version where you copied from was using much larger “pixels”, so less Thin Instances! (5 times less specifically :slight_smile: ) - which kinda kills the intent/effect I’m chasing…
@Blake - from where about did you take notes of those stats? Was it in somewhere in the Inspector?
Do I need to host the texture elsewhere to truly test it perhaps?
Maybe I’ll setup a test HTML page and see how it performs…

I wonder if there are any other optimizations I can be doing…
@PirateJC, any ideas? Ultimately trying to animate each of these 1920x1920 pixels… Thin Instance is the correct tool to be using to achieve this, right?

Not that I know very much about shaders and NME but I was wondering if rather than using thinInstances you created a new material instead.

This would be built using NME.

Check out the materials library

In particular lava might be useful (don’t be put off by the ground being named sphere rather than ground - just an uncorrected copy and paste)

Code for materials library found at

1 Like

Is a point cloud / particle system more efficient to use? Not that this is 3dimensional, but to give it a 3d effect?

Lava ? … ?

  • checking out the playground now …

Hi John, yeah this won’t work… I’m really trying to make a material the feels like a hologram:
Can be seen through to some degree;
Feels like floating objects (the pixels) when moving around it;
Slight glitch effect (the animation, though I’d like to improve its movement to feel more “glitchy”);
And for there to be a bit of bloom or glow to each pixel; so that the pixels can have black space around them to look through/beyond, but have a glow that when zoomed out, hides the gaps; much like a real screen where there is a black / void around each pixel.

Does that describe it better?

PS. Anyone with friends in the Halo team?? :wink:
This kind of thing would be amazing (noting they have a lens aberration effect happening too) >>>

1 Like

Sorry my idea didn’t work out. Keep trying I am sure that you will get there eventually. :smiley:

2 Likes

No thank you for the suggestion regardless!! I’m sure to return to the PG one day to understand that wavy motion!

2 Likes

I was just logging the variable PXLinstanceCount that controls how many thin instances are created for each mesh. I don’t think there’s a count in the Inspector, but you can also log the count for each mesh like below after you set their thin instance buffers. :slight_smile:

console.log(pixels.thinInstanceCount);
console.log(pixelsB.thinInstanceCount);

EDIT: PS, also it seems you’re using more thin instances than expected :point_down:

You’re actually using 25 times as many thin instances now, not just 5 times as many, because the grid of thin instances has 5 times as many rows and 5 times as many columns, so 5*5 times as many thin instances all together, which might just be too many for good frame rate on most devices (over 7 million all together). :slight_smile:

1 Like

aha! :+1:t5:

1 Like

@HirangaG
What’s up bro? I am chillin’ a bit now so what’s your progress?

1 Like

Roland! Good!! How are you!??

Sry I’m seeing this msg so late! I haven’t been here for a while as I have parked the bjs part of our project having reached a satisfactory point (except for efficiency/ frame rate) and am working on other parts of it right now.

Any thoughts on whether a particle system is more efficient than thin instances?
The number of instances I’m creating is 3686400 (1920 x1920) and then allocating pixel color from a jpeg to each TI; and then oscilating them to create a hologram-like effect.

Alternatively, I was wondering if particles are more efficient, and whether I can control particles to move only along one axis, like rain drops, and whether they can adopt the color of my jpeg (as per the previous setup). Would this me more efficient?

**trying to keep frames above 20fps

Any thoughts? I haven’t made time to test this hypothesis…

Hello!
The question was already excellently answered by @PirateJC here:

You have to use a shader to move the thin instances. Everything else will be slower. I get ~55 FPS with a 1000 x 1000 grid using PirateJC’s example, but only ~15 FPS with a 1920 x 1920 grid on a 3080Ti. There is only one draw call so it can’t be done better on the CPU side.

I’m thinking whether you could gain more FPS by optimizing the shader and replace the motion part with a precalculated Vertex Animation Texture but I am not sure what’s faster, texture lookups or the existing shader code generated for the motion by the NME. At least you could give it a try.

IMHO you are aiming too high and you have to came up with another idea for the effect you’d like to achieve.

:vulcan_salute:

r.

2 Likes

yeah ! that’s right - I’ve been letting it marinade in the background. I might shift approach when I get to the next round of attacking this… Thanks !

2 Likes