Is there a way of taking an image texture and mapping each pixel colour from the image to a “sea” array? of cubes, to look like a screen, and then offset the individual cubes in a sort of ripple or refresh type of movement pattern perpendicular to the screen – like a hologram?
Where there is a will there is a way 
You may use texture.readPixels - Babylon.js docs
Thin instances ripple - https://playground.babylonjs.com/#TWQZAU#1
See also video about it
https://youtu.be/0DaSvI_n7Us
3 Likes
Omg. this sounds promising! I am excited for my tomorrow to try these methods out. Thanks for the lead!
1 Like
Hey Labris. I’ve paused on my other challenges and come back to trying to figure out texture.readPixels.
Conceptually I think I know what to do:
collect all the RGB values from the JPEG in particular order into a matrix?
Then map the RGB colour as a material? to each thin instance object, via the matrix?
… I’ve just watched David’s tute: Faster Scenes, Smaller Graphs with Thin Instances - YouTube
and Jason’s tute: Fun with Legos Part 1: Thin Instances - YouTube
I successfully mapped the Tinstances with Jason’s code, but in trying to implement color, David’s seems to have the logic in his Line 29 …BUT…I’m confused between the difference in syntax, and how to implement…
I’ve just copied David’s code and currently stuck with this error in VSCODE:
Any idea what I’m doing wrong? I’ve made a mockup in PG
On mobile now so just couple of remarks:
Start with 8*8 texture to simplify all things.
Thin instance can have only the same material as mesh from which it was created.
The color of material is defined by diffuseColor (albedoColor for PBR material) or/and emissiveColor.
The error which you’ve got is related to the attempt to change unchangeable propeties. Thin instances should be used when you need a lot of static instances that you know won’t change often / at all. So, regular instances may still be the way to go, depending on your scene: if you must add/remove instances continuously, it may be better to use instances than thin instances.
1 Like
Thx Labris… So with my intent, I want the TInstances to animate/move in the Depth Axis, similar to Jason’s waves (next task); and I’d like to colour the TInstances like in David’s PG example, where (from what I understand) he maps a value to R, G, B and A, according to the matrix … so I’m guessing it might be possible to map Matrix of TInstances with a Matrix of Color Values extracted using texture.readPixels ??? That viable with Thin Instances right?
But in order to test this, my error has me stumped… I think i get what you’re saying - have I used a Constant or something somewhere, erroneously?? I copied this chunk of code from Davids example and renamed the objects…
I’ve got both David’s and Jason’s code in the PG, commenting one out to test/understand the problem…
Jason, any ideas? @PirateJC
I’ve just realised the error I’m having (snippy above) by copying Davids code may have to do with me trying to code in TS, instead of JS … 
Is this the problem causing the Matrix to only permit reading error???
Do i need completely different syntax here?
Babylon.js Playground <<< this code only works in the JS Playground. Whereas I am building in VSC in Typescript …
Here is the working TS example - https://playground.babylonjs.com/#HFN6RP#1
If you use TS be ready to add “any” as often as needed to make it work
(of course, one may use more strict definitions)
var m:any = BABYLON.Matrix.Identity();
2 Likes
Oh man. amazing… “m:any” fixed it!
I had another bug in one of my variables, resulting in a Null value 
Finally located that by checking through the original side by side. God it took so long to notice that null resulting equation… It has taken me 16hrs haha Slow progress… but that part is now working! woohoo!
1 Like
Labris, any tips on how I might use the readPixels function?
I need to get better at understanding the Documentation (right now I don’t really get it…). I tried the following to; attempting to read pixels from the image loaded at L-150. I’m obviously doing it wrong. ???
And once I get those RBGA values do I need to format them into an array? or another matrix?
I need to pass the results matrix? per index into this location (highlighted) I presume? 
??
Actually now I think you don’t need to readPixels function since you use just plain image.
Put this image onto 2D canvas (for example, as dynamic texture).
Then use something like (this is just quick example from the web, so you’'ll need to change some things according to your setup)
function getPixel(url, x, y) {
var img = new Image();
img.src = url;
var canvas = document.createElement('canvas');
var context = canvas.getContext('2d');
context.drawImage(img, 0, 0);
return context.getImageData(x, y, 1, 1).data;
}
getPixel('./bg.png', 10, 10); // [255, 255, 255, 0];
and cycle it across all your image coords to get a value for each image pixel.
2 Likes
Oh I think i get it! It seems I’m only getting an array with 4 values though when i console logged…
Here’s what I did:
Variables:
Image/Pixel functions:
Mapping it to the Thin Instance:
As I mentioned, my example was just copy-paste from somewhere. You have already canvas element with Babylon scene. So you dont’ need to use in your function
var canvas = document.createElement('canvas');
Instead of this use dynamic texture for your 2d canvas where you put an image - Dynamic Textures | Babylon.js Documentation
Then get context
var ctx = myDynamicTexture.getContext();
and you are almost there
Here are more links:
How to get Pixel from HTML Canvas ? - GeeksforGeeks.
Pixel manipulation with canvas - Web APIs | MDN
3 Likes
Here is the example - https://playground.babylonjs.com/#5ZCGRM#2222 and the desired Uint8ClampedArray filled with values
1 Like
Thx… I’m piecing this together … and reading a whole lot of stuff to fix errors…
1 Like
This is really visible progress which may lead to the final success
1 Like
hahaha 
*needless to say i feel so stupid haha. Update: that “:any” thing worked once again sort of
My array returns correctly in ConsLog #1;
and returns only 0’s in ConsLog #2.
I’ve noticed the original is a Float32Array:
How do i get the ConsLog #1 data out of there in the correct format?
Should “: any” at L:173 be set to something else to declare an Array?
As far as I can see your ConsLog #2 is outside a function which will be performed only after the image is loaded. That is why you have 0’s there. You need a callback or promise here, for example.
As for the line 173 you may try myInstance:MyArrayType[] = [];
See also TypeScript Array Type
1 Like
Ah… okay…so that works! I just had to put the log back into the function…
So now, I need to make sure the Thin Instance creation occurs only after this Array has been created…