Shadow from partly transparent (video/chromakey)texture

Hallo, and sorry for my terrible english :wink:
Is there a way to get the shadow from an plane with chroma-keyed video-texture not to throw the shadow form the plane itself but from the not transparent parts of the texture.
I’m paying an greenscreen-video with an person; i make the green pixels transparent by using following function:

videoMat.Fragment_Before_FragColor(vec3 chromaKey = vec3(0.15,0.64,0.15); // 0.15,0.64,0.15 // 0.40,0.90,0.52 vec3 tolerance = vec3(0.18); vec3 limitMin = vec3(chromaKey-tolerance); vec3 limitMax = vec3(chromaKey+tolerance); limitMin.r = max(limitMin.r, 0.); limitMin.g = max(limitMin.g, 0.); limitMin.b = max(limitMin.b, 0.); limitMax.r = min(limitMax.r, 1.); limitMax.g = min(limitMax.g, 1.); limitMax.b = min(limitMax.b, 1.); if( ((color.r >= limitMin.r) && (color.r <= limitMax.r)) && ((color.g >= limitMin.g) && (color.g <= limitMax.g)) && ((color.b >= limitMin.b) && (color.b <= limitMax.b)) ){ discard; }else{ color.a = 1.; });

Now i’d like to cast only the shadow of the person, not the videoplane itself.

MANY thanks for any help.

Welcome to the forum!

You can use BABYLON.ShadowGenerator, and push your render list into the render loop.

Here’s the documentation:
https://doc.babylonjs.com/babylon101/shadows

Galen

thanks for reply.
BUT, ialready pushed it to the shadowgenerator, but i only get an rectangular shadow of the whole plane.
Anything else i could do to get the shadow from the non transparent parts of the video-texture only?
i managed to get the correct shadow by using an transparent png as texture, but NOT using the video.

var videoTexture = new BABYLON.VideoTexture("s",["./cam4.mp4"], vrscene);

var videoMat = new BABYLON.CustomMaterial(“video material”, vrscene);
videoMat.Fragment_Before_FragColor(
vec3 chromaKey = vec3(0.15,0.58,0.18); // 0.15,0.64,0.15 // 0.40,0.90,0.52 vec3 tolerance = vec3(0.25); vec3 limitMin = vec3(chromaKey-tolerance); vec3 limitMax = vec3(chromaKey+tolerance); limitMin.r = max(limitMin.r, 0.); limitMin.g = max(limitMin.g, 0.); limitMin.b = max(limitMin.b, 0.); limitMax.r = min(limitMax.r, 1.); limitMax.g = min(limitMax.g, 1.); limitMax.b = min(limitMax.b, 1.); if( ((color.r >= limitMin.r) && (color.r <= limitMax.r)) && ((color.g >= limitMin.g) && (color.g <= limitMax.g)) && ((color.b >= limitMin.b) && (color.b <= limitMax.b)) ){ //discard; color.a = 0.1; discard; }else{ color.a = 1.; });
videoMat.diffuseTexture = videoTexture;
videoMat.diffuseTexture.hasAlpha = true;
videoMat.backFaceCulling = false;
videoMat.texture = videoTexture;
videoPlane.material = videoMat;
var shadowGenerator = new BABYLON.ShadowGenerator(1024, light);
shadowGenerator.getShadowMap().renderList.push(sphere);
shadowGenerator.getShadowMap().renderList.push(sphere2);
shadowGenerator.getShadowMap().renderList.push(videoPlane);

You have a valid point. Perhaps a bug? If you’re using a hemispheric light then the shadow generator wonk work as hemispheric light don’t cast shadows. But here’s a PG scene I modified which uses a point ligh and I would expect it to cast a shadow from the plane.

https://www.babylonjs-playground.com/#P75330#31

Here’s a PG scene with a texture.
https://www.babylonjs-playground.com/#P75330#32

If I place a sphere in the scene it works fine. Anyone have any ideas? I believe I tried everything I know.

Galen

1 Like

Unfortunately video do not contains alpha value so you also have to update the shadow generator code to update it to take your chroma key change into account

1 Like

could you give me any hint, how to do this??? :wink:

So maybe first step would be for you to prepare a plyaground which repro where you are at the moment and then we could help you :slight_smile:

First of all: Thanks for your help!

This is the playground:

https://playground.babylonjs.com/#II3Z1J#1

Problem:
.) the shadow of the video-texture is the whole plane-rectangle. (seems, i need to set the pixel alpha anyhow, instead of using discard)
.) is there a way to use the video without a light, so i don’t get the light’s reflection when doing chroma-key?

Here is an example on how to use your own shader for the shadow map as well:
https://www.babylonjs-playground.com/#WE7DRF#4

Following up on this thread. Anyone figure out a working version? (last PG throws a bunch of errors).

I would recommend to use Node Material Editor now for the same question

@Deltakosh, can you elaborate on your last response? I am trying to get shadows to work with transparent videos, and haven’t quite got things working with all the good stuff in this thread.

I meant: NME is a good candidate to help writing shaders (for both material or shadows)

You can use the new ShadowDepthWrapper class to easily generate shadows for your custom shaders:

https://playground.babylonjs.com/#II3Z1J#3

See Custom shadows with ShadowDepthWrapper

1 Like