[noob] Why do skyboxes use reflection textures? (And: Super weird behavior)

If I start with the skybox example from the lovely tutorial (https://playground.babylonjs.com/#KBS9I5#88), and change “skyboxMaterial.reflectionTexture” to “skyboxMaterial.emissiveTexture” (on lines 14 and 15), then things get weird! Instead of the pretty cloudy sky images, now I get… a copy of the “village green” ground texture that was used elsewhere.

Q1: Why is it reusing a different texture image without complaint?? Seems like an issue (maybe weird behavior because I’m doing something wrong?). Maybe a CubeTexture can only be used as a reflectionTexture??

Q2: Why do skybox examples use reflectionTexture in the first place? There’s no reflection going on. Naively, I’d just expect a giant cube with an emissive texture wrapped around the faces, to sit out there at ~infinity glowing like the fake sky it is.

Q3: What does the SKYBOX_MODE coordinate mode do?

Any help welcome, I am confuzzle.
– egnor

1 Like

Welcome aboard!

Q1: shader code is expecting a texture 2D for emissiveTexture, not a cube texture, that’s why it does not work. Babylon is not checking its inputs for performance sake, it’s up to the user to pass only valid inputs. As to why it’s the ground texture that is used instead in this case I don’t know…

Q2: I think it’s because reflection textures can work with a cube texture and a cube texture is what is needed for sky boxes.

Q3: SKYBOX_MODE is one of the coordinates mode you can use with a cube texture and indicate how to lookup into the cube. The available values are described here. If you want more detail, you will need to dig into the source code: Babylon.js/reflectionFunction.fx at master · BabylonJS/Babylon.js · GitHub

Hello and welcome to BJS,

I believe the first question would be, what do you need a ‘reflection texture’ for?
The skybox includes a reflection texture because the skybox can be used to project a reflection on objects or meshes in the scene. It is cubic because the skybox is, in essence, cubic. Though is has a ‘SKYBOX_MODE’ basically instructing that you will want to use your ‘skybox’ for reflection/refraction.

Now, this does not mean, you have to use your skybox for reflection. You can use it for environment and use another texture for reflection.
An emissive texture/color is mostly inappropriate to use in a skybox (except for some special effects)
To catch a level of your environment or skybox on a given mesh, it’s the mesh material you will want to ‘tune and tweak’ , may be also using your emissive. Note that very few materials really have a strong emissive color so you want to set this level very low. You will next work the reflection and refraction levels which you can also tune and attenuate using a normal texture (again on your mesh, not on the skybox).

Thanks both for the quick and thoughtful answers! If you don’t mind, please correct errors in my understanding? (I’m sure I got some things wrong!)

=====

(general graphics background)

In a scene, there are (at least) two major uses for “environment maps” (panoramic images capturing the view in all directions).

  • A “skybox” to capture very-far-away scenery – clouds, distant mountains, etc – to paint onto a flat box around the world. (Kind of like fake-sky-domes in the Truman Show or the Hunger Games, or scenery “flats” in a stage play.) It’s imperfect because there’s no parallax, but if the skybox objects are very far away that’s okay, and flat images are way faster than actually rendering all those clouds and mountains and stuff.

  • A “reflection map” that captures the view (of both near and far objects) from the vantage of a particular object, and gets warped and pasted onto the object (based on the camera’s vantage and the object’s shape) to simulate reflective surfaces (a mirror, shiny spoon, etc). This is again imperfect because there’s no parallax, but for reflective objects which are smaller than their surrounding this is okay, and it’s a lot faster than a complete re-render for the reflection (e.g. via raytracing).

(Babylon.js / GL specifics)

In Babylon.js, both use cases for environment mapping use some common mechanisms:

  • The “CubeTexture” class, which represent an image-based environment map. (The base CubeTexture uses six square images, subclasses take other inputs.) Note, though it’s a “texture”, it can’t be used in every place “ordinary” textures can be used.

  • The “reflectionTexture” property on Material, which holds an environment map (e.g. a CubeTexture) to be painted on surfaces, as if they were situated in and reflecting that environment.

For a reflective object – a shiny spoon, say – one might make a CubeTexture picturing the scene around the spoon, and assign it to the spoon’s material’s reflectionTexture. Very logical!

For a skybox, one might use a CubeTexture as a handy way to load six images (or some other panoramic-image format) into a box. (Or one might use ordinary UV mapping to paint a regular texture onto the skybox cube.) One would NOT expect to use reflectionTexture for a skybox, since a skybox isn’t “reflecting” anything, it’s just a big fake sky…

BUT, Babylon does not support using a CubeTexture as an “ordinary” texture (diffuse, emissive, specular) on an object, only as a reflectionTexture. THUS, if you want the convenience of using a CubeTexture to load a panoramic image for a skybox, you have to use it as a reflectionTexture.

BUT BUT, we don’t want to “reflect” the environment on the skybox – we want to show it directly, not act as if the skybox were a giant mirror. So we set coordinatesMode=SKYBOX_MODE on the CubeTexture, which says to “reflect” as if the environment is pasted directly on the object (not actually a reflection at all). In general, coordinatesMode controls how reflections are computed (does it have any effect on non reflection textures?).

(Again, for a skybox, one could apply the cube images to the sides of a cube as a classic emissiveTexture with appropriate UV mapping and sidestep the whole issue, but CubeTexture is very convenient, so setting reflectionTexture to a CubeTexture with coordinatesMode = SKYBOX_MODE is the standard way to make a skybox in Babylon.js.)

(Arguably it might be more sensible for CubeTexture and MirrorTexture and such to not be considered “textures” at all, but called something like “environment images”, and to have a material property like environmentMap to use them, and replace coordinatesMode with environmentMapMode, since as far as I can tell there’s no real commonality between “ordinary” textures and “environment” textures. But that’s not how Babylon is structured, partially because that’s not how GL is structured??)

=====

Is some of that even close to right?? (Thanks again for your help!)

1 Like

Oh, apologies… I can now see you do have a (good) level of skills and background. Thanks also for writing this text. Very nicely done and comprehensive (in my opinion). Also some good thinking and questioning/challenging and, I must say, that your suggestion of having ‘environment images’ as a method/class is interesting, though (knowing a little of BJS) I’d rather think that there must be something behind it.

Now I’m still not sure what your problem really is (or if there is one). I would have applied the solution you stated above. Why wouldn’t it work in your case?

Oh, sorry, I don’t have a problem in the sense of not being able to do something! My problem was just that I didn’t understand why things are done a certain way (and why attempting to do them a different way had surprising results).

So if my current understanding is good, you can consider the case closed! Perhaps I’ll submit a PR to the docs with some breadcrumbs for other folks.

Yes, I have started with BJS in december 2019 (after 25 years of digital service) and posted comments alike (after working with Flash to Unity, the Cry or Dunia Engine or Unreal). In 3D, I have been mostly served by my clients with 3ds and maya but found myself more inclined to use C4D or Blender.

There are indeed some cumbersome/disturbing parts to BJS when you hit them in the first place, but in the end, I must say I have mostly been unable to offer a better solution to the existing. May be we just need to get acquainted to these parts, knowing that, in my opinion, you can get huge advantages on other parts (and you’ll always find a solution for your case… if you are open to some compromise… but, this is also true with any other framework/engine and in general, isn’t it?

Also, on behalf of the BJS Community (if I can allow myself:) I’d like to thank you for your implication and assure you that if you spend some of your valuable time to place a PR (or even a challenge), it will be considered here.

Meanwhile, have an awesome WE,

1 Like

coordinatesMode is not a property of a reflection texture, it is a property of a cube texture (a reflection texture can be a cube texture but also a 2D texture). Depending on how you want the computation to be done, you must choose the right value.

MirrorTexture is not a texture in the same way Texture and CubeTexture are, it’s a render target texture (RTT): all objects are drawn in this RTT from a mirrored camera (relatively to the mirror plane) and then this RTT is used as a regular texture in a diffuse / emissive / … texture property of a material.

Note that the refraction texture can also be a cube texture, not only the reflection texture.

It may be an historical choice to have named it reflectionTexture in the material. Note that the property at the scene level is named scene.environmentTexture! But as there is a refractionTexture property too, reflectionTexture is a sensible name to me.

Yes, I think that for the cube texture the name was chosen because it is really how this kind of texture is named in GL (but also everywhere else).

I think you got it perfectly :slight_smile:

To close this loop: Here’s the PR with my proposed documentation update. It kind of grew over time, for which I’m sorry! (Also, I bet I got at least something wrong in there…!)

4 Likes

OMG. That’s an amazing contribution. Let me dig into all this a little, but I can tell you already I love the title.

I’ve been asked to break it up (which is fair) so expect that particular PR to disappear and be replaced by other smaller ones…

This is awesome, thank you! I’m eager to see the PRs for this get merged.

Can you use GHFMD for the tables instead of HTML?

I broke the original PR up – the new PR for the reflections doc is here: Revise the "diving deeper" (howto) page on reflections & refractions by egnor · Pull Request #230 · BabylonJS/Documentation · GitHub
Comments & corrections most welcome.

Markdown’s table support has issues (you have to have a header row and at least one data row, it’s not pretty when there are several columns with nontrivial content, etc) but I’ve tried to make it work and minimize raw HTML as much as possible.

1 Like

Hey I hear ya on that one! VSCode has a MD extension(s) that offer some table formatting commands, and Markdown Monster is an awesome stand alone (free+) MD editor

HTH

While you are on it, check out this one. May be you could explain the meaning of this (seen in a featured PG for this section). I mean, my english is mostly broken but this description is simply beyond my understanding :wink:

Edit: Don’t explain it to me. I know what it does. Just wanted to show this content.

1 Like

Wow. What does it mean? I gather that some parameter of refraction is being modulated by the material’s opacity (maybe even according to a texture??) but which parameter?

– egnor

Oh LoL, you don’t know it. Apologies. Well, you wouldn’t learn from this description, would you?
Basically (but then as I said my english is broken) what it does is that applies the level of alpha (transparency) of the material to its refraction texture level (as a multiplier or divider), meaning that changing your alpha also affects the level of your refraction texture.