Mastering Lighting - Specific Guide vs Trial And Error

Questions
1

Hi!

My name is Nicolas and I am making a mobile game / web browser game with a stained glass aesthetic / models.

In particular I love stained glass and the ability to reflect and emit dynamic, soft lighting and want to masterfully portray this visual effect / art form in Babylon.

I am a little overwhelmed by the variety of lighting methods in Babylon. There are multiple techniques / possible systems - and although the documentation is quite good, I am left with the question of which lighting techniques to test / use as a starting point.

In general I want ALL of the stained glass to sort of glow and be “radiant” and pleasing to the eye. Dull stained glass is just that - dull - so I am thinking a global light is good? But then there are pre-baked lights presumably inside of the model? Then there are shadow maps. and volumetric lighting?

I also see that there is emissive lighting which is a separate system, and my stained glass test model reflected but not not appear to transmit / radiate / allow light to pass through and illuminate it dynamically. The stained glass models I currently have are NOT transparent, and not emissive, so I am thinking this is the problem. I am having my technical artist make a variation of a test model with A) 1 model with emissive lighting, 2) 1 model with transparency in the glass and C) 1 model with both.

This is a good step in the right direction for testing creating lighting effects - but then there is the multivariable giga calculus of all of the lighting systems, techniques and settings to go alongside these specially modified models.

Instead of brute force testing EVERY single lighting technique, I was hoping to have some more insight from someone who has played with lighting as to some starting points in what will make glass "pop’ and really be colorful and energizing. I am open to very technical code, I just don’t want to try 100 techniques when maybe an expert / knowledgeable person can point me in the right direction.

Also, could some consideration be paid / could we discuss the performance implications of having 1 or more lights in a scene? At least one global light is necessary 100%, and I imagine the performance for that isn’t too terrible of a hit. But I am interested in experimenting with settings with the various dynamic shadows, multiple lights, gradient lights, shifting lights, and so on. Does anyone know any shorthands / techniques for premptively “guessing” or ball-park-figuring how much adding a few lighting sources / shadow sources will lower performance?

I am targeting 100-150 dollar phones (iOS and Android) and have already tested some base code on iOS and its snappy, but there’s plenty of optimization to add, models to add, and effects to add.

I’m thinking I will have a few performance settings that allow for higher end phones to be a little richer in graphics, whereas the default settings are more conservative.

Also, there is post processing and shaders…adding those can dramatically change the overall visual effect, but I didn’t want to get overwhelmed thinking about that when I still need to master basic lighting and achieving SOMETHING close to what I envision with the stained glass.

Do I need to KNOW the shaders / post processing im going to use BEFORE calibrating / deciding my lighting? Or can I simply get “pretty good” or “darn good” lighting and safely assume the post processing / shaders will be a little extra pizazz without ruining everything? And how big of a performance hit is it? Is there any source of data / experimental trials / documentation that shows the progressive performance hit of various layers of complexity being added?

We’re making a fairly intricate, ambitious game, so I really want it to be as beautiful as can be and show off Babylon’s capabilities. I’m also open to documenting my performance testing so that others in the future have ballpark estimates of performance levels with various settings and numbers of models / animations / lightings / etc on multiple tiers of phones / devices. I already will have to profile the hell out of the game and optimize the hell out of the engine and the models / graphics, so I think it only makes sense to give back to the community in that later phase and give really good documentation and help.

I know i’m asking a lot of open ended / complex questions, but ANY information or resources aside from just the master docs would be helpful.

Thanks so much for your time and happy holidays!

1 Like
2

cc @PatrickRyan, but please be patient as he is currently on vacation.

3

Thank you Evgeni, do I need to do anything or is your @ the cc we needed?

4

Yes, the “cc” will ping Patrick.

5

@Nicolas_Bulgarides, I wish there were some resource I could point you to that would answer every one of these questions, but as you noted, these are fairly large questions. When considering lighting in a scene, you have to take more into account than just how many you are using. Specifically, your experience with stained glass as a major part of your aesthetic adds a level of complexity that many other experiences will not require, making your benchmarking somewhat unique.

Without seeing some of the art targets you are aiming at - I understand you won’t be able to show these while in development - it would be helpful to see some of the reference images you are taking inspiration from so that we can help answer some of your questions on approach. For example, are these stained-glass windows the background in a side-scroller/platformer or are they elements in a 3D environment that the user can move through? Are you using a day/night cycle such that light coming through the windows needs to be dynamic? Is your art target based on stylization or realism?

When trying to determine your pipeline for creating your assets, the best thing to do is to simply prototype to see where performance issues may arise. This is because lighting is more than just type and number, but also how complex are the meshes that are being lit? How many meshes are in scene? What casts and catches shadows? What type of shadows do you need (hard, soft, contact hardening)? What in your scene is dynamic or are there some assets that are static and can be baked? Are you using transparency in your materials beyond the windows? How large is the scene? All of these can have an impact on what you choose for lighting. In some cases, lighting is important, but in what you are describing, light can be thought of as another character on screen as the color from the windows is an important part of your experience. This can push you in another direction that makes performance of your scene different than of a typical Babylon scene.

You also mentioned that you need to be able to run on low-end devices. While I can see a world where you have a high-end profile for devices that can render it, I would almost caution against relying on separating out high end devices for a different technique. If you can hit your art target with low-end devices, I would use the high-end devices to have access to larger textures or maybe another LoD on your assets rather than a separate feature set. This will keep your scope narrow. Also, I have found that limitations always result in more creative solutions to your technical challenges.

For a suggestion as to how to approach this experience, I would not look to identify one method that will take you to the finish line because right now you do not know what you do not know. A better approach is to fail fast. Put something into the engine as quickly and cheaply as possible and push it until it fails. Pick the next thing and repeat. Mix these approaches/systems and again push them until they fail. You will start to see trends on the devices you are targeting that will inform you of the best way to build your final product. This is much cheaper and faster than trying to find a close approximation to what you want to do and using their performance profile. Doing this can result in finding out you need to make major revisions to your pipeline or techniques midway in production when it’s very costly to make changes. Here are some things to try:

  • Test out your material model. Do you want PBR? Is Standard Material enough control if you don’t need IBL? Do you need something stylized where a custom shader would be best for you? Do you want parts of PBR but don’t need our entire PBR shader?
  • Do you want your windows to cast light onto your scene? If so, look into projection textures as an option to simplify the scene.
  • If you need your light to be dynamic and move within your window, can you do that with a shader rather than a light?
  • If you need volumetric light to cast from your windows, can you do that with mesh? A mesh is easy to control and there are no tricky lighting calculations to be done for volumes.
  • If you need to use multiple lights for your scene, how do you separate meshes so that you can limit which meshes fall within the lighting calculations for each light. Every mesh that is calculated adds a performance cost, but if a mesh is completely occluded from a light, removing it from the list of meshes for calculations will make some gains in performance.
  • Depending on your stylization you could explore a lightless approach where you handle lighting directly in the shader without the need for scene lights. Here is an example of a complex lighting setup with a simple texture which is a very cheap way to do lighting if your camera does not move as the view direction is used in the calculation for the texture position. But this is only one technique that could be considered as an alternative to scene lights, but work a look if you want to keep things really cheap for low-end devices.

These are just a few questions to ask while you are thinking about your approach, but they should lead to others. I could spend all day writing out suggestions, but without context, I would be wasting a lot of your time. If you have questions about specific techniques, I am happy to talk about those. Any additional context you can add about your specific scene without breaking any NDA would be helpful in anwering. Hope this helps for a primer.

3 Likes
6

Hi @PatrickRyan

Thank you so much for your response. I’m reading it as I go down through this reply and addressing your specific points.

To start with, here is a screenshot that shows the basic UI / controls at the bottom of the screen (very pleased I got that working and scaling correctly), and a 3d stained glass and metal model that our modeler made and I put into the engine. The character model is AI generated and was just a placeholder for test purposes so its quite dinky. Right now to achieve this overall graphics I am using 3 directional lights, one from left, right and above to add a universal sort of lighting ambiance that worked well at least for this bridge glass model. I am absolutely open to showing some of the art that is in development! And the game is a puzzle game that will be an overhead POV of a 3d unit (either humanoid or a mechanoid sphere, we’re experimenting) that will traverse a “boardfield” or “map” composed of stained-glass based tiles and decoration that are sized based off a standard unit of 1M. So the map is sort of slotted together like lego pieces, except instead of plastic the pieces are 3d models with the glass and metal texture to show off Babylon’s unique lighting capabilities. As players move through the map (tile by tile, or multiple tiles at a time similar to a board game) there will be minigames / “Filler puzzles” that are the core gameplay / time sink. Traversing the map will let them collect items and discover things. So a puzzle game with a mix of exploration and progression.

https://imgur.com/a/M1cOhSS

This link below shows some other misc models but they aren’t displayed in the context of my current implementation of the game engine / displayed with the 3 directional lights that I am currently using.

[42 new items added to shared album] Assets_images

As I currently imagine it a day / night cycle is unnecessary though perhaps it’s something I would experiment with just to add some visual pizazz.

Regarding shadows, I have not turned on shadows yet, though that is something I wish to eventually explore. Right now with the universal lighting I am not sure what would cast shadows, though I guess if I switched lighting to more of a lightbulb / torch point lighting system then shadows would come into play.

Most of the models are static, 3d tiles / blocks that are locked into a fixed position on a standardized grid. Could you explain what you mean by baked? Do you mean merged inside of Blender / modeling program? I think I saw a technique in the docs to merge meshes so there is a smaller amount of draw cycles. That makes sense and shouldnt too hard, I imagine. The scene would perhaps be 30 - 50 meters by 30 - 50 meters on a flat Y axis, MAYBE some additional floors. So if a tile is 1x1, think 30-50 tiles wide and deep, but level on the Y axis.

I’m going to look into your bulleted lists of questions -

  1. I have to ask the artist if the models currently are PBR or standard material. I don’t yet know what a PBR shader is so I have to research that more rather than waste your time.

  2. Regarding the windows casting light - I don’t think so - the stained glass is the accent texture of the gameboard structure. There aren’t actual stained glass windows like in a Cathedral, rather, the body of the game is itself stained glass meant to be illuminated and vibrant.

  3. Would a shader be more efficient than a light? Do you think it would look better? Do you maybe have any examples of shader effects vs lighting effects that could be informative to view?

  4. I don’t really know what volumetric light means, so I’m going to look into this more rather than waste your time. If you have any tips on the effects / quality of it versus other lighting systems, or maybe some ramifications of performance - any tips here would be appreciated, but I dont want to waste your time since I have to research it more.

  5. Currently as mentioned I am using directional lights from up, left and right, and not bothering with one from below since the character wont ever see “below the board” / floor. Think a board game made of 3d models, but you’re looking overhead. Technically, I could make models that are not edge models ignore lighting from left / right / in front / behind, since those surfaces would not be affected by light because they would be covered by adjacent models. Is this what you mean in terms of limiting specific lights from specific models / meshes?

  6. I’m testing the lighting setup you showed in the shader example by substituting the bridge model to see how it looks.

Also NDA isn’t really an issue, we’re a small startup / 5 person company and I think our vision and execution will be unique enough that it wont be practical for someone to both clone our art style, gameplay style, target market and every other characteristic of the game. But as of now I’m not worried about that! Maybe it’s naive. But for now the benefits of your assistance, which are profoundly appreciated, far outweigh the non-zero risk of imitation, I think.

Your information was super helpful and Im going to review it several times and read the documentation on some of the topics youve mentioned.

if the art samples / screenshot is helpful and you have any other ideas an insight, any more of your assistance is greatly appreciated but not expected / assumed. But I do appreciate you!

Thanks so much.

2 Likes
7

@Nicolas_Bulgarides, depending on how the model’s materials were prepared (i.e. were the textures authored in blinn-phong with specular-gloss textures or physically based rendering (PBR) with metallic-rough or PBR specular-gloss textures), your three directional lights setup may not be the most optimal. From the differences I am seeing in the renders on the Google photos link you shared to the render I am seeing in engine, these materials look to be authored as PBR materials. I say this because it appears in the source renders that the metallics look more accurate and they appear to have an environment light. For the Babylon render you show, the areas of metal look flatter and not as metallic.

This is due to needing some environmental reflections in the form of an Image Based Light (IBL). If your models are loading from glTF, you will be using PBRMaterial already, so you just need to provide an IBL through an HDRI conversion to the scene. You can find them at places like Poly Haven, and then you simply download a file in .hdr format and drop into the sandbox. The rest of the documentation for generating an .env file is accurate, but we expanded it to support .dds and .hdr files to eliminate the conversion step to .dds. Using an environment with a single directional light will get your scene render closer to the sample renders.

If you don’t have a day/night cycle and your lighting doesn’t dynamically change, you can take advantage of baking your lights into shadow or light textures in Blender. This will allow you to render high quality shadows on your objects that don’t move in a texture so no calculations are necessary. There is an older thread from a few years ago that talks about light and shadow textures in Babylon that is worth a look.

If you are snapping your pieces to a grid, they have likely four possible rotations. This means you would need to bake four light/shadow textures per piece and apply the correct texture to match the rotation of the mesh, but the cost of the extra textures can be mitigated by loading them manually as needed and assigning them. This becomes a tax paid at load rather than on every frame, which will generally increase performance and render quality.

In looking at the Google Photos references, there are a couple images showing light projected on the ground. The “rays” you see are volumetric light. Light rays formed by particulate in the air that gives the impression of a column of light. For something like this, I would create the rays in mesh and then use an additive shader with some fresnel falloff to soften the edges of the ray. A dark color on an additive shader will give the impression of a ray of light and is really cheap in terms of performance. If your ground is always flat and at the same level where the light strikes it, I would cast the light like you would a shadow with a piece of mesh holding an emissive texture. This will simulate light being cast on the ground through the glass, but again without any light calculations.

Some of the thicker meshes like the arch that appears to be made of thick glass or gemstone may need some techniques like transmission (glTF) known as translucency in Babylon. This is a bit of an expensive technique so you want to be judicious in its use. It also needs some WebGL2 tech, so if you know your target devices may be really cheap, you may want to avoid it. You can also fake it with some UV manipulation of textures in the shader, especially if you are always viewing the scene from a locked camera angle.

This should be enough to get you pointed in a direction to do some prototyping and then pop back with questions. Good luck with the project!

2 Likes
8

Thank you so much I’m going to reflect on this (several times) and discuss with my technical artist!

1 Like