Projected Ground Skybox

Leon · May 7, 2023, 10:37am

I would like to be able to create something like this.

https://threejs.org/examples/?q=env#webgl_materials_envmaps_groundprojected

I tried to convert the three shader to babylon, but only ended up with it projecting the top half of the isosphere.

Has anyone else tried doing this and succeeded?

carolhmj · May 8, 2023, 2:19pm

@sebavan if I recall you were talking about something like this recently?

sebavan · May 9, 2023, 6:03pm

I want this as a feature as well in Babylon

Could you share a playground with where you are at and let s try to make it a community developed feature ?

Leon · May 10, 2023, 7:29am

I created a lab for it

Here is the scene
https://labs.leon.id/2023-05-ground-skybox

and the source code:

github.com

leon/labs.leon.id/blob/main/2023-05-ground-skybox/src/ground-skybox.ts

import { BaseTexture } from '@babylonjs/core/Materials'
import { ShaderMaterial } from '@babylonjs/core/Materials/shaderMaterial'
import { Scene } from '@babylonjs/core/scene'

export interface GroundSkyboxMaterialOptions {
  map: BaseTexture
  height?: number
  radius?: number
}

/*
Conversion of https://threejs.org/examples/webgl_materials_envmaps_groundprojected.html
*/

/*
Babylon Uniforms
uniform mat4 world; // mesh global transforming state (move + rotate + scale)
uniform mat4 worldView; // global view part of mesh
uniform mat4 worldViewProjection; // global camera
uniform mat4 view; // mesh local view part

This file has been truncated. show original

and the original source code:

github.com

mrdoob/three.js/blob/44ab823157bcae95b01d857e55a1ce685709448b/examples/jsm/objects/GroundProjectedSkybox.js

import { Mesh, IcosahedronGeometry, ShaderMaterial, DoubleSide } from 'three';

/**
 * Ground projected env map adapted from @react-three/drei.
 * https://github.com/pmndrs/drei/blob/master/src/core/Environment.tsx
 */
class GroundProjectedSkybox extends Mesh {

	constructor( texture, options = {} ) {

		const isCubeMap = texture.isCubeTexture;

		const defines = [
			isCubeMap ? '#define ENVMAP_TYPE_CUBE' : ''
		];

		const vertexShader = /* glsl */ `
			varying vec3 vWorldPosition;

			void main() {

This file has been truncated. show original

All my labs are self contained, so if you checkout my code, cd into the 2023-04-ground-skybox and run the usual with your package manager of choice:

pnpm install
pnpm start

I’m not 100% that my code is converted correctly, but I hope someone smarter than me can figure out where I went wrong

mawa · May 10, 2023, 7:58am

Sry, I have no faen clue but just wanted to say ‘let me know when it’s ready’
This will become a great addition. I luv it already

Leon · May 15, 2023, 6:49am

@sebavan Have you had time to have a look at my code, where you able to get it up and running?

Kind regards
Leon

sebavan · May 15, 2023, 2:54pm

Not yet, planning to do it this week. I have got it assigned to me to be sure I never forget

sebavan · May 22, 2023, 12:30pm

Did not find where you were wrong this weekend but it looks like a nice base so I ll carry on next weekend @Leon

Baktrian · May 23, 2023, 5:02am

I took a crack at it this morning, is there a particular part of the fragment or vertex shader that is not correctly converted?

It looks like something is not working with just the ground part of the icosphere, when i change the height and radius, it does affect the ground, but like its got incorrect UVs or something

Leon · May 23, 2023, 8:15am

I also took another stab at it last night.

I checked the code, and to my knowledge it is converted correctly, so there must be some aspect that I’m missing.

The class it is based on is this one, so it is a good reference point.

github.com

pmndrs/three-stdlib/blob/main/src/objects/GroundProjectedEnv.ts

import { Mesh, IcosahedronGeometry, ShaderMaterial, DoubleSide, Texture, CubeTexture, BufferGeometry } from 'three'

export interface GroundProjectedEnvParameters {
  height?: number
  radius?: number
}

const isCubeTexture = (def: CubeTexture | Texture): def is CubeTexture => def && (def as CubeTexture).isCubeTexture

export class GroundProjectedEnv extends Mesh<BufferGeometry, ShaderMaterial> {
  constructor(texture: CubeTexture | Texture, options?: GroundProjectedEnvParameters) {
    const isCubeMap = isCubeTexture(texture)
    const w = (isCubeMap ? texture.image[0]?.width : texture.image.width) ?? 1024
    const cubeSize = w / 4
    const _lodMax = Math.floor(Math.log2(cubeSize))
    const _cubeSize = Math.pow(2, _lodMax)
    const width = 3 * Math.max(_cubeSize, 16 * 7)
    const height = 4 * _cubeSize

    const defines = [

This file has been truncated. show original

A few things that I’m not sure how they impact the function of the shader is:

github.com

pmndrs/three-stdlib/blob/main/src/objects/GroundProjectedEnv.ts#L22-L24


      
          `#define CUBEUV_TEXEL_WIDTH ${1.0 / width}`,
          `#define CUBEUV_TEXEL_HEIGHT ${1.0 / height}`,
          `#define CUBEUV_MAX_MIP ${_lodMax}.0`,

I cannot find any references to them in the shader code, but they might be used in one of the imports

github.com

pmndrs/three-stdlib/blob/main/src/objects/GroundProjectedEnv.ts#L93-L94


      
          #include <common>
          #include <cube_uv_reflection_fragment>

If you change the code to output projectedWorldPosition to the outcolor, you can see what it is trying to map.

github.com

leon/labs.leon.id/blob/main/2023-05-ground-skybox/src/ground-skybox.ts#L130-L131


      
          vec3 outcolor = textureCube(map, projectedWorldPosition).rgb;
          // vec3 outcolor = projectedWorldPosition;

And I think this could be the problem.
All the ground pixels are black, which when we try to get them from the cube map, they will get the same sand colored pixel.

If we can’t figure it out, I will try to replicate the setup using three, and then we can play around with the shader to debug what the projection should look like.

I also tried out a equirectangular image to rule out that the CubeTexture.CreateFromPrefilteredData was a problem, but it looked the same.

sebavan · May 23, 2023, 6:12pm

I will give it a go this weekend, I am way too curious now

qq2315137135 · May 23, 2023, 7:38pm

I think the cameraPosition uniform is not updated

qq2315137135 · May 23, 2023, 7:43pm

The point is to add the cameraPosition to the uniform array, otherwise it will not be updated automatically.

Give me more likes!!!

F4E6(J20GEUCLKF_BC6%W

sebavan · May 23, 2023, 7:49pm

Perfect Ship It !!!

qq2315137135 · May 23, 2023, 7:59pm

Explanation of the problematic phenomenon:

This shader uses several algorithms in ray casting to determine the poisition of the ground and sky spheres.
Since the algorithm calculates the distance from the camera to the ground (or the sky sphere), it will cause an error if the camera position is not updated.
The reason why the ground is a strange color is because the shader has eliminated the illegal positions (below the ground and outside the sphere), so that all illegal positions are reset to (0,1,0).

So the color of all illegal positions is the same color.

(from translation software, hope you can read it)

Translated with DeepL

Leon · May 23, 2023, 8:13pm

Damn so easy to fix
but great work finding the bug

Leon · May 23, 2023, 8:20pm

@sebavan I can create a PR for adding this as a thing in babylon.

it can either be a material as now, or we tie it to a mesh.

I personally like understanding the building blocks to connect a material to a mesh and so on, instead of having too much magic.

even though it is meant to help out newbies, it can also hurt them because they think its harder than it really is

what should we name the material?

sebavan · May 24, 2023, 4:31pm

@Leon I love the material way and we could put a helper function in sceneHelper for instance to create the mesh and mat for us similar to createDefaultEnvironment ?

About namimg… I am the worst to ask but something like GroundProjectionMaterial sounds descriptive enough ?

@bghgary, please help with naming

bghgary · May 24, 2023, 5:00pm

I’m not sure I understand the feature well enough to help with the naming. I haven’t looked at the code. What does this material do exactly?

Topic		Replies	Views
Projection mapping Questions	17	2219	July 12, 2021
Ground Projection Announcements material	4	531	October 28, 2023
Default SkyBox and Ground Babylon js viewer Questions	7	702	May 18, 2023
MapBox GL to BABYLON Camera Projection Questions	26	2943	November 17, 2024
Ground Projection for 360 photos? Questions	2	90	April 17, 2024

Projected Ground Skybox

Give me more likes!!!

Related topics