How to use Babylon with Deno and Fresh?

Questions
1

This could be a issue with Fresh and Deno but I thought I’ll ask here first since I’m using only ESM packages and they’re supposed to work without issues with Deno.

I’m importing the Babylon ESM packages through esm.sh, but I’m getting errors with @babylonjs/materials and @babylonjs/loaders (I’ve imported the glTF loader).

image
image1491×434 46.1 KB

babylon-fresh.zip (13.3 KB)

Relevant import URLs in deno.json:

{
    "babylon": "https://esm.sh/@babylonjs/core@6.21.2",
    "babylon/loaders/": "https://esm.sh/@babylonjs/loaders@6.21.2/",
    "babylon/materials": "https://esm.sh/@babylonjs/materials@6.21.2"
}

Main file is components\babylon\BabylonApp.ts.

BabylonApp.ts 
import "babylon/loaders/glTF";
import { GradientMaterial, SkyMaterial } from "babylon/materials";
import {
  BoundingBox,
  Camera,
  Color3,
  Color4,
  Constants,
  DefaultRenderingPipeline,
  DirectionalLight,
  Engine,
  FreeCamera,
  GizmoManager,
  ImageProcessingConfiguration,
  MeshBuilder,
  PBRMaterial,
  Quaternion,
  ReflectionProbe,
  RenderTargetTexture,
  Scene,
  SceneLoader,
  ScenePerformancePriority,
  ShadowGenerator,
  SSAO2RenderingPipeline,
  SSRRenderingPipeline,
  StandardMaterial,
  Vector3,
} from "babylon";

export default class BabylonApp {
  engine: Engine;
  scene: Scene;
  canvas: HTMLCanvasElement;
  camera: FreeCamera;
  gizmoManager: GizmoManager;

  constructor(canvas: HTMLCanvasElement) {
    this.engine = new Engine(canvas);
    this.canvas = canvas;

    this.scene = new Scene(this.engine);
    this._setPerformancePriority("intermediate");

    this.camera = this._createController();
    this.gizmoManager = this._createGizmoManager();
    this._createEnvironment();

    this._setSnapshotMode("standard");
  }

  _createController() {
    if (!this.scene) {
      throw new Error("No scene");
    }

    const camera = new FreeCamera(
      "Camera",
      new Vector3(1.5, 2.5, -15),
      this.scene,
    );
    camera.setTarget(Vector3.Zero());
    camera.attachControl(this.canvas, true);

    camera.applyGravity = false;
    camera.checkCollisions = false;
    camera.ellipsoid = new Vector3(1, 1, 1);

    camera.minZ = 0.45;
    camera.angularSensibility = 4000;

    camera.speed = 0.25;
    addEventListener("keydown", (event) => {
      if (event.key === "Shift") {
        camera.speed = 0.5;
      }
    });
    addEventListener("keyup", (event) => {
      if (event.key === "Shift") {
        camera.speed = 0.25;
      }
    });

    camera.keysUp.push(87);
    camera.keysLeft.push(65);
    camera.keysDown.push(83);
    camera.keysRight.push(68);
    camera.keysUpward.push(69);
    camera.keysDownward.push(81);

    this.scene.onPointerDown = (evt) => {
      if (!this.engine) {
        throw new Error("No engine");
      }

      if (evt.button === 2) {
        this.engine.enterPointerlock();
      }
    };

    this.scene.onPointerUp = (evt) => {
      if (!this.engine) {
        throw new Error("No engine");
      }

      if (evt.button === 2) {
        this.engine.exitPointerlock();
      }
    };

    return camera;
  }

  _createGizmoManager() {
    if (!this.scene) {
      throw new Error("No scene");
    }

    const gizmoManager = new GizmoManager(this.scene);
    gizmoManager.clearGizmoOnEmptyPointerEvent = true;

    gizmoManager.positionGizmoEnabled = true;
    gizmoManager.rotationGizmoEnabled = false;
    gizmoManager.scaleGizmoEnabled = false;

    return gizmoManager;
  }

  async _createEnvironment(): Promise<void> {
    if (!this.scene) {
      throw new Error("No scene");
    }

    if (!this.camera) {
      throw new Error("No camera");
    }

    if (!this.gizmoManager) {
      throw new Error("No gizmo manager");
    }

    this.scene.shadowsEnabled = true;
    this.scene.imageProcessingConfiguration.toneMappingEnabled = true;
    this.scene.imageProcessingConfiguration.toneMappingType =
      ImageProcessingConfiguration.TONEMAPPING_ACES;
    this.scene.clearColor = new Color4(1, 1, 1, 1);
    this.scene.ambientColor = new Color3(0.6, 0.6, 0.6);

    this._setupPostProcessEffects(this.camera);

    const skySun = new DirectionalLight(
      "skySun",
      new Vector3(0, 0, 0),
      this.scene,
    );
    skySun.direction = new Vector3(-0.95, -0.28, 0);
    skySun.intensity = 2;
    skySun.shadowEnabled = true;
    skySun.autoCalcShadowZBounds = true;
    const sunShadowGenerator = new ShadowGenerator(1024, skySun);
    sunShadowGenerator.setDarkness(0);
    sunShadowGenerator.filter =
      ShadowGenerator.FILTER_BLURCLOSEEXPONENTIALSHADOWMAP;
    sunShadowGenerator.transparencyShadow = true;

    this._setupSkybox(this.scene, skySun);

    const { meshes: kenneyPlayground } = await SceneLoader
      .ImportMeshAsync(
        "",
        "https://raw.githubusercontent.com/fazil47/assets/master/3d/environments/",
        "KenneyPlayground.glb",
        this.scene,
      );
    kenneyPlayground.forEach((mesh) => {
      mesh.isPickable = true;
      mesh.checkCollisions = true;
      mesh.receiveShadows = true;

      if (mesh.material) {
        if (
          mesh.material instanceof PBRMaterial ||
          mesh.material instanceof StandardMaterial
        ) {
          mesh.material.ambientColor = new Color3(1, 1, 1);
          mesh.material.backFaceCulling = true;
        }
      }

      sunShadowGenerator.addShadowCaster(mesh);
    });

    if (!this.gizmoManager.attachableMeshes) {
      this.gizmoManager.attachableMeshes = kenneyPlayground.slice(1);
    } else {
      this.gizmoManager.attachableMeshes.push(...kenneyPlayground.slice(1));
    }

    const { meshes: bmw } = await SceneLoader.ImportMeshAsync(
      "",
      "https://raw.githubusercontent.com/fazil47/assets/master/3d/vehicles/",
      "bmw_m4_2021.glb",
      this.scene,
    );

    const bmwBoundingBox = new BoundingBox(
      new Vector3(0, 0, 0),
      new Vector3(0, 0, 0),
    );

    bmw.forEach((mesh) => {
      mesh.isPickable = true;
      mesh.receiveShadows = true;
      sunShadowGenerator.addShadowCaster(mesh);

      if (mesh.material) {
        if (
          mesh.material instanceof PBRMaterial ||
          mesh.material instanceof StandardMaterial
        ) {
          mesh.material.ambientColor = new Color3(1, 1, 1);
          mesh.material.backFaceCulling = true;
        }
      }

      bmwBoundingBox.reConstruct(
        Vector3.Minimize(
          bmwBoundingBox.minimumWorld,
          mesh.getBoundingInfo().boundingBox.minimumWorld,
        ),
        Vector3.Maximize(
          bmwBoundingBox.maximumWorld,
          mesh.getBoundingInfo().boundingBox.maximumWorld,
        ),
      );
    });
    bmw[0].position.y += 0.09;
    bmw[0].rotationQuaternion = Quaternion.RotationAxis(
      Vector3.Up(),
      Math.PI / 6,
    );

    const bmwBoundingBoxMesh = MeshBuilder.CreateBox(
      "bmwBoundingBox",
      {
        width: bmwBoundingBox.maximumWorld.x - bmwBoundingBox.minimumWorld.x,
        height: bmwBoundingBox.maximumWorld.y - bmwBoundingBox.minimumWorld.y,
        depth: bmwBoundingBox.maximumWorld.z - bmwBoundingBox.minimumWorld.z,
      },
      this.scene,
    );
    bmw[0].parent = bmwBoundingBoxMesh;

    this.gizmoManager.attachableMeshes.push(bmwBoundingBoxMesh);
    bmwBoundingBoxMesh.isPickable = true;
    bmwBoundingBoxMesh.isVisible = false;

    this._resetSnapshot();
  }

  _setupSkybox(scene: Scene, skySun: DirectionalLight) {
    const skyboxMaterial = new SkyMaterial("skyboxMaterial", scene);
    skyboxMaterial.backFaceCulling = false;

    skyboxMaterial.useSunPosition = true;
    skyboxMaterial.sunPosition = skySun.direction.scale(-1);

    const quaternionDelta = 0.02;
    addEventListener("keydown", (event) => {
      if (skySun.direction.y <= 0) {
        if (event.key === "1") {
          this._rotateSun(skySun, skyboxMaterial, quaternionDelta);
        } else if (event.key === "2") {
          this._rotateSun(skySun, skyboxMaterial, -quaternionDelta);
        }
      } else {
        skySun.direction.y = 0;
      }
    });

    skyboxMaterial.luminance = 0.4;
    skyboxMaterial.turbidity = 10;
    skyboxMaterial.rayleigh = 4;
    skyboxMaterial.mieCoefficient = 0.005;
    skyboxMaterial.mieDirectionalG = 0.98;
    skyboxMaterial.cameraOffset.y = 200;
    skyboxMaterial.disableDepthWrite = false;

    const skybox = MeshBuilder.CreateBox(
      "skyBox",
      { size: 1000.0 },
      scene,
    );
    skybox.material = skyboxMaterial;
    skybox.infiniteDistance = true;
    skybox.isPickable = false;

    const groundboxMaterial = new GradientMaterial(
      "groundboxMaterial",
      scene,
    );
    groundboxMaterial.topColor = new Color3(1, 1, 1);
    groundboxMaterial.topColorAlpha = 0;
    groundboxMaterial.bottomColor = new Color3(0.67, 0.56, 0.45);
    groundboxMaterial.offset = 0.5;
    groundboxMaterial.smoothness = 1;
    groundboxMaterial.scale = 0.1;
    groundboxMaterial.backFaceCulling = false;
    groundboxMaterial.disableDepthWrite = true;
    groundboxMaterial.freeze();

    const groundbox = MeshBuilder.CreateSphere("groundbox", {
      diameter: 500,
    }, scene);
    groundbox.layerMask = 0x10000000;
    groundbox.position.y = 0;
    groundbox.infiniteDistance = true;
    groundbox.material = groundboxMaterial;
    groundbox.isPickable = false;

    const reflectionProbe = new ReflectionProbe(
      "ref",
      64,
      scene,
      false,
    );
    reflectionProbe.renderList?.push(skybox);
    reflectionProbe.renderList?.push(groundbox);
    reflectionProbe.refreshRate =
      RenderTargetTexture.REFRESHRATE_RENDER_ONEVERYTWOFRAMES;

    scene.environmentTexture = reflectionProbe.cubeTexture;
    scene.environmentIntensity = 2;
  }

  _setupPostProcessEffects(camera: Camera) {
    if (!this.scene) {
      throw new Error("No scene");
    }

    const defaultPipeline = new DefaultRenderingPipeline(
      "default",
      false,
      this.scene,
      [camera],
    );
    defaultPipeline.fxaaEnabled = true;
    defaultPipeline.glowLayerEnabled = true;

    if (SSAO2RenderingPipeline.IsSupported) {
      const ssao = new SSAO2RenderingPipeline(
        "ssao",
        this.scene,
        0.5,
        [camera],
      );

      ssao.totalStrength = 1.2;
      ssao.base = 0;
      ssao.radius = 1.0;
      ssao.epsilon = 0.02;
      ssao.samples = 16;
      ssao.maxZ = 250;
      ssao.minZAspect = 0.5;
      ssao.expensiveBlur = true;
      ssao.bilateralSamples = 16;
      ssao.bilateralSoften = 1;
      ssao.bilateralTolerance = 1;
    }

    {
      const ssr = new SSRRenderingPipeline(
        "ssr",
        this.scene,
        [camera],
        false,
        Constants.TEXTURETYPE_UNSIGNED_BYTE,
      );

      ssr.thickness = 0.1;
      ssr.selfCollisionNumSkip = 2;
      ssr.enableAutomaticThicknessComputation = false;
      ssr.blurDispersionStrength = 0.02;
      ssr.roughnessFactor = 0.05;
      ssr.enableSmoothReflections = true;
      ssr.step = 20;
      ssr.maxSteps = 100;
      ssr.maxDistance = 1000;
      ssr.blurDownsample = 1;
      ssr.ssrDownsample = 1;

      ssr.isEnabled = false;
    }
  }

  _rotateSun(
    skySun: DirectionalLight,
    skyMaterial: SkyMaterial,
    angle: number,
  ) {
    skySun.direction.applyRotationQuaternionInPlace(
      Quaternion.RotationAxis(Vector3.Forward(), angle),
    );
    skyMaterial.sunPosition = skySun.direction.scale(-1);

    const sunColor = this._getSunColor(Math.abs(skySun.direction.y));
    skySun.diffuse = sunColor;
    this.scene!.ambientColor = sunColor;
  }

  _getSunColor(sunElevation: number) {
    const sunriseSunsetColor = new Color3(1, 0.65, 0);
    const daySkyColor = new Color3(0.6, 0.6, 0.6);
    const dimWhiteSkyColor = new Color3(0.4, 0.4, 0.4);

    const sunriseSunsetThreshold = 0.2;

    if (sunElevation <= sunriseSunsetThreshold) {
      const t = sunElevation / sunriseSunsetThreshold;
      const interpolatedColor = this._interpolateColors(
        dimWhiteSkyColor,
        sunriseSunsetColor,
        daySkyColor,
        t,
      );
      return interpolatedColor;
    } else {
      return daySkyColor;
    }
  }

  _interpolateColors(
    color1: Color3,
    color2: Color3,
    color3: Color3,
    t: number,
  ) {
    const interpolatedColor12 = Color3.Lerp(color1, color2, t);
    const interpolatedColor23 = Color3.Lerp(color2, color3, t);
    const finalInterpolatedColor = Color3.Lerp(
      interpolatedColor12,
      interpolatedColor23,
      t,
    );

    return finalInterpolatedColor;
  }

  _setPerformancePriority(
    priority: "compatible" | "intermediate" | "aggressive",
  ) {
    if (!this.scene) {
      throw new Error("No scene");
    }

    switch (priority) {
      case "aggressive":
        this.scene.performancePriority = ScenePerformancePriority.Aggressive;
        break;
      case "intermediate":
        this.scene.performancePriority = ScenePerformancePriority.Intermediate;
        break;
      case "compatible":
      default:
        this.scene.performancePriority =
          ScenePerformancePriority.BackwardCompatible;
    }
  }

  _setSnapshotMode(mode: "disabled" | "standard" | "fast") {
    if (!this.scene) {
      throw new Error("No scene");
    }
    this.scene.executeWhenReady(() => {
      if (!this.engine) {
        throw new Error("No engine");
      }
      switch (mode) {
        case "disabled":
          this.engine.snapshotRendering = false;
          break;
        case "standard":
          this.engine.snapshotRenderingMode =
            Constants.SNAPSHOTRENDERING_STANDARD;
          this.engine.snapshotRendering = true;
          break;
        case "fast":
          this.engine.snapshotRenderingMode = Constants.SNAPSHOTRENDERING_FAST;
          this.engine.snapshotRendering = true;
          break;
      }
    });
  }

  _resetSnapshot() {
    if (!this.scene) {
      throw new Error("No scene");
    }
    this.scene.executeWhenReady(() => {
      if (!this.engine) {
        throw new Error("No engine");
      }
      this.engine.snapshotRenderingReset();
    });
  }
}
2

cc @RaananW build stuff

3

this is a side-effects issue. Is there a way to force side-effects when using fresh? I haven’t tried fresh yet, so I don’t have a lot of experience with it.

4

Fresh uses preact so you can use useEffect. Link to docs.

5

I’m also trying deno fresh. I didn’t manage to solve the loader plugins import issue yet, but I find the manual register way works.

  1. import the GLTFFileLoader:
import { GLTFFileLoader } from "babylon/loaders/glTF";
  1. register the loader manually:
if (SceneLoader) {
    SceneLoader.RegisterPlugin(new GLTFFileLoader());
}

Then you could use it to load glb file.
You could also check whether the loader is activated or not:

if (SceneLoader.IsPluginForExtensionAvailable('.glb'))
    console.log("GLTF Loader activated");
else
    console.log("No GLTF Loader!!!");

P.S. in your code, you are missing the main render loop to show everthing.

// run the main render loop
this.engine.runRenderLoop(() => {
    this.scene.render();
});

and some more light e.g., HemisphericLight

const _light1: HemisphericLight = new HemisphericLight("light1", new Vector3(1, 1, 0), this.scene);

This is the result:

image
image1920×1114 63 KB

4 Likes
6

Thanks! Everything except importing materials from the @babylonjs/materials works now :smiley:

7

i’ll be happy to help with that (or at least see why it doesn’t work). If there is a project you can share, I will be able to check it.

8

Here’s an updated project:
babylon-fresh.zip (54.4 KB)

It doesn’t work because it searches for these:

missing_shaders
missing_shaders1192×151 18 KB

I think it will work if I store those shaders in ./static/src/Shaders/ (got rid of the console errors by doing that with empty files), but what do I store in those files?

Edit: Got it working by pasting in the shader code from the corresponding .js files.

looking_blue
looking_blue1920×902 46 KB

Thanks everyone for your help!

Final working project:
babylon-fresh.zip (58.5 KB)

2 Likes
9

BTW, if you have time, can you tell me if this happens because of a bug in Deno? I can file an issue in that case.

10

side effects are being ignored. This is quite obvious from all of the examples of issues you all had. Why don’t they run? that’s a question for both the runtime and the bundler. I’ll have time tomorrow to update deno and play with fresh a bit, see why side effects are ignored.

1 Like