Server can't find folder, only files

Hey,

I’m currently working on implementing a server as a provider using Node/Express.
I’ve tried this before and never has it worked, failing at the exact same spot everytime.

When starting the server and connecting to localhost, there is an 404 error on the babylonjs/core import.
image

The imports look like this

import * as BABYLON from './node_modules/@babylonjs/core';

I’ve clicked on it, and it gives me the correct folder (or index.d.ts file I guess).
In the browser network tab, it for some reason looks like this
image
What does this mean? It can find core but not core/ ???


I know the server does provide the entire contents of node modules, because doing this

import {Vector3} from './node_modules/@babylonjs/core/Maths/math.js';

works. But I don’t want to import everything manually, trying to find the file where it’s from.

Is there something I’m missing?


Here’s my server js if the problem is here

import express from "express";
import path from 'path';
import { fileURLToPath } from 'url';

const app = express();
const port = process.env.PORT || 3000;
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

app.listen(port, () => {
     console.log(`Server is up on port ${port}`);
});

app.use(express.static('public'));

app.get('/', (req, res) => {
     res.send('Welcome to my server!');
});


Here is the package.json on root directory

{
  "name": "projectcs",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "vite build",
    "preview": "vite preview"
  },
  "devDependencies": {
    "vite": "^5.0.8"
  },
  "dependencies": {
    "express": "^4.18.2"
  }
}

Here is the package.json on public directory

{
  "name": "projectcs",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "vite build",
    "preview": "vite preview"
  },
  "devDependencies": {
    "@babylonjs/core": "^6.34.3",
    "cannon": "^0.6.2",
    "vite": "^5.0.8"
  },
  "dependencies": {
    "@babylonjs/havok": "^1.3.0",
    "express": "^4.18.2"
  }
}

Hey, if you could provide repo would be easier (at least for me to verify it).
As for now you can check my repo where BabylonJS is working on the NodeJS

Hey, thanks for sharing

Here is a repo I just created.
In contrast to yours, I’m not using typescript and don’t have files such as tsconfig.

Hi, I already run into same issue before.
I was using NPM and trying, as you, to import Babyl stuff the same way you’re doing it (with JS ES6 classes).

You have 2 solutions (for what I know):

1. First solution
JavaScript ES6 classes don’t allow folder import such as
import * as BABYLON from './node_modules/@babylonjs/core';
You have to import things manually, and this, will really make you suffer. :sweat_smile:
import { Vector3, MeshBuilder } from "./node_modules/babylonjs/core/index.js";
Not recommend, it fulfills your code and make it heavy.

2. Second solution
Use webpack as explain here Babylon.js ES6 support with Tree Shaking | Babylon.js Documentation
This will make it possible to do things like follow:
import { Vector3, MeshBuilder } from "@babylonjs/core";
Recommended.

Hey, I made it working locally. Here you have all the file public/main.js:

I used import { Animation, etc } from "@babylonjs/core";

// import * as BABYLON from 'http://localhost:3000/node_modules/@babylonjs/core';
// import * as BABYLON from './node_modules/@babylonjs/core';
// import {Vector3} from './node_modules/@babylonjs/core/Maths/math.js';
//import * as CANNON from './node_modules/cannon';

import {
  Animation,
  CascadedShadowGenerator,
  Color3,
  CubeTexture,
  Curve3,
  DirectionalLight,
  Engine,
  GlowLayer,
  HemisphericLight,
  MeshBuilder,
  Scene,
  SceneInstrumentation,
  StandardMaterial,
  UniversalCamera,
  UtilityLayerRenderer,
  Vector3,
} from "@babylonjs/core";

// this is a lookup table for a bezier curve
const bezierPrecision = 256;
var p1 = new Vector3(0.0, 0.0, 0.0);
var p2 = new Vector3(0.2, 0.0, 0.0);
var p3 = new Vector3(0.8, 1.0, 0.0);
var p4 = new Vector3(1.0, 1.0, 0.0);
var bezier3 = Curve3.CreateCubicBezier(p1, p2, p3, p4, bezierPrecision);

Animation.prototype.floatInterpolateFunction = function (
  startValue,
  endValue,
  gradient
) {
  return (
    startValue +
    (endValue - startValue) *
      bezier3._points[Math.ceil(gradient * bezierPrecision)].y
  );
};

// special map for the key input
class CyborgMap extends Map {
  constructor() {
    super();
    this.constructor.prototype.increment = function (key) {
      this.has(key) && this.set(key, Math.min(65535, this.get(key) + 1));
    };
    this.constructor.prototype.decrement = function (key) {
      this.has(key) && this.set(key, Math.max(0, this.get(key) - 1));
    };
  }
}

class Avatar {
  constructor() {}
}

// function to check if two sets are 100% equal
const setsEqual = function (xs, ys) {
  return xs.size === ys.size && [...xs].every((x) => ys.has(x));
};

// key presses
const keyRate = 100;
const keysRated = new Set(["f", "q"]);
const keys = new CyborgMap();
keys.set("w", 0);
keys.set("a", 0);
keys.set("s", 0);
keys.set("d", 0);
keys.set(" ", 0);
keys.set("f", 0);
keys.set("q", 0);

function keyPressed(key = "f") {
  return keys.get(key) == keyRate;
}
document.body.addEventListener(
  "keydown",
  function (e) {
    if (keys.get(e.key) == 0) {
      keys.set(e.key, keyRate);
    }
  },
  false
);
document.body.addEventListener(
  "keyup",
  function (e) {
    keys.set(e.key, 0);
  },
  false
);

// raycast
class Raycast {
  constructor(scene) {
    const indicatorPointMat = new StandardMaterial("indicatorPointMat", scene);
    indicatorPointMat.emissiveColor = new Color3(1, 0, 0);
    indicatorPointMat.alpha = 0.7;
    this.indicatorPoint = MeshBuilder.CreateSphere(
      "indicatorPoint",
      { diameter: 0.1 },
      scene
    );
    this.indicatorPoint.isVisible = true;
    this.indicatorPoint.isPickable = false;
    this.indicatorPoint.material = indicatorPointMat;
    this.indicatorPoint.hitId = undefined;
  }
  write(hitId, hitPos) {
    this.indicatorPoint.hitId = hitId;
    this.indicatorPoint.position = hitPos;
  }
  read() {
    return [this.indicatorPoint.hitId, this.indicatorPoint.position];
  }
  show() {
    console.log(read());
  }
}

// Create the scene space
const canvas = document.getElementById("renderCanvas");
const engine = new Engine(canvas);
const scene = new Scene(engine, {
  fogStart: 150,
  fogEnd: 400,
  fogColor: Color3.Black,
  collisionsEnabled: true,
  autoClear: true,
});
scene.createDefaultSkybox(
  new CubeTexture("/textures/environment.env", scene),
  true,
  300,
  false
);
// scene.enablePhysics(new BABYLON.Vector3(0,-12,0)); // physics engine

const glowlayer = new GlowLayer("glow", scene);
const ambientlight = new HemisphericLight(
  "light",
  new Vector3(0, 20, 0),
  scene
);
ambientlight.intensity = 0.4;
const sunlight = new DirectionalLight(
  "spotlight",
  new Vector3(-1, -1, 1),
  scene
);
sunlight.position = new Vector3(6, 5, 6);
sunlight.intensity = 5.0;
sunlight.shadowEnabled = true;
sunlight.shadowMaxZ = 10000;
sunlight.autoCalcShadowZBounds = true;
const shadowGeneratorCascaded = new CascadedShadowGenerator(2048, sunlight);
shadowGeneratorCascaded.lambda = 1.0;
shadowGeneratorCascaded.cascadeBlendPercentage = 0.0;
shadowGeneratorCascaded.debug = false;
shadowGeneratorCascaded.autoCalcDepthBounds = true;
shadowGeneratorCascaded.numCascades = 0;
shadowGeneratorCascaded.shadowMaxZ = 5000;
shadowGeneratorCascaded.bias = 0.02;
const camera = new UniversalCamera("MyCamera", new Vector3(0, 1, 0), scene);
camera.minZ = 0.1;
camera.attachControl(canvas, true);
camera.speed = 0.05;
camera.angularSpeed = 0.05;
const utillayer = new UtilityLayerRenderer(scene);
const instrumentation = new SceneInstrumentation(scene);
instrumentation.captureRenderTime = true;

// global variables
const divFps = document.getElementById("fps");
const divDrawCalls = document.getElementById("drawcalls");
const debug = false;
const raycast = new Raycast(scene);

// game running functions
engine.runRenderLoop(function () {
  // profiling
  divFps.innerHTML = engine.getFps().toFixed() + " fps";
  divDrawCalls.innerHTML =
    "vcalls: " + instrumentation.drawCallsCounter.current;

  // render frame
  scene.render();
});
window.addEventListener("resize", function () {
  engine.resize();
});

// before render
scene.registerAfterRender(function () {
  // key press handler
  keysRated.forEach((value, key, map) => {
    keys.decrement(key);
  });
});

function pressAction() {
  const pickInfo = scene.pick(scene.pointerX, scene.pointerY);
  if (pickInfo.pickedMesh == null) {
    return false;
  }
}

// after render
scene.registerBeforeRender(function () {
  // main update loop
  if (keyPressed("f")) {
    pressAction();
  }

  // holding item
});

1 Like

That sounds good.
I have to setup a new project I assume… vite and webpack are rivaling plugins if I understood it correctly.

That means I’ll be running

Node / Express / Webpack / Babylon

That should be fine right?

A bit annoyed, having to restart the project for a 5th time, and many more times, just because one detail is missing.

@Shirhix No, you don’t have to. I can create PR to your repo with this working (what I wrote before). I can help you with that if you will still have any problems

Sorry, but it doesn’t make much sense to me. I already injected webpack, which didn’t work.
I can’t use the code you wrote, since I can reference without / ./ …/

I should mention that I don’t have any problems running this locally with the vite server.
The problem is only when starting the node server to localhost:3000.

Okey, so first of all, with my changes you need to do like npm run build. The build goes to the public/dist.
So here

you need to change it to
app.use(express.static("public/dist"));

Then at the root you run
node server.js and at localhost:3000 I have it running

Did you manage to have it working?

Ok, now I am completely lost. I’ve followed so many tutorials and everyone says something different.
I have so many questions I don’t know where to begin with.
I do not understand folder structure and why I’m required to name certain folders in a certain way.

If anyone can answer my questions below, I just need to know every detail before I drive the project into the wall for a fifth time.
Please correct my thought process…

  • I want to create an app on a ec2 instance, so I need to write a server that serves the client with files.
  • This server is in the root directory of the project. It is accompanied by a package.json…
  • A package.json tells the server what dependencies it needs.
  • A folder called node_modules contains the dependencies.
  • In the root directory, there is a folder called public, in there is what the clients are served for security reasons.
  • In the public directory, there is yet another package.json, this time filled with dependencies only for the client.
  • In here are also the index.html, main.js and other js files that entail the functionality of the game
  • A node_modules folder contains the dependencies for clients
  • The client will also find all resources such as textures and models here

image

  • When I run the server via node, it opens localhost:3000 like I want it to.
  • I am presented with the index.html and from there, the main.js file is executed.
  • In the main.js, imports to babylonjs are called. Individual files are loadable, but not folders.
    image
  • I’m required to call on imports using ./ .// so the server doesn’t search for them in my system, but rather in cd, which is public, thanks to
app.use(express.static('public'));
  • Now I’m supposed to install webpack to be able to import folders.
  • I follow the tutorial, but I can’t translate it to my scenario, they talk about having an index.js file and a main.js file, that must mean index.js is the server and main.js is the client.
  • Why is the index.js file filled with non-server stuff? I thought babylon stuff should be in main.js.
  • Tutorial never mentions main.js again.
  • Why is the src folder not in the public folder, why doesn’t the client get a folder? Why should I make a src folder if I can just put files in the public folder or keep them at root?
  • Now I should call
npx webpack serve

and check localhost:8080, I have installed vite, so I’m using port 5173.

  • I can’t check if it worked or not, because it worked before installing webpack.
  • I run
node server.js

to start the node server on port 3000 after going up one dir.

  • Same problem as before installing webpack.
  • call last instruction from tutorial
npx webpack

image

  • I didn’t plan on creating a src folder and screw over my entire project again
  • I also missed when to create src.js, src.json, src.wasm (I don’t think those are things I need)
  • That means I must start over, and potentially screw up again.

What file structure am I supposed to have?

projectname
- public
  - node_modules
    - @babylon
  - index.html
  - main.js
  - package.json
  - package-lock.json
- node_modules
server.js
package.json
package-lock.json

Every folder needs a package.json file.


Where am I supposed to install what?

root:
- npm init
- npm create vite@latest
- npm install express
- npm install webpack webpack-cli webpack-dev-server --save-dev

public:
- npm init
- npm install @babylonjs/core

This is what I gathered after watching countless tutorials.

Yes, this did work. I don’t understand why though, if you can explain why this works, like what is changed internally?

Bit of a hassle to rebuild each time, I guess this is why I was instructed to install webpack.

Okey, so first thing is that I would recommend you to spend some time learning the Web basics. So far in this thread I see that you don’t know what is what and what tool you should use.
Second thing is that you don’t need Webpack. Vite is enough what I did prove already. They are both bundlers.

  • I want to create an app on a ec2 instance, so I need to write a server that serves the client with files. - I can’t help you with that, I don’t have experience with ec2
  • A folder called node_modules contains the dependencies - to be more clear in the node_modules you have downloaded required libraries and then while importing eg. function from any library the bundler should add that function to YOUR bundled JS file which is later executed by the browser. In other words you shouldn’t import directly from the node_modules
  • I’m required to call on imports using ./ .// so the server doesn’t search for them in my system, but rather in cd, which is public, thanks to - no, you don’t
  • Now I’m supposed to install webpack to be able to import folders. - no, you don’t
  • Why is the index.js file filled with non-server stuff? I thought babylon stuff should be in main.js. - there is no strict rule in the naming convention, you can have index.js in the client and in the server
  • Why is the src folder not in the public folder, why doesn’t the client get a folder? Why should I make a src folder if I can just put files in the public folder or keep them at root? - in the src folder you should have source code (src is short for source), and from that src folder you should build whatever you need to the public/dist folder. So in the src/ folder you can have things that you don’t want to be reachable online - that’s why you don’t want to put everything in the public/ folder

First of all, in the public/ folder via build script you build things into the dist folder. So you want to serve public/dist via Express (node). That’s why I wrote you need to change it to
app.use(express.static("public/dist"));
Secondly, as I already wrote, you don’t want to import things from the node_modules. Please learn about ES modules and bundlers. You import from @babylonjs/core and bundler is looking for that in the node_modules and takes it and put it in your output file.

No, webpack is not for that. Webpack is for the client part and for the dev part. Not for the server part.
For the refreshing the server on the change you can use nodemon or forever or some other watcher.

As for the learning part I suggest you to read this

It has 3 years and isn’t exactly the same as yours example - but I think it’s very well written by Raanan and as he said - you can take it or leave it :wink:

To sum up, you don’t need to use webpack at all. Please learn about the tools and the environment step by step.
The flow is like eg.:

  1. as for the development you have two consoles opened
    1a) in the first one you have client running on vite and rebuilding on the change
    1b) on the second one you have Node server running and serving whatever vite is building
  2. for the production part you build your stuff via build script to the public/dist folder and you serve it from the Node server. The names of the folders are free to be changed. You can look at the BabylonJS template (where Webpack is used) or into my repo (where I use Parcel).

Good luck :crossed_fingers:

only the compiled files of your vite app go into the folder that is served via node , not all your project folders.

When you build your vite app , it creates the minified code for this , normally in a dist directory. These are the only files that should be on your server within the nodejs directory serving static files

you can have the two projects completely separate on your dev machine , and run them on two different ports.

here , look at these examples etc…

First of all thanks for sticking with me.
I want to learn, I just don’t have the time. My boss has even less knowledge than me and explaining to them why something doesn’t work puts pressure on me.

I went into this blind, hoping to use the least amount of plugins and third party tools. I did not watch any tutorials explaining what they do, only that I need them.

I have a hard time getting the project to work on builds and on dev without duplicate content.

  • In your example, your app.use points to client/dist, which is filled when you run build. But when you have additional files like textures or models, those are not built to dist, so from where are they served?
  • I only managed to serve them from root, but then I couln’t load them when running dev because they’re above public. Do I have to duplicate this content?

My dist is in public and I changed app.use to public/dist, but my textures are not in dist (and they get deleted from dist when built), they are in public for my dev runs. And copied in root for my build runs.
I call

app.use(express.static("public/dist"));
app.use("/textures", express.static("textures")); // the textures folder in root

to serve them for builds.

My file structure looks like this
image
The textures folder is in root and in public.

so it’s

  • src/public/dist
  • I serve public/dist, so clients can’t reach src, only public and dist
  • If I serve public/dist, am I also serving any other folder in public automatically?

I removed node_modules from the import line. Though I could swear it didn’t work for dev runs before. Now it does.

  1. I have that set up exactly like you describe and it works like it’s supposed to.
  2. I have deinstalled Webpack, and don’t have Parcel either. The console says Vite is doing this. If I didn’t know this, I would have installed Parcel this instant because you showed me a tutorial using it, therefore I need it. But I don’t.

All of this is very confusing. I just wanted to serve a texture without having security issues, duplication and headaches.

Hey, thanks for joining in.

So textures shouldn’t be in root. I wanted to serve them from where a dev build would use them (in public/textures) but I can’t serve that when already serving public/dist. I could only manage to serve textures in root.

I feel like I’m done going through tutorials, they all use similar setups but completely different methods. Trying to pair my structures to theirs is not what I’m capable of at the moment.

well this is not a great statment to make ? this just leads to you running in circles because you do not understand the tools you are using.

you do not need more than one directory serving static assets from nodejs , The entire vite/vue/whatever app is self contained within the directory you place it

When you build with vue or vite etc…, it will copy/recreate ALL required static assets of the app within the dist directory and paths are resolved if required, of course that is if you have even set up your app properly in the first place , knowing the structures of the development directories and the general rules of bundlers in regards to scripts and assets.

eg an image in

src/assets/images/home_image_portrait.png

and one in

src/public/assets/images/home_image_portrait.png

is referenced in the app like this :

import imageUrl1 from ‘@/assets/images/home_image_portrait.png’

this is because the build will create the assets directory at the root level of your build directory

image files from src/assets/ are copied or recreated to dist/assets , depending on the rules of the bundler for the file type and size

image files from src/public/assets/ are copied to dist/assets without any modifications from the bundler.

again look at the path used to reference the image , ‘@/assets/images/home_image_portrait.png’

it looks for the image in an assets directory at the root of the app … which is where the build will put it and when running dev , the path is resolved as needed to find it in either directory

these are basics you have to learn. Im not going to give anymore feedback here as I think the ball is in your court now to go learn the fundimentals of these build tools you are using

I don’t even know what counts as fundamental

Is this fundamental? I typed “vite how to import an image” into google.

  • files have to be imported so that vite bundles them to dist

I want to load an .env texture now, but there’s no import for such file. That means I’m googling again.

  • A vite.config.js is needed at root level, with rules for imports and a couple other things.
  • For special file types I need to add **/*.ext to my rule, in my case **/*.env.
import { defineConfig } from 'vite';

export default defineConfig({
    assetsInclude: ['**/*.gltf', '**/*.env']
});

A bit problematic, since environmental variables are also .env and building the app like this will cause vite to freeze without an error.

It’s getting more difficult to formulate a google question on why the environment texture which only babylon uses is getting mistaken for an environment variable.
Actually, I can’t find a single person having the same problem.

I admit that lacking fundamentals is terrible, I learned very shallowly over the two weeks.
But I still can’t load an environment texture.

That doesn’t sound good. I don’t know your situation but the only thing about that I can say is I can recommend you to tell the truth to your boss - if you won’t spend time on learning it will be bad for the project overall. Even if you would manage to get it done you will not understand how it is working and you will suffer with every problem found.

The common pattern is to copy the assets to the dist/ after bundling. Bundlers usually have it as an option. So eg. you have public/assets folder where you have your assets. In that configuration they are available to you while development. With Parcel and some copy plugin I have it working like this:

For Vite I’ve found something like this

You should put the textures folder inside public folder.

What src folder do you have in mind? I don’t see any folder called src in your repo.
If so the convention is to have eg.

public
--dist
--src
----assets

No, it depends on what you write in Express.
For app.use(express.static("public/dist")); you serve only public/dist and not public/otherFolderOrFile

About the env textures - What problem do you have with them exactly?

Btw,
Have you checked out this?

Thanks for taking your time.
I haven’t updated the repo, I should do that.

A bundler does this automatically I presume.
I did this for vite, it worked, but I have to assume why.

Loading the gltf from both dev and build + server worked well.
As long as they are common file types.

I did specify **/*gltf to be built to dist/assets in the vite.config.js
And so it would have worked for the env too, but a .env file is not commonly known as the environment texture file, but rather environment variable file.

import envUrl from '/textures/environment.env';

trying to build with an env gave this error

I’d need a plugin or add the **/*.env in my asset include in the vite.config.js.

import { defineConfig } from 'vite';

export default defineConfig({
    assetsInclude: ['**/*.gltf', '**/*.env']
});

But if I add that, vite freezes on both run dev and run build.

Trying to read environment variables from a texture is what I imagine doing LSD is like.
At least that is what I think is happening.

I’m glad that you moved forward with your project after our comments.
Please update the repo as it is easier for me to check if I can help.

I haven’t worked with the textures. Is it possible to convert env file to some other type?
Might it be also bug in the Vite itself?
I guess as the last resource you can migrate to webpack.

I apologize for the late response. I’ve been on vacation and the office was closed over then. I am now back and have updated the repo.

I think env-files are interchangable with dds files, and I’m going to try these.