Single esm module that can provide asset paths when called from front or backend

Questions
1

Short version : I’d like to create a single package that contains my game’s content assets (meshes, textures etc) that can be loaded from both a vite frontend and a node.js backend.

Short why : My game can run fully in browser without connecting to the server, or I can run the game server on a node.js backend and have clients connect to it. I want to use Babylon’s headless/null mode to load assets on the server so that things like hitboxes are sync’d

Setup

I have a package called kit-game-assets that I want to hold all my static game assets "textures, meshes etc

This package is referenced by another package called kit-game-core which I want to load the assets in kit-game-assets.

The kit-game-core package is referenced by a vite based frontend project called kit-frontend AND a node based backend project called kit-backend

I want the assets to be loaded in both envs.

Problem
In kit-game-core if I load that asset using

import assetTest from "kit-game-assets/assets/test/wf.png"
console.log(assetTest)

On the frontend this returns “/@fs/home/azim/code/kit/core/packages/kit-app/kit-game-assets/src/assets/test/wf.png” from the vite dev server, this is correct and vite intercepts @fs paths during development and correctly bundles them during prod build.

But on the backend this results in

node:internal/modules/esm/get_format:185
  throw new ERR_UNKNOWN_FILE_EXTENSION(ext, filepath);
        ^

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".png" for /home/azim/code/kit/core/packages/kit-app/kit-game-assets/src/assets/test/wf.png

I get this is more of a general node thing, but was hoping other people have had to tackle something like this to have a server and frontend aware of their assets.

2

People often get confused when dealing with node.js on the server and treat it like a browser environment. Asset loading works differently. Node.js doesn’t handle static files automatically. You can still import js/ts though.

On the backend you must load your assets yourself using the filesystem, something like this:

This should work:

const assetPath = resolve(dirname(fileURLToPath(import.meta.url)), "textures/city.env");
const buffer = await fs.readFile(assetPath);
2 Likes
3

well, I think I have it working, plus I kinda hate it so it’s probably the right solution :expressionless:

I’ve got a solution that :

  • loads the assets correctly on frontend when running in dev mode
  • loads the assets correctly on frontend after production build
  • loads the assets correctly on backend

I’ve uploaded an example because that might be clearer than my long-ass explanation :smiley:

This is what an asset library would have to do in order to make assets available in both a front and backend context
kit-game-assets.zip (2.3 MB)

This enables you to write something like the following code an not have to do anything else special for asset handling

import { assetPaths, getAssetPath } from "kit-game-assets";

async loadAssetTest(isInDevMode: boolean) {
    return getAssetPath(assetPaths.wfPng, isInDevMode);
  }

To do this properly you need to handle the following 3 conditions in the source library.

Assuming you have an asset in the dist/assets/test/wf.png folder of your asset package you have to do the following for each scenario

For all scenarios
Export a list of relative paths, for this example this is

export const assetPaths = {
  wfPng: '/assets/test/wf.png'
};

Scenario 1 : Load the assets correctly on frontend when running in dev mode

Scenario 2 : Load the assets correctly on frontend after production build

  • return the relative path ‘/assets/test/wf.png’
  • ensure the assets are copied from the source library to the frontend’s dist on build

Scenario 3 : Load the assets correctly on backend

  • return the “filePath”, doing this as follows
const __dirname = dirname(fileURLToPath(import.meta.url));

export function getAssetPath(relativePath: string, isRunningInDevMode: boolean): string {
  return join(__dirname, 'assets', relativePath);
}

The slightly tricky part is that in order to cover both back and front end working properly the library needs to provide different functions depending on the context, this is handled by providing 2 different implementations of the same function and letting, you can configure the loading behaviour in package.json’s exports

  "exports": {
    ".": {
      "node" : {
        "default" : "./dist/index.node.js",
        "types" : "./dist/index.node.d.ts"
      },
      "default" : {
        "default" : "./dist/index.browser.js",
        "types" : "./dist/index.browser.d.ts"
      }
    },
    "./assets/*": "./src/assets/*"
  },

This works but I really hope it’s stupid and really overcomplicated and someone can show me a much much more straightforward way of doing this :smiley:

(for context here the Interim message I was writing before I managed to get a first working solution)

===================================

Hey, appreciate the speedy reply but that doesn’t quite answer what I’m trying to accomplish but I think it’s because my question wasn’t clear enough.

I’m looking to have a single package that contains the assets, referenced by another packages which I can call in both a frontend and backend context.

Current setup

  • a package called kit-game-assets holds assets
  • a package called kit-game-core has a function called loadHexploreGameMode inside this function it starts the game and, ideally, creates entities with their textures and assets
  • a react/vite package called kit-frontend which calls loadHexploreGameMode in kit-game-core
  • a node package called kit-frontend which calls loadHexploreGameMode in kit-game-core

The problem is I can’t see a way to have kit-game-core load the assets in a way that would work for both the front and backend.

===================================

4

I think you are overengineering the whole stuff :slight_smile:

Put all your assets in a public/assets dir. You can then read them from both environments. Use fetch in a web app and fs.readFile on a server. When deploying a web app these files will be available on the web server. When running on a node.js server these will be available on the filesystem.

You can use a factory class with a unified interface to get the correct loader. Use dependency injection to inject the loader from your current environment.

export interface ILoader { 
   loadAsset(partialUrl: string):Blob
}

export class BrowserLoader implements ILoader {
// loadAsset uses fetch
}

export class NodeLoader implements ILoader {
// loadAsset uses fs
}

export class Loader {
  constructor(private loader: Loader) {}

  async load(partialUrl: string): Promise<Blob> {
    return this.loader.loadAsset(partialUrl);
  }

What do you think?

3 Likes
5

You can also “fake” the requests in the node/deno/bun environment so you can load the files from your asset server, like the public folder in your client. That is what I do currently in my game, so I just provide the asset path url to the server

Currently traveling so cannot show snippets xD

2 Likes
6

Example for meshes: Loading GLTF models in Nodejs

1 Like
7

Cheers all really appreciate the help !

I almost did exactly that dep injection approach and I might still switch, but when I was thinking about what I’d need to make it work during dev and build time I thought:

The server and client aren’t going to be able to share a single source of assets in production

  • the frontend files are either going to be on a CDN or, more likely, in an app wrapper like capacitor
  • this means I’d need to ensure new / updated assets are copied out to both assets folders both during dev and build time.
  • It’s easy to have a watch script to do that, but
  • I’d definitely still want a single source of truth for my assets, which means at least a separate folder, which may as well be a package so that it can provide typed metadata for the available assets which may as well also contain the watch/copy script which… you see where I went :slight_smile:

I’m torn, I do like that I have :

  • a single package that servers as my single source of truth for the raw assets
  • exports metadata about the assets, other packages that need it can pnpm i in
  • only have to worry about copying the assets at prod build time because during dev it’s taking advantage of the symlinks created by the package manager

The loading examples are also helpful ty ! I’d just gotten to getting a filepath for the assets on node.js I hadn’t yet verified if the babylon loaders knew to check & handle filepaths !

8

Don’t use filepaths but load the file content into an ArrayBuffer. Loaders accepts SceneSource as the source for loading:

1 Like
9

I’m fetching and caching objects into ArrayBuffers as suggested, however when passing these into babylon loaders I’m getting an error, calling :

const meshAssetContainer = await babylon.LoadAssetContainerAsync(new DataView(asset), this.scene);

results in
gameStartupHelper.ts:299 Uncaught (in promise) When using ArrayBufferView to load data the file extension must be provided.

I can’t figure out what to pass to let it know what it should be loading (in this case it WAS a glb file)

I’m looking at Babylon.js docs and wondering if I need to base64 encode the string before passing it in, but this feels like I’m missing something.

I’ve tried options like

        pluginExtension : "gltf"
      });

But no joy. In the meantime I’m looking at https://github.com/BabylonJS/Babylon.js/blob/c190296494a0c192edb26689045c929902476937/packages/dev/core/src/Loading/sceneLoader.ts#L576 for more clues

10

Update for future people : this seems to work

const base64String = "data:;base64," + btoa(String.fromCharCode(...new Uint8Array(asset)));
          meshAssetContainer = await babylon.LoadAssetContainerAsync(base64String, this.scene);

However, it’s very slow, it feels like even when it’s being loaded from the string it’s doing… ??something?? that results in a really slow load and tons of ‘network’ calls like this (which might just be standard when blobs are read… not sure

image
image430×441 13.5 KB

This is inside of LoadImage (not sure where as it’s minified) and just seems to happen when an image is loaded from base64

image
image1415×219 32.5 KB

      let meshAssetContainer: babylon.AssetContainer = null;
      if (useCache) {
        if (this.assetLoader.getAssetCacheItem(`${assetPath}:assetContainer`)) {
          meshAssetContainer = this.assetLoader.getAssetCacheItem(`${assetPath}:assetContainer`);
        } else {
          const asset = await this.assetLoader.getAssetFromPath(assetPath);
          const base64String = "data:;base64," + btoa(String.fromCharCode(...new Uint8Array(asset)));
          meshAssetContainer = await babylon.LoadAssetContainerAsync(base64String, this.scene);
          this.assetLoader.addAssetCacheItem(`${assetPath}:assetContainer`, meshAssetContainer);
        }
      } else {
        meshAssetContainer = await babylon.LoadAssetContainerAsync(assetPath, this.scene);
      }

To speed this up I’m caching the loaded asset container, which I think is a reasonable thing to do (happy to be given other recommendations / gotchas of doing this).

I think the original thread’s intent is mostly resolved now, I have a working way of loading typed assets from a package on the front and backend, but very happy if anyone has more insight to share.

11

This should be “.glb” instead of “gltf”

    const response = await fetch(assetUrl)
    const url = URL.createObjectURL(new Blob([await response.arrayBuffer()]))
    await BABYLON.AppendSceneAsync(url, scene, {
        pluginExtension: ".glb"
    });

However Blobs are supported in the browser environment by default.