Hotloading with Node and webpack causes OOM

Questions
1

I am creating a game based on this boilerplate from RaananW and node v10.13.0. It uses webpack and supports hotloading ← dont take this to mean I have any idea of what I’m talking about, I just know that npm start runs npx webpack serve --config webpack.dev.js.

When I started using the boilerplate, assets loaded fine and hotloading was pretty quick. However, after I refactored scene creation a bit, it takes much longer for the game to load the first time, and hotloading fails with an out-of-memory error roughly 80% of the time.

I’m wondering if any Bablyon JS gurus know what I might be doing wrong. Here is where I load in assets, here is the file where I create the scene, and here is where I start the render loop (called by this bit of code that’s basically from the boilerplate).

And, finally, here is what the startup printout looks like:

(node:20876) [DEP_WEBPACK_COMPILATION_ASSETS] DeprecationWarning: Compilation.assets will be frozen in future, all modifications are deprecated.
BREAKING CHANGE: No more changes should happen to Compilation.assets after sealing the Compilation.
        Do changes to assets earlier, e. g. in Compilation.hooks.processAssets.
        Make sure to select an appropriate stage from Compilation.PROCESS_ASSETS_STAGE_*.
i ｢wdm｣: assets by path *.glb 219 KiB
  asset f1be5aa2d46a7f0ee5873282d82afc34.glb 128 KiB [emitted] (auxiliary name: main)
  asset 4493b64ae5efa5687afd18111a91d836.glb 68.5 KiB [emitted] (auxiliary name: main)
  asset a9fca8e4f2efa6172032ce70c93fcd52.glb 22.1 KiB [emitted] (auxiliary name: main)
asset js/babylonBundle.js 34.4 MiB [emitted] (name: main)
asset index.html 617 bytes [emitted]
runtime modules 25.4 KiB 12 modules
modules by path ./node_modules/@babylonjs/core/ 8.76 MiB 921 modules
modules by path ./node_modules/@babylonjs/gui/ 660 KiB 61 modules
modules by path ./node_modules/@babylonjs/loaders/glTF/ 349 KiB 38 modules
modules by path ./src/ 69.2 KiB 23 modules
modules by path ./node_modules/webpack-dev-server/ 21.2 KiB
  modules by path ./node_modules/webpack-dev-server/client/ 20.9 KiB 10 modules
  modules by path ./node_modules/webpack-dev-server/node_modules/ 296 bytes 2 modules
modules by path ./node_modules/webpack/hot/ 4.46 KiB 5 modules
modules by path ./node_modules/html-entities/lib/*.js 61 KiB 5 modules
modules by path ./assets/*.glb 2.57 KiB 4 modules
modules by path ./node_modules/url/ 37.4 KiB 3 modules
modules by path ./node_modules/querystring/*.js 4.51 KiB 3 modules
5 modules
webpack 5.13.0 compiled successfully in 108251 ms

and the error

<--- Last few GCs --->

[20876:000002AAB32DD690]   741000 ms: Scavenge 1308.6 (1422.0) -> 1308.0 (1422.5) MB, 4.5 / 0.0 ms  (average mu = 0.255, current mu = 0.243) allocation failure
[20876:000002AAB32DD690]   741009 ms: Scavenge 1309.0 (1422.5) -> 1308.4 (1423.0) MB, 4.7 / 0.0 ms  (average mu = 0.255, current mu = 0.243) allocation failure
[20876:000002AAB32DD690]   741018 ms: Scavenge 1309.5 (1423.0) -> 1308.8 (1423.5) MB, 5.4 / 0.0 ms  (average mu = 0.255, current mu = 0.243) allocation failure


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 000000DC5625C5C1]
Security context: 0x00027581e6e1 <JSObject>
    1: /* anonymous */ [0000033A6B202251] [path\to\guardin-gnomes\node_modules\source-map\lib\source-node.js:~342] [pc=000000DC57D65533](this=0x02cb55d1ad49 <JSGlobal Object>,chunk=0x01a358506621 <String[1]: .>,original=0x02f54db2b3b9 <Object map = 00000213D37BC069>)
    2: SourceNode_walk [000002BBDBEEB811] [path\to\guardin-gnomes\...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 00007FF79F59ECE5 
 2: 00007FF79F578196 
 3: 00007FF79F578BA0
 4: 00007FF79F808D5E
 5: 00007FF79F808C8F
 6: 00007FF79FD469D4
 7: 00007FF79FD3D137
 8: 00007FF79FD3B6AC
 9: 00007FF79FD44627
10: 00007FF79FD446A6
11: 00007FF79F8E7767 
12: 00007FF79F97F44A
13: 000000DC5625C5C1 
npm ERR! code ELIFECYCLE
npm ERR! errno 134
npm ERR! guardin-gnomes@0.0.1 start: `npx webpack serve --config webpack.dev.js`
npm ERR! Exit status 134
npm ERR!
npm ERR! Failed at the guardin-gnomes@0.0.1 start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
2

Loading a few assets should not lead to an OOM exception. This is strange.

Care to share the project with me? So I will try and understand what happened? Would be great to see what changes were made and whether or not it fails locally for me as well.

I doubt it is a node version issue, but just in case - what node are you using?

3

Help from the master himself, I’m honored!

I didnt realize the project was set to private (so my links are useless). I just set it to public, but the repo is at GitHub - austinmilt/guardin-gnomes: tower defense game

1 Like
4

I am cloning the repo now, I hope to find the reason soon :slight_smile:

5

So, I now can buy towers and have 200$ :wink:

I don’t know if you have a tsconfig file, but it was missing in the repo (and it is in the gitignore as well). Once I added it (including the downlevelIteration which you were missing, since you are iterating over a set) it compiled correctly.

  "compilerOptions": {
    "module": "esNext",
    "target": "es5",
    "moduleResolution": "node",
    "strict": true,
    "noImplicitAny": true,
    "noImplicitReturns": true,
    "noImplicitThis": true,
    "strictNullChecks": true,
    "strictFunctionTypes": true,
    "skipLibCheck": true,
    "preserveConstEnums": true,
    "sourceMap": true,
    "downlevelIteration": true,
    "rootDir": "src"
  }
}

It did take ca. 30 seconds for the initial compile (which I assume has something to do with the amount of assets that should be cached in the init stage), but after that hot loading takes 5-10 seconds to reload the page with the right content.

Also, it seems like the code itself has an issue (setting a material for an instanced mesh), but this is not a compile issue but a runtime issue.

Does this solve the issue for you, or do you already have a tsconfig file set like that and it happens with this file in the repo?

6

Ah! Good catch. I do have a .tsconfig (below). I just removed it from the gitignore and added it to the github repo.

My builds are taking ~60-90 seconds for the initial compile, and hotloading another 30 seconds or so followed by failure or success.

For posterity, here’s the tsconfig

{
	"compilerOptions": {
        "module": "esNext",
		"target": "es6",
        "lib": [
            "es6",
            "dom"
        ],
		"moduleResolution": "node",
		"strict": true,
		"noImplicitAny": true,
        "noImplicitReturns": true,
        "noImplicitThis": true,
        "strictNullChecks": true,
        "strictFunctionTypes": true,
        "skipLibCheck": true,
		"preserveConstEnums":true,
        "sourceMap": true,
        "rootDir": "src"
	}
}

I see I have a couple differences from you. I can try using your version later to see if that makes a difference (still doing my day job atm).

7

Also, I’m using node v10.13.0

8

Went on and checked the build process.
There are 2 issues here - first, it seems like webpack 5 has a different memory management than webpack 4. 4 never failed with OOM, but was a bit slower in my tests.
But the main issue is actually… us. You are importing all of the babylon classes from “@babylonjs/core”. Since babylon es6 module has side effects (legacy support for an older way of building the framework) importing from the main directory will technically load the entire framework. It seems like this caused slowing down the build process (and over-used a lot of memory).

When changed all imports to the correct directories and modules, build time was reduced to ca. 20 seconds initially and 2-5 seconds hot-loading.

My recommendation is to use the absolute importer with an extra step (if you are using vscode). Don’t auto-import automatically, but use this:

2021-03-10 10_49_49-● camera.ts - guardin-gnomes - Visual Studio Code
2021-03-10 10_49_49-● camera.ts - guardin-gnomes - Visual Studio Code865×249 17.4 KB

1 Like
9

Thanks so much for the detective work sir! I’ll give this a shot later and report back the result.

1 Like
10

That did the trick, thanks! It greatly reduced the build size and hotloading time. :bowing_man:

Now, if I can just remember to use the absolute imports for new code :sweat_smile:

1 Like