Running automated tests on macOS vs Ubuntu

Questions
,
1

I’ve run the automated tests on macOS (Sequoia 15.6.1) and on Ubuntu 24.04 LTS and get different results. In both cases some tests fail. I’ve several related questions/requests:

  1. Should the documented test commands work on both operating systems? (and perhaps on Windows too?)
  2. Why are there more failures on macOS?
  3. Would changing the documentation to use npx jest … be acceptable?

The documented command npm run test:unit fails on macOS but works on Ubuntu. I can run the tests using npx jest --selectProjects=unit on both these operating systems.

I found a tweak to jest.config.js reduces some of the failures and happy to submit a PR for this and anything else that’d help.

FWIW The error when running the unit tests with npm run … on macOS follows:

Babylon.js % npm run test:unit                                

> @babylonjs/root@1.0.0 test:unit

> npm run build:assets:smart-filters && jest --selectProjects=unit

> @babylonjs/root@1.0.0 build:assets:smart-filters

> tsx ./packages/dev/smartFilters/src/utils/buildTools/buildShaders.ts ./packages/dev/smartFilterBlocks/src/blocks smart-filters core

/Users/julianharty/sandbox/Babylon.js/packages/dev/smartFilters/src/runtime/shaderRuntime.ts:5

import { EffectWrapper } from "core/Materials/effectRenderer.js";

         ^

SyntaxError: The requested module 'core/Materials/effectRenderer.js' does not provide an export named 'EffectWrapper'

    at #_instantiate (node:internal/modules/esm/module_job:249:21)

    at async ModuleJob.run (node:internal/modules/esm/module_job:357:5)

    at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:665:26)

    at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:99:5)

Node.js v24.3.0
2

cc @BabylonNative

3

ping @docEdub

4

@amoebachant had a fix for this that I thought went in already. Any idea why it’s failing for unit tests?

5

Yeah, my fix went in last Tuesday. I’ll try running the UTs on a Mac with a fresh clone and see if I can repro this.

6

Actually, looking at the error again, it doesn’t look like you had my fix @Julian_Harty - could you grab the latest and try again? With the fix we aren’t using tsx to build the smart filter assets any longer.

7

Thank you. I’ve pulled the latest changes and the command now runs without complaining. There still seems to be a TypeError (log output below). Also, I notice it seems to be checking for cached results. How can I clear the cache(s) and see the unit tests actually run?

Here’s the output

Node.js v24.3.0
(base) julianharty@Mac Babylon.js % npm run test:unit  

> @babylonjs/root@1.0.0 test:unit
> npm run build:assets:smart-filters && jest --selectProjects=unit


> @babylonjs/root@1.0.0 build:assets:smart-filters
> npm run build:assets:smart-filters:prerequisites && node ./packages/public/@babylonjs/smart-filters/dist/utils/buildTools/buildShaders.js ./packages/dev/smartFilterBlocks/src/blocks smart-filters core


> @babylonjs/root@1.0.0 build:assets:smart-filters:prerequisites
> nx run-many --target=build --parallel --maxParallel=6 --projects=@babylonjs/core,@babylonjs/gui,@babylonjs/shared-ui-components,@babylonjs/smart-filters


   ✔  nx run @dev/build-tools:build  [existing outputs match the cache, left as is]
   ✔  nx run @dev/core:build  [existing outputs match the cache, left as is]
   ✔  nx run @lts/core:build  [existing outputs match the cache, left as is]
   ✔  nx run @dev/gui:build  [existing outputs match the cache, left as is]
   ✔  nx run @babylonjs/core:build  [existing outputs match the cache, left as is]
   ✔  nx run @dev/shared-ui-components:build  [existing outputs match the cache, left as is]
   ✔  nx run @lts/gui:build  [existing outputs match the cache, left as is]
   ✔  nx run @babylonjs/smart-filters:build  [existing outputs match the cache, left as is]
   ✔  nx run @babylonjs/gui:build  [existing outputs match the cache, left as is]
   ✔  nx run @babylonjs/shared-ui-components:build  [existing outputs match the cache, left as is]

———————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 NX   Successfully ran target build for 4 projects and 6 tasks they depend on (682ms)

Nx read the output from the cache instead of running the command for 10 out of 10 tasks.

node:path:1277
      validateString(arg, 'path');
      ^

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received undefined
    at Module.join (node:path:1277:7)
    at ConvertShaders (file:///Users/julianharty/sandbox/Babylon.js/packages/public/@babylonjs/smart-filters/dist/utils/buildTools/convertShaders.js:41:42)
    at file:///Users/julianharty/sandbox/Babylon.js/packages/public/@babylonjs/smart-filters/dist/utils/buildTools/buildShaders.js:11:5
    at ModuleJob.run (node:internal/modules/esm/module_job:365:25)
    at async onImport.tracePromise.__proto__ (node:internal/modules/esm/loader:665:26)
    at async asyncRunEntryPointWithESMLoader (node:internal/modules/run_main:99:5) {
  code: 'ERR_INVALID_ARG_TYPE'
}

Node.js v24.3.0
(base) julianharty@Mac Babylon.js % git log -1
commit 398a94e6ef25bcfa95ba11915d4b67358c734466 (HEAD -> master, origin/master, origin/HEAD)
Author: Drew Fillebrown <afillebrown@microsoft.com>
Date:   Mon Nov 3 11:17:42 2025 -0500
8

The cached items are build outputs that were not rebuilt because there were no relevant changes since the last build for those items.

I’m not seeing any unit test messages from your output. Did the unit tests not run due to the error, or did you not include the entire output?

9

They didn’t run as far as I can tell, I’d pasted the entire command and response and included the next command (the git log -1 to confirm I’d got the latest changes from the repo. To be clear, I’m running these on my main macbook (an M4 macbook air).

10

I’ve also run the unit tests again with the latest changes on Ubuntu 24.04 LTS.

Apart from one problem, they run cleanly. (The only change I’ve made is to add:

testPathIgnorePatterns: ["/dist/", "/node_modules/"],

to jest.config.ts to fix the error that was reported when running the unit tests.

11

Hi @Julian_Harty it looks like you’re using nmp v24, but we only support “>=20.11.0 <23.0.0” today. Could you install 22.21.1 or older and try again? Thanks!

12

I’ve installed node 22.21.1 on my macbook using homebrew and added it to the path. (I guessed nmp was related to node, please correct my if I’ve misunderstood.)

This seems to have done the trick, thank you. The tests now run and once I added the testPathIgnorePatterns I mentioned earlier all the unit tests now pass. Thank you very much.

npm run test:unit
<-- cut -->
Test Suites: 63 passed, 63 total
Tests:       1932 passed, 1932 total
Snapshots:   0 total
Time:        15.7 s
Ran all test suites.
(base) julianharty@Mac Babylon.js % node --version
v22.21.1

FWIW on Ubuntu 24.04 LTS Node is version v23.11.1

1 Like