Unit tests fail when run locally

Hi,
i notice that many of Babylon’s UnitTests fail when run locally.

duncan@lapdancer:~/Working/Babylon.js/Tools/Gulp$ gulp tests-babylon-unit
[21:42:27] Using gulpfile ~/Working/Babylon.js/Tools/Gulp/gulpfile.js
[21:42:27] Starting 'tests-babylon-unit'...
[21:42:27] Starting 'tests-unit-transpile'...
[21:42:32] Finished 'tests-unit-transpile' after 4.69 s
[21:42:32] Starting '<anonymous>'...
06 02 2019 21:42:32.359:WARN [config]: urlRoot normalized to "/karma/"
06 02 2019 21:42:32.464:WARN [watcher]: Pattern "/home/duncan/Working/Babylon.js/!./**/*.d.ts" does not match any file.
06 02 2019 21:42:33.949:INFO [karma-server]: Karma v3.1.4 server started at http://0.0.0.0:3000/karma/
06 02 2019 21:42:33.949:INFO [launcher]: Launching browsers PhantomJS with concurrency 1
06 02 2019 21:42:33.952:INFO [launcher]: Starting browser PhantomJS
06 02 2019 21:42:34.301:INFO [PhantomJS 2.1.1 (Linux 0.0.0)]: Connected on socket 58W2TYy-cr48jlf6AAAA with id 59970459
LOG: 'Scripts loaded'
LOG: 'Scripts loaded'
LOG: 'BJS - [21:42:35]: Babylon.js v4.0.0-alpha.22 - Null engine'
PhantomJS 2.1.1 (Linux 0.0.0) Babylon position and rotation #position and rotation: converts between quaternions/euler FAILED
	expect(v.subtract(originalRotation).length() < 0.00001).to.equal is not a function. (In 'expect(v.subtract(originalRotation).length() < 0.00001).to.equal(true)', 'expect(v.subtract(originalRotation).length() < 0.00001).to.equal' is an instance of ProxyObject)
	tests/unit/babylon/src/Mesh/babylon.positionAndRotation.tests.js:39:77
LOG: 'BJS - [21:42:35]: Babylon.js v4.0.0-alpha.22 - Null engine'
PhantomJS 2.1.1 (Linux 0.0.0) Babylon position and rotation #position and rotation: reorders vector in place FAILED
	expect(v.subtract(new BABYLON.Vector3(0.3, 0.2, 0.1)).length() < 0.00001).to.equal is not a function. (In 'expect(v.subtract(new BABYLON.Vector3(0.3, 0.2, 0.1)).length() < 0.00001).to.equal(true)', 'expect(v.subtract(new BABYLON.Vector3(0.3, 0.2, 0.1)).length() < 0.00001).to.equal' is an instance of ProxyObject)
	tests/unit/babylon/src/Mesh/babylon.positionAndRotation.tests.js:45:95
LOG: 'BJS - [21:42:35]: Babylon.js v4.0.0-alpha.22 - Null engine'
[...SNIP...]
PhantomJS 2.1.1 (Linux 0.0.0) Babylon.Promise #Error handling should correctly handle exceptions in a reject handler FAILED
	expect(error.constructor).to.equal is not a function. (In 'expect(error.constructor).to.equal(Error)', 'expect(error.constructor).to.equal' is an instance of ProxyObject)
	tests/unit/babylon/src/Tools/babylon.promise.tests.js:316:55
	_reject@http://localhost:3000/dist/preview%20release/babylon.max.js:100913:33
	_reject@http://localhost:3000/dist/preview%20release/babylon.max.js:100926:30
	http://localhost:3000/dist/preview%20release/babylon.max.js:100849:39
PhantomJS 2.1.1 (Linux 0.0.0): Executed 71 of 71 (54 FAILED) (22.731 secs / NaN secs)
TOTAL: 54 FAILED, 17 SUCCESS
[21:42:57] '<anonymous>' errored after 25 s
[21:42:57] Error: 1
    at formatError (/usr/local/lib/node_modules/gulp/node_modules/gulp-cli/lib/versioned/^4.0.0/format-error.js:20:10)
    at Gulp.<anonymous> (/usr/local/lib/node_modules/gulp/node_modules/gulp-cli/lib/versioned/^4.0.0/log/events.js:31:15)
    at Gulp.emit (events.js:187:15)
    at Gulp.EventEmitter.emit (domain.js:442:20)
    at Object.error (/home/duncan/Working/Babylon.js/node_modules/undertaker/lib/helpers/createExtensions.js:61:10)
    at handler (/home/duncan/Working/Babylon.js/node_modules/now-and-later/lib/mapSeries.js:43:14)
    at f (/home/duncan/Working/Babylon.js/node_modules/once/once.js:25:25)
    at f (/home/duncan/Working/Babylon.js/node_modules/once/once.js:25:25)
    at tryCatch (/home/duncan/Working/Babylon.js/node_modules/async-done/index.js:24:15)
    at done (/home/duncan/Working/Babylon.js/node_modules/async-done/index.js:40:12)
    at removeAllListeners (/home/duncan/Working/Babylon.js/node_modules/karma/lib/server.js:354:9)
    at webServer.close (/home/duncan/Working/Babylon.js/node_modules/karma/lib/server.js:365:13)
    at Server.close (net.js:1565:9)
    at Object.onceWrapper (events.js:273:13)
    at Server.emit (events.js:187:15)
    at Server.EventEmitter.emit (domain.js:460:23)
    at emitCloseNT (net.js:1618:8)
    at process._tickCallback (internal/process/next_tick.js:63:19)
[21:42:57] 'tests-babylon-unit' errored after 30 s

All the failed tests follow the same pattern:
expect(true).to.equal is not a function. (In 'expect(true).to.equal(true)', 'expect(true).to.equal' is an instance of ProxyObject)

It appears PhantomJS is no longer supported.
This blog post details exactly the same issue: Headless Firefox with Karma | The Cloistered Monkey
If i implement the fix described in that post i get 100% success rate…

So,
i could easily submit a PR to implement the fix that works for me but…
does anyone know why my system fails but the Babylon build servers do not?
Do we think the build servers will fail due to this issue sometime soon or is this something weird with my setup?

Here’s what i’m running against:

duncan@lapdancer:~/Working/Babylon.js$ uname -a
Linux lapdancer 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

duncan@lapdancer:~/Working/Babylon.js$ npm list --depth=0 | grep typescript
├── awesome-typescript-loader@5.2.1
├── gulp-typescript@4.0.2
├── typescript@3.3.1

duncan@lapdancer:~/Working/Babylon.js$ npm list --depth=0 | grep phantom
├── karma-phantomjs-launcher@1.0.4
├── phantomjs-prebuilt@2.1.16

duncan@lapdancer:~/Working/Babylon.js$ npm list --depth=0 | grep chai
├── @types/chai@4.1.7
├── chai@4.2.0
├── karma-chai@0.1.0

It does not fail on my computer either
Maybe this is due to your Linux version

oh and please send the PR. If we can remove phantomJS :slight_smile:

More likely the version of one of the npm packages.

Anyway, FirefoxHeadless seems like the way to go here anyway.

I don’t know whether this is still relevant to this project, but I encountered this today in a different context. The issue turned out to be a system-wide install of PhantomJS in /usr/bin/phantomjs on Linux. Removing the system-wide install of PhantomJS, removing and then re-installing dependencies so that the “phantomjs-prebuilt” package downloaded a fresh copy of the binary solved the problem.

1 Like