Visualization tests

Questions
1

@Evgeni_Popov @RaananW @sebavan

Hello!

How do you guys take the screenshots for the visualization tests? Do you use the Screenshot function in the debug layer? Capture with RTT? Code snippet? Do you go to fullscreen, screenshot the whole screen and you resize the screenshot afterwards to 600x400 or is there something I am not aware of how to make the screenshot in fullscreen directly to 600x400 more easily? Must it be 600x400 or can I use a fullscreen FullHD screenshot?

Thank you very much!

My first try looks like this :see_no_evil:

image
image1212×1890 133 KB

EDIT:
Guys, would someone please give me detailed information how to create a new Visualization test from A to Z? The documentation is outdated and even after I created a 600x400 png (fullscreen screenshot + photoshop - I don’t even know if I need to center the image :see_no_evil: ), edited the config, I couldn’t get the tests running (wrong url in the docs). Enlighten me please! :pray: Thank you!

2

It can’t be simpler:

  1. Create a playground
  2. Add the playground the config.json - Babylon.js/packages/tools/tests/test/visualization/config.json at master · BabylonJS/Babylon.js · GitHub
  3. run visualization tests with the -u parameter. This will generate the screenshot for you. You can run your test only, but running npm run test:visualization -- -i "webgl2" -t "Your Test Name" -u

I guess you are referring to this page that is outdated: Validation tests | Babylon.js Documentation (babylonjs.com). You are totally right. I’ll take it as a task to update it.

2 Likes
3

This rocks! :metal:

Thanks a lot for the information!

1 Like
5

I love this :heart:

webgl2 without problems, but failed to create the screenshot on webgpu:

Evaluation failed: TypeError: navigator.gpu.getPreferredCanvasFormat is not a function

      at new WebGPUEngine (../dev/core/src/Engines/webgpuEngine.ts:361:76)
      at __puppeteer_evaluation_script__:52:24
      at ExecutionContext._evaluateInternal (node_modules/puppeteer/src/common/ExecutionContext.ts:273:13)
      at async ExecutionContext.evaluate (node_modules/puppeteer/src/common/ExecutionContext.ts:140:12)

I just changed the -i parameter to webgpu and ofcourse edited the webgpu config.

Do I need to do anything else?

6

oh, that’s probably related to the puppeteer version. can you try fetching/merging from master and running npm install?

1 Like
7

and the screenshot should probably be the same. there is no reason to generate two. so if this test is running on webgl2 - this should also fit the screenshot of webgpu

8

I was curios whether it’s just an issue on Mac so I’ve tried that on PC just right now. I have the most recent version of the repo, did an npm i, but no luck. Same error.

9

:face_with_monocle: yerrr! putting the same image in the webgpu folder :wink:

10

quick question for @Evgeni_Popov - did you have this issue when using vis-tests?

11

Sacre bleu!!!

I just added the png and modified the config file and now:

image
image1388×324 32.9 KB

https://dev.azure.com/babylonjs/ContinousIntegration/_build/results?buildId=26008&view=results

12

that’s node.js init on azure…

every time there is a new version of node azure goes a little crazy. I re-ran the tests, I hope it will pass this time.

1 Like
13

We use Azure Devops at work as well so I am already immune against these quirks :joy:

1 Like
14

When testing for WebGPU, I have to manually change the path to Chrome in jest-puppeteer.config.js because it seems the Chrome version embedded with puppeteer does not support WebGPU.

So, I’m doing something like:

const browserPath = process.env.BROWSER_PATH || "C:/Program Files (x86)/Google/Chrome/Application/chrome.exe";

If I want to test with Canary instead, I use the path C:/Users/micro/AppData/Local/Google/Chrome SxS/Application/chrome.exe.

Note that the image does not need to be copied in ReferenceImages/webgpu/: as your test is declared in config.json, the picture must be copied in ReferenceImages/.

This part of the doc is still relevant (except for the url) to explain what is a test list and that pictures are related to a list.

A test list is orthogonal to the engine you use to launch the tests: you can test webgpu.json with WebGL2 and config.json with WebGPU.

By default, when using WebGL2, it’s config.json that is used, and webgpu.json for WebGPU. But you can change this by editing tests/test/visualization/visualization.webgl2.test.ts or visualization.webgpu.test.ts: the second parameter of the evaluateTests call is the name of the list, without the .json part.

In any case, I think we are going to add all the tests from webgpu.json to config.json, now that the WebGPU engine is up to date.

3 Likes
15

Thank you very much!

16

Yes, this is obvious. :sunglasses:

EDIT: Just to be precise, I was trying to figure out what and where to put before I got your answers, so the image may have appeared in one of my commits in the webgpu folder as well.

1 Like