BabylongJS 5.0.0-beta.8 make our custom WebGPU renderer goes blank

Bugs
1

I have custom engine with core webgpu implementation and Babylon WebGPU renderer.
Both engines renders the webgpu visualisation output on canvas side-by-side.

Everything was working fine yesterday evening. This morning all of sudden I noticed that out of the two renderers my custom webgpu rendering goes blank, however the BabylonJS rendering appearing intact.

Further, upon investigation I found the new beta version 5.0.0-beta.8 is available and is causing this behaviour. The 5.0.0-beta.7 works fine.

My custom webgpu engine uses Webgpu types @webgpu/types": "0.1.9"

With 5.0.0-beta.8 here was an error reported in my custom code which I was not expecting as an update of BabylonJS beta.

Following was the error reported:-

ERROR in /Users/parminder.singh/Dev/giraphics/gitlab/typescript-lib-example/src/renderer/webgpu/renderer.ts
./src/renderer/webgpu/renderer.ts 195:8-23
[tsl] ERROR in /Users/ps/Dev/giraphics/gitlab/typescript-lib-example/src/renderer/webgpu/renderer.ts(195,9)
      TS2741: Property 'loadOp' is missing in type '{ view: GPUTextureView; loadValue: { r: number; g: number; b: number; a: number; }; storeOp: "store"; }' but required in type 'GPURenderPassColorAttachment'.
 @ ./src/graffiti.ts 32:0-69 32:0-69

ERROR in /Users/ps/Dev/giraphics/gitlab/typescript-lib-example/src/renderer/webgpu/renderer.ts
./src/renderer/webgpu/renderer.ts 202:10-25
[tsl] ERROR in /Users/parminder.singh/Dev/giraphics/gitlab/typescript-lib-example/src/renderer/webgpu/renderer.ts(202,11)
      TS2739: Type '{ view: GPUTextureView; depthLoadValue: number; depthStoreOp: "store"; stencilLoadValue: "load"; stencilStoreOp: "store"; }' is missing the following properties from type 'GPURenderPassDepthStencilAttachment': depthLoadOp, stencilLoadOp
 @ ./src/graffiti.ts 32:0-69 32:0-69

From the below code:-

    let colorAttachment: GPURenderPassColorAttachment = {
      view: this.colorTextureView,
      loadValue: { r: this._backGroundColor[0], g: this._backGroundColor[1], b: this._backGroundColor[2], a: this._backGroundColor[3] },
      storeOp: 'store',
    };

    const depthAttachment: GPURenderPassDepthStencilAttachment = {
      view: this.depthTextureView,
      depthLoadValue: 1,
      depthStoreOp: 'store',
      stencilLoadValue: 'load',
      stencilStoreOp: 'store',
    };

Which I fixed as below:-

    let colorAttachment: GPURenderPassColorAttachment = {
      view: this.colorTextureView,
      loadValue: { r: this._backGroundColor[0], g: this._backGroundColor[1], b: this._backGroundColor[2], a: this._backGroundColor[3] },
      storeOp: 'store',
      loadOp: 'clear',  // Babylon 5.0.0-beta.8 reports this in my custom engine code, why? 
    };

    const depthAttachment: GPURenderPassDepthStencilAttachment = {
      view: this.depthTextureView,
      depthLoadValue: 1,
      depthLoadOp: 'clear', // Babylon 5.0.0-beta.8 reports this in my custom engine code, why? 
      depthStoreOp: 'store',
      stencilLoadOp: 'clear', // Babylon 5.0.0-beta.8 reports this in my custom engine code, why? 
      stencilLoadValue: 'load',
      stencilStoreOp: 'store',
    };```

Here is output before with *Babylon 5.0.0-beta.7* and after with *Babylon 5.0.0-beta.8* 
![Babylon 5.0.0-beta.7 - Works Good|690x310](upload://pqOe5jtmXfYGNQHto6q5lFiUIQD.jpeg)
![Babylong 5.0.0-beta.8 - Broken, bland output|690x310](upload://l2Ktda3crQzzzrgXo9OfNPI3LnO.png)

Query 1: I am not sure why my WebGPU instance got affected by BabylonJS beta upgrade, its a separate engine with its own context. 

Query 2: How can make it fix? is my added code for fixing error looks correct to you, I have tried **'clear'** and **'load'** values but the output remains blank. Kindly suggest.
2

Put each canvas in an iframe. If you use type=“module” as an attribute on the iframe tag, the isolates share es modules so you will actually get better performance too

3

cc @Evgeni_Popov

4

I’m not sure how can Babylon.js make your engine fail?

I don’t think your problem is related to Babylon.js, it’s probably related to a change in the WebGPU spec that happened lately regarding clearing the color/depth/stencil attachments: see the GPURenderPassColorAttachment and GPURenderPassDepthStencilAttachment interfaces, and more precisely the new clearValue / depthClearValue / stencilClearValue properties. Also, the loadOp property of these interfaces now doesn’t accept a color anymore.

5

giraphics, are you sure the bundler you’re using is emitting code when typecheck fails?

6

Thanks for the performance tip Jeremy I will give it a try for better performance.

However, this issue is more related to the module itself.
I have a module where my custom webgpu code and babylon webgpu resides side-by-side.
I found instances where standard data types and code is influenced BabylonJS beta version.

Yesterday, I logged the first bug https://forum.babylonjs.com/t/webgpu-types-inter/27717/1
which seems Akismet has temporarily hidden my post (https://forum.babylonjs.com/t/akismet-has-temporarily-hidden-your-post/27718) waiting for the team to review and appear on the forum.

7

Yes, there are chances where my current code is making the output appear blanks.
Let me review the webgpu type version one-by-one and the same process for babylon js.

In the meantime, I have another bug that seems is related to the current issue, it’s under https://forum.babylonjs.com/t/akismet-has-temporarily-hidden-your-post/27718, it is being hidden by Akismet has temporarily.

8

Types for webgpu get out of sync with browser implementations all the time. Plus, which browser vendor and version is correct? There is no proper answer, so you cant have your bundler fail on type errors. Types dont really matter, the only types that affect runtime code are enums, thats actually why i said put them in different frames so if there were conflicting enum values they wouldnt over write each other, but i checked the types, there are no enums in the webgpu types, so it actually doesnt matter

9

@Evgeni_Popov I investigated the matter for both engines(custom & webgpu) in chronological order:-

Scenario 1:
Orignal configuration, “babylonjs/core”: “5.0.0-beta.7” and “webgpu/types”: “0.1.9”, everthing is good.

(1a) Test my custom webgpu engine with latest webgpu types upgrade: “webgpu/types”: “0.1.13”
Result:
(i) Error Count: 1

[tsl] ERROR in /src/renderer/webgpu/renderer.ts(195,9)
      TS2741: Property 'loadOp' is missing in type '{ view: GPUTextureView; loadValue: { r: number; g: number; b: number; a: number; }; storeOp: "store"; }' but required in type 'GPURenderPassColorAttachment'.

(ii) Fix: loadOp field provided GPURenderPassColorAttachment

[Note]: This is expected here because my custom engine making using of webgpu type and the upgrade indicates the error in my code.

Output:
Linux: Version 100.0.4895.0 (Developer Build) (64-bit)
Customer Renderer works: Good
BabylonJS Renderer works: Blank, (No, worries, assuming it expected as Linux is not supported)

Mac: Version 97.0.4682.3 (Official Build) dev (64-bit)
Customer Renderer works: Good
BabylonJS Renderer works: Good

(1b) Now, Upgraded: babylon to “@babylonjs/core”: “5.0.0-beta.8”,
Result:
(i) Error Count: 0

Output:
Same as 1a Output everything is good with “babylonjs/core”: “5.0.0-beta.8” and “webgpu/types”: “0.1.13”

Scenario 2:
Again, reset to orignal configuration, “babylonjs/core”: “5.0.0-beta.7” and “webgpu/types”: “0.1.9”, everthing is good.

But this time we will upgrade the BabylonJS first and webgpu types later to make it similar to Scenario 1.

(2a) Upgraded: babylon to “@babylonjs/core”: “5.0.0-beta.8”,
Result:
(i) Error Count: 2

[tsl] ERROR in /src/renderer/webgpu/renderer.ts(195,9)
      TS2741: Property 'loadOp' is missing in type '{ view: GPUTextureView; loadValue: { r: number; g: number; b: number; a: number; }; storeOp: "store"; }' but required in type 'GPURenderPassColorAttachment'.

and

[tsl] ERROR in /src/renderer/webgpu/renderer.ts(202,11)
      TS2739: Type '{ view: GPUTextureView; depthLoadValue: number; depthStoreOp: "store"; stencilLoadValue: "load"; stencilStoreOp: "store"; }' is missing the following properties from type 'GPURenderPassDepthStencilAttachment': depthLoadOp, stencilLoadOp

(ii)
Fix 1: loadOp field provided GPURenderPassColorAttachment.
Fix 2: depthLoadOp and stencilLoadOp field provided GPURenderPassDepthStencilAttachment.

[Note 1]: This is not expected here because my custom engine webgpu type are not upgraded explicitly.
If I click on the reported error 1 in the vscode the IntelliSense took me to follow the declaration:

Property 'loadOp' is missing in type '{ view: GPUTextureView; loadValue: { r: number; g: number; b: number; a: number; }; storeOp: "store"; }' but required in type 'GPURenderPassColorAttachment'.ts(2741)
engine.d.ts(2169, 5): 'loadOp' is declared here.
let colorAttachment: GPURenderPassColorAttachment

engine.d.ts is from BabylonJS file, whereas I expect my IntelliSense to point webgpu types declaration.

[Note 2]: On, the other hand, I also tested if my webgpu type version does affect the BabylonJS which I did not find a case. I upgraded “webgpu/types”: “0.1.9” to “webgpu/types”: “0.1.13” one by one but BabylonJS does not report any error which I believe is the expected behavior. now the vice-versa is not true for BabylonJS upgrade.

Output:
Linux: Version 100.0.4895.0 (Developer Build) (64-bit)
Customer Renderer works: Blank,not expected
BabylonJS Renderer works: Blank, (No, worries, assuming it expected as Linux is not supported)

Mac: Version 97.0.4682.3 (Official Build) dev (64-bit)
Customer Renderer works: : Blank, not expected
BabylonJS Renderer works: Good

(2b) Now, Upgraded custom webgpu engine with latest webgpu types upgrade: “webgpu/types”: “0.1.13”
Result:
(i) Error Count: 0

Output:
Same as 2a Output, my custom engine is rendering blank.

Confliction:
(1) Scenario 1 and 2 are same configuration wise but Scenario 2 is fixed with GPURenderPassDepthStencilAttachment error. If I revert back the fix 2, my custom engine works and with (2b) step it does not report any GPURenderPassDepthStencilAttachment error.

(2) The other influence I saw is I notice is reported in https://forum.babylonjs.com/t/akismet-has-temporarily-hidden-your-post/27718 here two webgpu data types are conflicted with the types declaration in engine.d.ts is from BabylonJS file.

(a) NavigatorGPU.gpu
(b) renderPipeline.getBindGroupLayout(0)

Original code:-
(a) const entry: GPU = navigator.gpu;
(b) layout: this.renderPipeline.getBindGroupLayout(0),

The fix all over my code was
(a) const entry: GPU = (navigator as unknown as NavigatorGPU).gpu;
(b) layout: this.renderPipeline.getBindGroupLayout(0) as unknown as GPUBindGroupLayout

10

Thanks, @jeremy-coleman yes it may not matter for my library code(it’s an npm package). The library code takes the canvas handle for the sake of rendering surface purposes. The UI or View is client-specific, they use our lib and pass the canvas handle in the engine. The error I am getting is way before producing the bundle.

I checked my code it is not producing any bundle if the type checking fails as you asked. In the above code, I have figured out the solution to make it work but I want to know how to avoid Babylon webgpu declaration in my custom code.

Hierarchically, my engine code exists in a separate folder and does not include anything from BabylonJS.

src
 |
 |----renderer
 |		 |
 |		 |----babylon
 |               | 	 |
 |		 |       |----renderer.ts
 |		 |
 |		 |----babylon
 |      		 |
 |		         |----renderer.ts
 |		         |----scene.ts
 |		         |----etc.
11 
src
 |
 |----renderer
 |		 |
 |		 |----babylon
 |       | 		 |
 |		 |       |----renderer.ts
 |		 |
 |		 |----custom-engine
 |      		 |
 |		         |----renderer.ts
 |		         |----scene.ts
 |		         |----etc.
12

It would seem there’s a conflict between the webgpu/types package you use and our own webgpu.d.ts declaration file (found in LibDeclarations/webgpu.d.ts).

However, I don’t see how a declaration file can change the behaviour of the program, as it is simply that, some declarations that don’t generate code by themselves… It’s only a mean for typescript to check things.

Can’t you separate the source code for your engine from the source code of Babylon and make two separate projects? That way you can have separate configurations for both and they won’t conflict with each other.

13

Thank you @Evgeni_Popov and @jeremy-coleman for your help.
Curious to know why webgpu/types are being removed from BabylonJS. At least, it will help in learning the webgpu type library version used in Babylon JS is using, it may fix the problem if webgpu/type library is used in Babylon JS?

For now, I can live with the code as the two declarations are goes hand-in-hand. We may like to separate the implementation once the code is mature, it seems the only option for now is to safely maintain the two code bases. Hopefully, we may not require our custom engine if met the performance requirement from Babylon JS. Thanks again for your help.

14

We were using webgpu/types at the very beginning but I recall it was not easy to work with (the spec could have evolved but the file not yet), and also we had some other problems (can’t remember which ones…). So we use our own file type and maintain it in sync with the spec.

15

Sure, Thank you for your reply, closing this issue.