Support for offscreen canvas

Current solution is little bit ugly. For example class HTMLElement {} line is needed to work in stable version but it will be a bit confusing for documentation.

I was wondering, could we have offscreen canvas using worker-threads of nodeJS?

nodeJS does not support offscreen canvas :slight_smile:

1 Like

Thanks, following the doc then :wink:

Another question about this great feature.

If we have several canvas on one page, does that mean that each canvas will have its own worker and thus its own thread?

If you use offscreen canvas: yes

1 Like

Wow :scream:

I noticed that the function

Engine.isSupported();

does not work properly inside a worker.

Based on @kyptov awesome work, I build a typescript version and added stuff.
You can check it here: https://github.com/nakerwave/naker-services/tree/Adding_offscreenCanvas/Viewer

screen and worker are the two classes making the connection.
As you requested @Deltakosh I guess a link to these files would be great in the documentation?

This is not possible to not have webGL if offscreen canvas is supported :slight_smile:

We tested this feature on mobile.

And we get this message with an iphone

So I guess offscreencanvas are not yet available on some smartphone?

Plus I use that test which I found in your demo page @Deltakosh

if (‘OffscreenCanvas’ in window)

to make sure I can use offscreen canvas but again this test doesn’t work on mobile
From the bug above I guess we should also test ?

if (‘transferControlToOffsceen’ in canvas)

If right, I would add that to the documentation. Pretty sure a lot of people will have that issue too! :wink:

1 Like

It depends on the phone:) some supports a partial version of the offscreen canvas so yeah you can definitely update the doc :wink:

Hi. I saw you added use of ImageBitmap instead of Image class in a worker, but there is an issue. According to the WebGl spec

UNPACK_FLIP_Y_WEBGL, UNPACK_PREMULTIPLY_ALPHA_WEBGL , UNPACK_COLORSPACE_CONVERSION_WEBGL …
If the TexImageSource is an ImageBitmap, then these three parameters will be ignored. Instead the equivalent ImageBitmapOptions should be used to create an ImageBitmap with the desired format.

invertY has default true in Texture constructor and it cause that some ImageBitmap a flipped.
https://www.babylonjs-playground.com/#08GVKY#8 (to restore Image class its need to reload page)

Yes this is a very good point!

We have no choice but using ImageBitmap. Then you are forced to not use invertY and instead either:

  • invert source image
  • use texture.vScale with -1
2 Likes

Just thought about something similar and searched the web for this. Surprised to see this in babylon forums, since I thought I read all about this topic here. Super nice and simple implementation! Going to try this. Thank you!
Please @Deltakosh please place at least a link in the docs to this post :slight_smile: (Comparing BabylonJS:master...dennemark:patch-1 · BabylonJS/Documentation · GitHub)

2 Likes

Let me add @PatrickRyan our doc keeper :slight_smile:

I’ve found @kyptov event handling methods to still be relevant for passing UI events back and forth with an offscreenCanvas. I created this code sandbox demo to show how I implemented his code in my project. It’s an attempt to put the UI specific code into their own files so they’re not interfering with other project code.
Much appriciation to @kyptov

1 Like

There are some very interesting solutions in this post and I managed to get this working for our app but with all the window overrides and event hacks it feels like this will „break soon“ :upside_down_face: any plans from the Babylon-Team to support this in a clean way? We are thinking about a flag for Babylon on init that just abstractes something like the above away and is integrated with Babylon at core level? One of many problems i see is that Babylon references window multiple times in its core this could be abstracted away as a global context behaving differently in offscreen and window canvas for example…

Found this: GitHub - ampproject/worker-dom: The same DOM API and Frameworks you know, but in a Web Worker.

It seems its possible to *upgrade dom Elements like Canvas so that events etc. are managed and controllable from within a worker. If you pass your offscreencanvas to this worker and run Babylon in it - Babylon may use the dom events from the above lib and booom…

There is the class BABYLON.AutoReleaseWorkerPool - Babylon.js docs - which is designed to work with workers :slight_smile:

86 issues. Basic stuff as the query selector is still not fully supported. Issues with fetch, etc.
The project is very cool however it it just has to mature. My second concern is security.