Drawing on dynamic texture with "drawImage" in webworker/offscreen canvas

Questions
1

I have a problem with using drawImage in webworker, mainly regarding the loading of the image.

const img = new Image();
img.src = 'PATH TO IMAGE';
img.onload = function() {

ctx.drawImage(this, 0, 0);
myDynamicTexture.update();

This form of loading images doesn’t work, I guess because webworker (babylon offscreen canvas) do not have access to the DOM.

2

You can create an offscreen canvas and transfer it to the WebWorker

3

That’s what I did basically the problem is the creation of the Image itself and loading from the webworker context. Image is a DOM type in typescript that I cannot access, also even if I add DOM to my libs in tsconfig, the load still fails.

4

This is how would I proceed if I needed to stuck with loading the data in the WebWorker:

Load the image using fetch into a blob in the WebWorker.
Send the blob data back to the main thread via postMessage.
Create the Image in the main thread using the received data.

Or I would choose a totaly different approach to solve my problem.

1 Like
5

Here is the example of worker which gets imageURL and returns RawTexture as Uint8Array.

import * as BABYLON from "@babylonjs/core";

onmessage = function (evt) {
  let canvas;
  let scene;

  if (evt.data.canvas) {
    canvas = evt.data.canvas;

    const engine = new BABYLON.Engine(canvas, true);
    scene = new BABYLON.Scene(engine);
  }

  if (evt.data.imageURL) {
    const imageURL = evt.data.imageURL;

    const customImage = new BABYLON.Texture(imageURL, scene);

    BABYLON.RawTexture.WhenAllReady([customImage], () => {
      customImage.readPixels()?.then((bv) => {
        postMessage({
          bv,
        });
      });
    });
  }
};

The main thread

offscreen = _canvas.transferControlToOffscreen();

        newWebWorker = new Worker(
          new URL("./worker.js", import.meta.url)
        ) as any;
        newWebWorker.postMessage({ canvas: offscreen }, [offscreen]);
      }
      const texture = BABYLON.RawTexture.CreateRGBATexture(
        null,
        img.width,
        img.height,
        scene,
        undefined,
        true
      );

      newWebWorker.postMessage({ imageURL });

      newWebWorker.onmessage = (e: any) => {
        const { bv } = e.data;
        texture.update(bv);
  };
2 Likes