HtmlElementTexture creates a baseTexture instead of a Texture

I am trying to create a canvas element from scratch and use it as a texture.

https://playground.babylonjs.com/#21QRSK#151

In the playground, if I load the texture from HtmlElementTexture:

    //----- Switch these two lines
    //mat.diffuseTexture = new BABYLON.Texture("https://upload.wikimedia.org/wikipedia/commons/thumb/f/fd/UV_checker_Map_byValle.jpg/1920px-UV_checker_Map_byValle.jpg", scene);
    mat.diffuseTexture = texture
    //-----------------------------

none of the UV commands has any effect. But if I use the “new BABYLON.Texture” command, the UV commands work fine.

Also, I noticed that the HtmlElementTexture creates a Class “baseTexture” for the texture, but the other command creates a Class of “texture”

My objective is to create a Canvas Element, place it in a texture, then control how that texture is aligned on the faces of meshes.

Thanks

By design htmlElementTexture is pretty tiny as it is use to not have maths classes and such being bundled with it.

In your case you could maybe rely on a dynamicTexture which also wraps a canvas or you need to override the getTextureMatrix function like the texture class does:

public getTextureMatrix(uBase = 1): Matrix {
        if (
            this.uOffset === this._cachedUOffset &&
            this.vOffset === this._cachedVOffset &&
            this.uScale * uBase === this._cachedUScale &&
            this.vScale === this._cachedVScale &&
            this.uAng === this._cachedUAng &&
            this.vAng === this._cachedVAng &&
            this.wAng === this._cachedWAng &&
            this.uRotationCenter === this._cachedURotationCenter &&
            this.vRotationCenter === this._cachedVRotationCenter &&
            this.wRotationCenter === this._cachedWRotationCenter &&
            this.homogeneousRotationInUVTransform === this._cachedHomogeneousRotationInUVTransform) {
            return this._cachedTextureMatrix!;
        }

        this._cachedUOffset = this.uOffset;
        this._cachedVOffset = this.vOffset;
        this._cachedUScale = this.uScale * uBase;
        this._cachedVScale = this.vScale;
        this._cachedUAng = this.uAng;
        this._cachedVAng = this.vAng;
        this._cachedWAng = this.wAng;
        this._cachedURotationCenter = this.uRotationCenter;
        this._cachedVRotationCenter = this.vRotationCenter;
        this._cachedWRotationCenter = this.wRotationCenter;
        this._cachedHomogeneousRotationInUVTransform = this.homogeneousRotationInUVTransform;

        if (!this._cachedTextureMatrix || !this._rowGenerationMatrix) {
            this._cachedTextureMatrix = Matrix.Zero();
            this._rowGenerationMatrix = new Matrix();
            this._t0 = Vector3.Zero();
            this._t1 = Vector3.Zero();
            this._t2 = Vector3.Zero();
        }

        Matrix.RotationYawPitchRollToRef(this.vAng, this.uAng, this.wAng, this._rowGenerationMatrix!);

        if (this.homogeneousRotationInUVTransform) {
            Matrix.TranslationToRef(-this._cachedURotationCenter, -this._cachedVRotationCenter, -this._cachedWRotationCenter, TmpVectors.Matrix[0]);
            Matrix.TranslationToRef(this._cachedURotationCenter, this._cachedVRotationCenter, this._cachedWRotationCenter, TmpVectors.Matrix[1]);
            Matrix.ScalingToRef(this._cachedUScale, this._cachedVScale, 0, TmpVectors.Matrix[2]);
            Matrix.TranslationToRef(this._cachedUOffset, this._cachedVOffset, 0, TmpVectors.Matrix[3]);

            TmpVectors.Matrix[0].multiplyToRef(this._rowGenerationMatrix!, this._cachedTextureMatrix);
            this._cachedTextureMatrix.multiplyToRef(TmpVectors.Matrix[1], this._cachedTextureMatrix);
            this._cachedTextureMatrix.multiplyToRef(TmpVectors.Matrix[2], this._cachedTextureMatrix);
            this._cachedTextureMatrix.multiplyToRef(TmpVectors.Matrix[3], this._cachedTextureMatrix);

            // copy the translation row to the 3rd row of the matrix so that we don't need to update the shaders (which expects the translation to be on the 3rd row)
            this._cachedTextureMatrix.setRowFromFloats(2, this._cachedTextureMatrix.m[12], this._cachedTextureMatrix.m[13], this._cachedTextureMatrix.m[14], 1);
        } else {
            this._prepareRowForTextureGeneration(0, 0, 0, this._t0!);
            this._prepareRowForTextureGeneration(1.0, 0, 0, this._t1!);
            this._prepareRowForTextureGeneration(0, 1.0, 0, this._t2!);

            this._t1!.subtractInPlace(this._t0!);
            this._t2!.subtractInPlace(this._t0!);

            Matrix.FromValuesToRef(
                this._t1!.x, this._t1!.y, this._t1!.z, 0.0,
                this._t2!.x, this._t2!.y, this._t2!.z, 0.0,
                this._t0!.x, this._t0!.y, this._t0!.z, 0.0,
                0.0, 0.0, 0.0, 1.0,
                this._cachedTextureMatrix
            );
        }

        let scene = this.getScene();

        if (!scene) {
            return this._cachedTextureMatrix;
        }

        // We flag the materials that are using this texture as "texture dirty" because depending on the fact that the matrix is the identity or not, some defines
        // will get different values (see MaterialHelper.PrepareDefinesForMergedUV), meaning we should regenerate the effect accordingly
        scene.markAllMaterialsAsDirty(Constants.MATERIAL_TextureDirtyFlag, (mat) => {
            return mat.hasTexture(this);
        });

        return this._cachedTextureMatrix;
    }
1 Like

Thanks for the quick response!

I don’t really understand how to do what you are suggesting. I have solved the problem, but it feels like a hack.

I convert the canvas into a dataURL and then use the BABYLON.Texture.CreateFromBase64String.

https://playground.babylonjs.com/#21QRSK#152

    var textureInBase64 = window.drawCanvas.toDataURL()
    var texture = new BABYLON.Texture.CreateFromBase64String(
        textureInBase64,
        "theTexture",
        scene
    )

I have been confused about canvas elements for a while now. It seems like there should be an easy way to use a canvas element as an image in textures. Most of the use cases in the Playground I found are focused on displaying canvas elements over and over as they change. This is certainly interesting, but my use case tends more toward having some other system (flow chart or something) create an image that I want to include in my Babylon scene. Many times, those images are in 2d Canvas.

I understand there are CORS rules and such that cause dirty conditions. And maybe my solution is how it needs to be done. It seems inefficient to me to turn a canvas element into a base64 string and then convert it back to a canvas element.

Thanks again for your help.

One further note: I have benchmarked this and in my particular case, the conversion (which is quite a large canvas element) takes almost a second. This is a pretty high penalty when I have a canvas element already created and ready to use and all I want to do is push it to a texture.

Doesn’t this method suit your needs - https://playground.babylonjs.com/#5ZCGRM#3 ?

Thanks.

I am using this approach for some things.

This pattern is to create a canvas element in babylon, then grab a context and then write to the canvas element. Then canvas.update(). Actually, this might work for this particular case, because I am writing the canvas code by hand. But there are situations where another process (one that I didn’t write) creates the canvas element in a complete way. And then we want to bring it into the Scene after it is created.

Thanks for this suggestion. It might work better for me.

You can do it with a dynamicTexture:

https://playground.babylonjs.com/#21QRSK#154

Perfect!

Thank you so much.

1 Like