[Question about how to copy all texture of DynamicTexture]

Hi everyone, i am creating a feature whiteboard in babylonjs.
i created the plane and asign to this plane a Dynamictexture. I add eventListener to the mainCanvas and at event pointerMove, i getContext of this Dynamictexture and fill by my pencil color. After that i call texture.update to update my Dynamictexture. It work fine if i play offline on my local device. But now i want create online multiple whiteboard. How can i get all content of my DynamicTexute everytime when pointerMove event trigger in function onPointerMove to pass through socket event and update at another user screen. I also try to Json.Stringify the groundTexture to pass to socket event but get the error like that:
image
Maybe cannot JSON.stringify Dynamictexture interface
This is my code

import { ActionManager, Color3, DynamicTexture, ExecuteCodeAction, Mesh, MeshBuilder, Scene, StandardMaterial, Vector3 } from '@babylonjs/core'

import { MainScene } from '../../pages/RoomPlay/MainRoom/MainScene'
export const DefaultSetting = {
  boardColor: '#ffffff',
  pencilColor: '#000000',
  pencilRadius: 5,
  cursor: 'crosshair',
  width: 4,
  height: 2
}
export class WhiteBoard {
  private plane: Mesh
  private WhiteBoardControlPlane: Mesh
  private scene: Scene
  private name: string
  private position: Vector3
  private rotation: Vector3
  private scale: number
  private tempMat: any
  public itemId: string
  private mainCanvas: HTMLCanvasElement
  private isDrawing: boolean
  private mainScene: MainScene
  private groundTexture: DynamicTexture
  constructor(
    _scene: Scene,
    _name: string,
    _position: Vector3,
    _rotation: Vector3,
    _scale: number,
    _itemId: string,
    canvas: HTMLCanvasElement,
    mc: MainScene
  ) {
    this.name = _name
    this.position = _position
    this.rotation = _rotation
    this.scale = _scale
    this.scene = _scene
    this.mainCanvas = canvas
    this.isDrawing = false
    this.mainScene = mc
  }

  public createTexture = async () => {
    let _this = this
    this.plane = MeshBuilder.CreatePlane(
      this.name,
      {
        width: DefaultSetting.width,
        height: DefaultSetting.height,
        sideOrientation: Mesh.DOUBLESIDE
      },
      this.scene
    )
    this.plane.position = this.position
    this.plane.rotation = this.rotation
    this.plane.scaling = new Vector3(this.scale, this.scale, this.scale)
    this.plane.material = this.tempMat
    this.plane.isPickable = true
    let textureResolution = 512
    this.groundTexture = new DynamicTexture('dyntex1', textureResolution, this.scene, true)
    this.clearWhiteBoard()
    var dynamicMaterial = new StandardMaterial('mat', this.scene)
    dynamicMaterial.diffuseTexture = this.groundTexture
    dynamicMaterial.specularColor = new Color3(0, 0, 0)
    dynamicMaterial.backFaceCulling = true
    this.plane.material = dynamicMaterial
    this.detectCursor()
    this.mainCanvas.addEventListener(
      'pointerdown',
      function (event) {
        _this.onPointerDown(event)
      },
      false
    )
    this.mainCanvas.addEventListener(
      'pointerup',
      function (event) {
        _this.onPointerUp(event)
      },
      false
    )
    this.mainCanvas.addEventListener(
      'pointermove',
      function (event) {
        _this.onPointerMove(event)
      },
      false
    )

    //dispose
    this.scene.onDispose = function () {
      _this.mainCanvas.removeEventListener('pointerdown', function (event) {
        _this.onPointerDown(event)
      })
      _this.mainCanvas.removeEventListener('pointerup', function (event) {
        _this.onPointerUp(event)
      })
      _this.mainCanvas.removeEventListener('pointermove', function (event) {
        _this.onPointerMove(event)
      })
    }
  }

  private onPointerDown(evt: any) {
    this.isDrawing = true
  }
  private onPointerUp(evt: any) {
    this.isDrawing = false
  }

  private onPointerMove(event: any) {
    if (this.isDrawing) {
      let pickResult = this.scene.pick(this.scene.pointerX, this.scene.pointerY)
      if (pickResult.pickedMesh && pickResult.pickedMesh.id && pickResult.pickedMesh.id == this.name) {
        // document.body.style.cursor = DefaultSetting.cursor
        this.mainScene.lockRotation = true
        let texcoords = pickResult.getTextureCoordinates()
        let centerX = texcoords.x * this.groundTexture.getSize().width
        let centerY = this.groundTexture.getSize().height - texcoords.y * this.groundTexture.getSize().height
        let context1 = this.groundTexture.getContext()
        context1.beginPath()
        context1.arc(centerX, centerY, DefaultSetting.pencilRadius, 0, 2 * Math.PI, false)
        context1.fillStyle = DefaultSetting.pencilColor //red: #ff0000,  green: #00ff00, blue: #0000ff
        context1.fill()
        context1.lineWidth = 2
       // socket.emit - how to get all content of this.groundTexture
        console.log(JSON.stringify(this.groundTexture)) // bug right here
        this.groundTexture.update()
      } else {
        // document.body.style.cursor = 'default'
        this.mainScene.lockRotation = false
      }
    }
  }

  private detectCursor = () => {
    let planeActionManager = new ActionManager(this.scene)
    this.plane.actionManager = planeActionManager
    planeActionManager.registerAction(
      new ExecuteCodeAction(ActionManager.OnPointerOverTrigger, (ev) => {
        this.scene.hoverCursor = DefaultSetting.cursor
      })
    )
    planeActionManager.registerAction(
      new ExecuteCodeAction(ActionManager.OnPointerOutTrigger, (ev) => {
        document.body.style.cursor = 'default'
      })
    )
  }

  private clearWhiteBoard() {
    let context = this.groundTexture.getContext()
    context.rect(0, 0, this.groundTexture.getSize().width, this.groundTexture.getSize().height)
    context.fillStyle = DefaultSetting.boardColor
    context.fill()
    this.groundTexture.update()
  }
}

Thanks for reading my question and hope somebody can help me!

Probably you may use dynTex.clone() ? Docs - Babylon.js docs

Does DynamicTexture have .readPixels(). Then I would just send that?

Or just send the drawing instructions?

I don’t think you should transfer everything from Dynamictexture, but rather transfer texcoords.x with texcoords.y and redraw on another user’s screen.
If you want to ensure data consistency, you can cache an array, and the transmitted data contains the subscript of the coordinates in the array.
You only need to encapsulate the drawing code into a function to achieve the consistency of local drawing and socket drawing.

function drawLine(texture,centerX,centerY) {
        let context1 =texture.getContext()
        context1.beginPath()
        context1.arc(centerX, centerY, DefaultSetting.pencilRadius, 0, 2 * Math.PI, false)
        context1.fillStyle = DefaultSetting.pencilColor //red: #ff0000,  green: #00ff00, blue: #0000ff
        context1.fill()
        context1.lineWidth = 2
       // socket.emit - how to get all content of this.groundTexture
        console.log(JSON.stringify(this.groundTexture)) // bug right here
        texture.update()
}

1 Like

It is also possible to pass canvas data as blob

const blob = await new Promise(resolve => DynTexCanvas.toBlob(resolve))

when i have readPixels data. Which function to set back pixels data to the dynamicTexture?

But i can not pass all dynTex interface through socket event. I also try JSON.stringify but i received above error

But i have another problem that how to retrieve all data to fill to the whiteboard when initial (such as in case i just join to the room which have the board, and the board was draw before that). Can i have any methods to get and set the context of the DynamicTexture.

And when i had blob variable, how to set again to the canvas to draw the right texture

Yeah sorry my bad. Only RawTextures can take back the raw data.

I think you can look into DynamicTexture.getContext to get a CanvasContext. Then you have getImageData and putImageData; where ImageData should be the actual HTML thing.

But keep in mind what was mentioned above. If performance becomes an issue, you are probably better of only sending drawing instruction and replicate them on clients.

The socket service stores the data drawn each time, and when a new socket connection comes in, all the stored data is directly pushed.

This is an image blob that can be drawn as an image on the canvas, but I don’t recommend it. Because every time the canvas is changed, the complete canvas data is transmitted, which is a great redundancy.

lineDrawSyncExample.zip (10.7 KB)

This is a simple canvas lineDraw example, and after downloading, the command line executes

npm i
node app.js

This example requires a nodejs environment.