Synchronize multiple objects in multiple canvases

Hello :slight_smile:

I’m working on a project where we have to display differents objects (.obj files) in order to compare them. The tricky point is that we have to synchronize movements between them !

Which stack ?

  • Vue 3
  • TypeScript
  • Babylon.js 5.0.0-alpha.44
  • Babylon.js loaders 5.0.0-alpha.44

Which structure ?

  • Each object is in a separated canvas (easier for me to manage the UI and the users actions).
  • Each object is initialized by a new instance of ViewerService (the class initializing the engine, the scene and loading the object).
  • To manage the synchronization, I have a GlobalViewerService (a singleton in the application). This class interacts with user events and each synchronized ViewerService.

How to manage synchronization ?

The synchronization can be toggled by buttons.
We set the lead canvas with a button and the others canvases are following it.

I have tried differents things to make the synchronization work but two principles have retained my attention :

  • Listen to pointer events on the lead canvas, clone the events and dispatch them to the following canvases : very simple and easy to implement, but at every dispatched event the focus was dispatching too…

    • The lead canvas don’t keep the focus and it’s breaking the synchronization !
  • For each following canvas : set the engine.inputElement with the lead canvas (idea from babylon documentation). It was not working great : the pointer events was buggy. To fix this, two things :

    • I create a “control” canvas over the lead canvas. This is the one to controll the following canvases. Without it, the lead canvas don’t move :frowning:
    • I listen to a keyboard event. Keypress = set engine.inputElement and camera control. Keyup = reset engine.inputElement and camera control.
      • It works ! But it’s not convenient for the user : kreypress"s", keep “s” pressed, click with the mouse to do the manipulation, keyup “s” to end the manipulation.

My questions

  • Is there a trick to dispatch the events without dispatching the focus ?
  • Any tips to improve the user experience ?
  • Any better solution to synchronize these canvases ?

I hope everything was clear, don’t hesitate to ask me question about the code or the user actions !
Thank you and have a great day :slight_smile:

Code interesting for you in GlobalViewerService :

Almost everything.

/* eslint-disable @typescript-eslint/no-explicit-any */

import ViewerService from './ViewerService';

export default class GlobalViewerService {
  public synchronizedCanvases: Map<string, HTMLCanvasElement> = new Map();

  public viewerServices: Map<string, ViewerService> = new Map();

  public leadCanvas: { id: string, canvas: HTMLCanvasElement } | undefined = undefined;

  public controlCanvas: HTMLCanvasElement | undefined = undefined;

  public scrollableContainer: HTMLElement | undefined = undefined;

  public oldScroll: number | undefined = undefined;

  private synchronized = false;

  constructor() {
    document.addEventListener('keypress', this.synchronize);
    document.addEventListener('keyup', this.desynchronize);
  }

  private synchronize = (event: any) => {
    if (event.key && event.key === 's') {
      if (!this.synchronized) {
        this.synchronized = true;
        if (this.controlCanvas) {
          this.viewerServices.forEach((viewer) => {
            viewer.resetCanvasControl();
            if (this.controlCanvas) {
              viewer.setCanvasControl(this.controlCanvas);
            }
          });
          this.controlCanvas.classList.add('active');
          this.controlCanvas.focus();
        }
      }
    }
  }

  private desynchronize = (event: any) => {
    if (event.key && event.key === 's') {
      this.synchronized = false;
      this.viewerServices.forEach((viewer) => {
        viewer.resetCanvasControl();
      });
      if (this.controlCanvas) {
        this.controlCanvas.blur();
        if (this.leadCanvas) {
          this.leadCanvas.canvas.focus();
        }
        this.controlCanvas.classList.remove('active');
      }
    }
  };

  public addViewerService = (id:string, viewer: ViewerService) => {
    this.viewerServices.set(id, viewer);
  }

  public removeViewerService = (id:string) => {
    this.viewerServices.delete(id);
  }

  public setLeadCanvas = (id: string, canvas: HTMLCanvasElement) => {
    this.leadCanvas = {
      id,
      canvas,
    };
    if (canvas.parentElement) {
      // Create a new canvas over the lead canvas to control the synchronization
      // Necessary for the lead canvas to move too
      const controlCanvas = document.createElement('canvas');
      controlCanvas.id = `${id}_control`;
      controlCanvas.classList.add('control-canvas');
      canvas.parentElement.appendChild(controlCanvas);
      this.controlCanvas = controlCanvas;
    }
  }

  public unsetLeadCanvas = () => {
    if (this.leadCanvas) {
      if (this.controlCanvas) {
        this.controlCanvas.remove();
        this.controlCanvas = undefined;
      }
      this.leadCanvas = undefined;
    }
  }

  public addSynchronizedCanvas = (id: string, canvas: HTMLCanvasElement, viewer: ViewerService) => {
    this.synchronizedCanvases.set(id, canvas);
    this.viewerServices.set(id, viewer);
  }

  public removeSynchronizedCanvas = (id: string) => {
    const canvas = this.synchronizedCanvases.get(id);
    if (canvas) {
      const viewer = this.viewerServices.get(id);
      if (viewer) {
        viewer.resetCanvasControl();
      }
      this.viewerServices.delete(id);
      this.synchronizedCanvases.delete(id);
    }
  }

  /**
   * Triggerred when we leave the comparator
   */
  public resetAll = () => {
    this.synchronizedCanvases = new Map();
    this.viewerServices = new Map();
    this.controlCanvas = undefined;
    this.leadCanvas = undefined;
    this.scrollableContainer = undefined;
    this.oldScroll = undefined;
    this.synchronized = false;
  }
}

Code interesting for you in ViewerService :

To load an object I use SceneLoader.Load();

The functions triggered by the keydown and the keyup events :


  public setCanvasControl = (canvas: HTMLCanvasElement) => {
    this.camera.detachControl();
    this.scene.detachControl();
    this.engine.inputElement = canvas;
    this.scene.attachControl();
    this.camera.attachControl(canvas);
  }

  public resetCanvasControl = () => {
    this.camera.detachControl();
    this.scene.detachControl();
    this.engine.inputElement = this.canvas;
    this.scene.attachControl();
    this.camera.attachControl(this.canvas);
  }

New user, so 1 upload per post :frowning:

controls

  1. For addSynchronizedCanvas() and removeSynchronizedCanvas() (see code below)
  2. For setLeadCanvas() and unsetLeadCanvas() (see code below)

Demo !

You can see the mouse events but keep in mind that I’m triggering the keyboard between manipulations too :wink:
demo(2)

That’s a very cool interface already!

Another idea might be storing the references to each scene’s cameras on the GlobalViewerService, keep the pointer events on their respective canvas and just synchronize the cameras themselves :smiley:
What is the type of camera you’re using? ArcRotate?

1 Like

Thank you !

Yes I’m using the ArcRotateCamera :slight_smile:

The first thing my collegue have done was to synchronize the cameras (using the Vue store and updating manualy the cameras coordinates). We were able to synchronize only the rotation and the zoom, but not the right clic move. That why we have tried this engine.inputElement solution.

We also have tried using only camera.detachControl(); camera.attachControl(canvas); or updating the camera.parent but it don’t seems working (certainly due to the fact that we have several engines/scenes).

Do we have another solution to synchronize the cameras ?

The right click move controls the camera target property, have you synchronized that?

Hi ! A bit late to respond !

We have tried to synchronized that too, but maybe not the right way…

Anyway, my coworker created a POC to reproduce the right behavior and he made it ! My project’s complexity maybe brought some side effects…

The documentation guidelines was good :

this.scene.detachControl();
this.engine.inputElement = document.getElementById(id) as HTMLCanvasElement;
this.scene.attachControl();

His repo is public, if it can help anybody : GitHub - Dreadbond/bc-bjs-poc

Thank you for your help @carolhmj :slight_smile:

2 Likes