MeshBuilder Not Work With Custom Mesh On Ubuntu18 Tauri Webkit

On Ubuntu18.04 and Tauri webkit framework, PointCloudSystem and MeshBuilder can not render at the same time. PointCloudSystem only render in initialization frame, but the mesh of MeshBuilder is rendered normally.
The following is my useragent, babylonjs version is 6.41.0
mozilla/5.0 (x11; ubuntu; linux x86_64) applewebkit/605.1.15 (khtml, like gecko) version/14.0 safari/605.1.15
I created a PG, but it cannot be reproduced.

The following is the actual comparison between Webkit and Webgl.

I even test on Ubuntu22.04, Macos and Windows. Everything is normal in Webkit/WebView2/Webgl under these system environments.

After some searching and trying, I can basically confirm that pointcloud mode cannot be supported in the target environment.

customMeshMaterial.pointsCloud = true; // not work
 // customMeshMaterial.fillMode = BABYLON.Material.PointFillMode; // not work
// customMeshMaterial.fillMode = BABYLON.Material.LineListDrawMode; // Work
// customMeshMaterial.fillMode = BABYLON.Material.TriangleFillMode; // Work

As a newbie, I’m not sure if this is a babylonjs bug or a webkit bug

As a compatible alternative, I must consider using Solid Particle System to complete point cloud display. Compared with manually implementing Custom Mesh, how is its performance?

It’s probably not a bug with Babylon.js, if other modes than “Point” work, and if it also works on other systems.

Using SPS is a good alternative, and you will get the same performance than doing it yourself with a custom mesh, as it’s basically what SPS does (it fills a vertex buffer for position, color, 
 and draws everything in a singled draw call).

My description may be a little misleading. If you only use Point Mode (such as PCS, Custom Material) and do not use MeshBuilder (such as ground, sphere), then the point cloud will be displayed and refreshed normallyColor is abnormal.
WX20240205-101753

Could you setup a small repro in the Playground so that we are able to test it?

I can’t reproduce it in the Playground, You can see that this PG contains the same code, but it’s normal. It basically only happens in Ubuntu18 native apps

I created a minimal project(use tauri + babylonjs).
tauri-babylonjs.zip (214.1 KB)

If you have an Ubuntu18 system or virtual machine and installed node16 and yarn, it can be easily reproduced. You only need to execute

yarn install
yarn tauri dev

Thank you very much for this

Sorry, I don’t have such OS. Maybe others will be able to reproduce and help finding the problem, but it really looks like a bug of your system (maybe a driver problem?), unfortunately.

Thanks for the help, I’ll be using SPS as an alternative, hopefully someone in the future can identify the problem

After some efforts, I have no way to use SPS and customize the Shader at the same time
For this I had to try to use Threejs for test, surprisingly it is almost no problem at all, the following is a screenshot of my test and the corresponding code

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <title>Babylon Template</title>

    <style>
      body {
        overflow: hidden;
        width: 100%;
        height: 100%;
        margin: 0;
        padding: 0;
      }
    </style>
  </head>

  <body>
    <div id="canvas-holder">
      <canvas id="maincanvas" width="300" height="300"></canvas>
    </div>
    <button id="btnRepeat" style="position: absolute; margin: 5px;">Repeat</button>
    <script type="module">
      import * as THREE from "three";
      import { OrbitControls } from 'three/addons/controls/OrbitControls.js';
      var scene = new THREE.Scene();
      var camera = new THREE.PerspectiveCamera(
        60,
        window.innerWidth / window.innerHeight,
        1,
        1000
      );
      camera.position.set(0, 8, 13);
      camera.lookAt(0, 5, 0);
      var renderer = new THREE.WebGL1Renderer({
        antialias: true,
      });
      renderer.setSize(window.innerWidth, window.innerHeight);
      renderer.setClearColor(0x101010);
      document.body.appendChild(renderer.domElement);
      window.addEventListener("resize", onWindowResize, false);

      var controls = new OrbitControls(camera, renderer.domElement);

      scene.add(new THREE.GridHelper(10, 10));

      var rMin = 5;
      var rMax = 15;
      var hMin = 0;
      var hMax = 5;

      var centerH = 6;

      var delayMax = 1; // seconds
      var colors = [0xd50f30, 0x039bdc, 0xfc5411, 0x1a2683, 0xfbe23d, 0x12923b];
      var c = new THREE.Color();
      var MAX_POINTS = 5000;
      var pointsCount = 0;
      var points = []; //3

      var delay = []; //1
      var color = []; //3
      var turns = []; //1
      var turnsMin = 5;
      var turnsMax = 10;
      var angle = [];
      while (pointsCount < MAX_POINTS) {
        let a = THREE.MathUtils.randFloat(0, Math.PI * 2);
        let vec = new THREE.Vector3().setFromCylindricalCoords(
          THREE.MathUtils.randFloat(rMin, rMax),
          a,
          THREE.MathUtils.randFloat(hMin, hMax)
        );

        points.push(vec);

        delay.push(THREE.MathUtils.randFloat(-delayMax, 0));

        c.set(colors[THREE.MathUtils.randInt(0, colors.length - 1)]);
        color.push(c.r, c.g, c.b);

        angle.push(a);

        turns.push(THREE.MathUtils.randFloat(turnsMin, turnsMax));

        pointsCount++;
      }

      var pointsTornadoGeom = new THREE.BufferGeometry().setFromPoints(points);
      pointsTornadoGeom.setAttribute(
        "color",
        new THREE.BufferAttribute(new Float32Array(color), 3)
      );
      pointsTornadoGeom.setAttribute(
        "delay",
        new THREE.BufferAttribute(new Float32Array(delay), 1)
      );
      pointsTornadoGeom.setAttribute(
        "angle",
        new THREE.BufferAttribute(new Float32Array(angle), 1)
      );
      pointsTornadoGeom.setAttribute(
        "turns",
        new THREE.BufferAttribute(new Float32Array(turns), 1)
      );

      var pointsTornadoMat = new THREE.PointsMaterial({
        vertexColors: THREE.VertexColors,
        size: 0.2,
        map: new THREE.TextureLoader().load(
          "https://threejs.org/examples/textures/sprites/ball.png",
          (tex) => {
            tex.center.setScalar(0.5);
            tex.rotation = -Math.PI * 0.5;
          }
        ),
        alphaTest: 0.5,
      });

      var uniformsTornado = {
        time: { value: 0 },
        centerH: { value: centerH },
        duration: { value: 10 - delayMax }, // seconds
        turnsMax: { value: turnsMax },
      };

      pointsTornadoMat.onBeforeCompile = (shader) => {
        shader.uniforms.time = uniformsTornado.time;
        shader.uniforms.centerH = uniformsTornado.centerH;
        shader.uniforms.duration = uniformsTornado.duration;
        shader.uniforms.turnsMax = uniformsTornado.turnsMax;

        shader.vertexShader =
          `
    uniform float time;
    uniform float centerH;
    uniform float duration;
    uniform float turnsMax;
    
    attribute float delay;
    attribute float angle;
    attribute float turns;

    varying float vRatio;
` + shader.vertexShader;

        shader.vertexShader = shader.vertexShader.replace(
          `#include <begin_vertex>`,
          `#include <begin_vertex>
    
    float t = time + delay;
    t = t < 0. ? 0. : t;

    float tRatio = 0.;
    tRatio = clamp( (t * (turnsMax / turns)) / duration, 0., 1.);
    vRatio = step(0.01, tRatio) - step(0.999, tRatio);
    
    float durationA = PI * 2. * turns * tRatio;;
    float A = angle + durationA;    

    transformed.x = cos(A);
    transformed.z = sin(-A);
    
    float l = length(position.xz);
    transformed.xz *= l * mix(1., 0., tRatio);

    transformed.y = mix(position.y, centerH, tRatio);
   
`
        );

        shader.vertexShader = shader.vertexShader.replace(
          `gl_PointSize = size;`,
          `gl_PointSize = size * smoothstep(0.01, 0.1, tRatio);`
        );

        shader.fragmentShader =
          `
    varying float vRatio;
` + shader.fragmentShader;
        shader.fragmentShader = shader.fragmentShader.replace(
          `#include <clipping_planes_fragment>`,
          `
    if (floor(vRatio + 0.5) < 1.) discard;
    #include <clipping_planes_fragment>`
        );
      };

      var pointsTornado = new THREE.Points(pointsTornadoGeom, pointsTornadoMat);
      scene.add(pointsTornado);

      var clock = new THREE.Clock();
      var t = 0;

      btnRepeat.addEventListener(
        "click",
        (event) => {
          t = 0;
        },
        false
      );

      renderer.setAnimationLoop(() => {
        t += clock.getDelta();
        uniformsTornado.time.value = t;
        renderer.render(scene, camera);
      });

      function onWindowResize() {
        var width = window.innerWidth;
        var height = window.innerHeight;
        camera.aspect = width / height;
        camera.updateProjectionMatrix();
        renderer.setSize(width, height);
      }
    </script>
  </body>
</html>

So maybe it’s not a system/driver issue? Do you think I have to use Threejs because of this problem?

I think Threejs doesn’t use uniform buffers, so maybe that’s the problem.

You could try disabling the use of uniform buffers in Babylon.js by doing engine.disableUniformBuffers = true; right at the start of your program and see if that helps.

Tried your code, unfortunately still not right

I can see you use a WebGL1 renderer in Threejs. Can you try the same in Babylon.js, by doing something like const engine = new BABYLON.Engine(canvas, true, { disableWebGL2Support: true }); to create the engine?

1 Like

Still not working, so frustrating :face_exhaling:
I did some searches and found similar issues related to threejs,

but for ubuntu18.04, I checked the following version, which can use threejs normally.

demo@ubuntu:~/Downloads/tauri-babylonjs$ dpkg -l|grep libwebkit2gtk
ii  libwebkit2gtk-4.0-37:amd64                 2.32.4-0ubuntu0.18.04.1                         amd64        Web content engine library for GTK+
ii  libwebkit2gtk-4.0-dev:amd64                2.32.4-0ubuntu0.18.04.1                         amd64        Web content engine library for GTK+ - development files

For comparison, I wrote similar code using threejs and it works fine under ubuntu18.04. Both Sphere and PointCloud render normal.

import * as THREE from 'three'
import { OrbitControls } from 'three/addons/controls/OrbitControls.js'
import Stats from 'three/addons/libs/stats.module.js'
let scene: THREE.Scene

function generate_points() {
  const points = []
  for (let i = 0; i < 10000; i++) {
    const x = THREE.MathUtils.randFloatSpread(10)
    const y = THREE.MathUtils.randFloatSpread(10)
    const z = THREE.MathUtils.randFloatSpread(10)

    points.push(x, y, z)
  }
  return points
}

function test_point_cloud() {
  const vertices = generate_points()

  const geometry = new THREE.BufferGeometry()
  geometry.setAttribute('position', new THREE.Float32BufferAttribute(vertices, 3))
  const colorAttribute = new THREE.Uint8BufferAttribute(vertices, 3)
  colorAttribute.normalized = true // this will map the buffer values to 0.0f - +1.0f in the shader
  geometry.setAttribute('color', colorAttribute)

  const material = new THREE.RawShaderMaterial({
    uniforms: {
      time: { value: 1.0 }
    },
    vertexShader: `
                    precision mediump float;
                    precision mediump int;

                    uniform mat4 modelViewMatrix; // optional
                    uniform mat4 projectionMatrix; // optional

                    attribute vec3 position;
                    attribute vec3 color;

                    varying vec3 vPosition;
                    varying vec4 vColor;

                    void main()	{
                        gl_PointSize = 2.0;
                        vPosition = position;
                        vColor = vec4(color, 1.0 );

                        gl_Position = projectionMatrix * modelViewMatrix * vec4( position, 1.0 );

                    }`,
    fragmentShader: `
                    precision mediump float;
                    precision mediump int;

                    uniform float time;

                    varying vec3 vPosition;
                    varying vec4 vColor;

                    void main()	{

                        vec4 color = vec4( vColor );
                        color.r += sin( vPosition.x * 10.0 + time ) * 0.5;

                        gl_FragColor = color;

                    }`,
    side: THREE.DoubleSide,
    transparent: true
  })

  const mesh = new THREE.Points(geometry, material)
  scene.add(mesh)

  setInterval(() => {
    const vertices = generate_points()
    geometry.setAttribute('position', new THREE.Float32BufferAttribute(vertices, 3))
    geometry.attributes.position.needsUpdate = true
  }, 100)
}

export function init_scene(canvas: any) {
  scene = new THREE.Scene()
  const camera = new THREE.PerspectiveCamera(
    60,
    canvas.clientWidth / canvas.clientHeight,
    0.01,
    10000
  )
  camera.position.set(0, 8, 13)
  camera.lookAt(0, 5, 0)
  const renderer = new THREE.WebGL1Renderer({
    canvas: canvas,
    antialias: true
  })

  const stats = new Stats()
  document.body.appendChild(stats.dom)
  renderer.setClearColor(0x101010)
  window.addEventListener('resize', onWindowResize)
  renderer.setSize(canvas.clientWidth, canvas.clientHeight)

  const controls = new OrbitControls(camera, renderer.domElement)

  scene.add(new THREE.GridHelper(10, 10))

  const geometry = new THREE.SphereGeometry(1.5, 32, 16)
  const material = new THREE.MeshBasicMaterial()
  const sphere = new THREE.Mesh(geometry, material)
  scene.add(sphere)

  test_point_cloud()

  renderer.setAnimationLoop(() => {
    stats.update()
    renderer.render(scene, camera)
  })

  function onWindowResize() {
    camera.aspect = canvas.clientWidth / canvas.clientHeight
    camera.updateProjectionMatrix()
    renderer.setSize(canvas.clientWidth, canvas.clientHeight)
  }
}

Unfortunately, I don’t have Ubuntu 18.04, so I’m a bit lost.

cc @sebavan in case he has any ideas.

Could you share both samples in jsfiddle for instance or wherever we can use them online in order to ensure the setup is identical ?

As I mentioned before, this is a problem that only occurs under ubuntu18.04 webkitgtk. It is completely normal in the browser, and even firefox of ubuntu18.04 is completely normal.
I can’t reproduce the problem in a browser environment, sorry about that
If you are interested in using a virtual machine to install the ubuntu18.04 system, I think this problem will be easy to reproduce.

It would still be great to have both the broken PG and your threejs example so we can figure which webgl feature might be the cause.

cc @Cedric who might have a linux VM to try out ?

Unfortunately, I don’t have a linux box atm. Moreover, I try to keep up with versions so I don’t think I had older than 20.04.

1 Like

I got it running using WSL + GUI using Ubuntu 18.04.6 LTS. The instructions are pretty thin. I had to figure out how to install a bunch of dependencies before it built. For future reference: Getting Tauri working in WSL (penner.me) and Prerequisites | Tauri Apps.

Someone tell me what I need to debug.

4 Likes