Description
I am developing an App which runs on two Platforms (desktop and web). For the desktop variant i’m using tauri. In the App there is a babylon scene which (in the end) consists of a couple of meshes. And if I click on some of the meshes i let a line, rectangle and textblock appear with babylonjs-gui. Which works just fine in the browser (Firefox 91.12.0esr (64-Bit)
) but doesnt work in the desktop app (webview). So my guess is that babylonjs-gui just doesnt work in the tauri webview.
This is how it works in the browser:
In the desktop variant however it doesnt work (and yes I’m clicking):
Reproduction
I’m using the following babylon versions:
"babylonjs": "5.18.0",
"babylonjs-gui": "5.18.0",
"babylonjs-loaders": "5.18.0",
And this is my onPointerDown function:
I also get “we hit gold” logged in the desktop app so the code which should display the gui elements gets executed but it just doesnt display it .
// TODO i dont know if we need var to use it in the onPointerDown callback down below
var label: GUI.TextBlock;
var rect1: GUI.Rectangle;
var line: GUI.Line;
var sphere: BABYLON.Mesh; // The point to which the line rect and label get attached
// TODO this is kinda bad but since we need it in the callback
var scene = this.scene;
var camera = this.camera;
var errors = this.errors;
this.scene.onPointerDown = function castRay(evt: PointerEvent, pickInfo: BABYLON.PickingInfo, type: BABYLON.PointerEventTypes) {
// src: https://playground.babylonjs.com/#AC8XPN, https://www.youtube.com/watch?v=dgsWKpa7RcY
let ray = scene.createPickingRay(scene.pointerX, scene.pointerY, BABYLON.Matrix.Identity(), camera);
console.log('in onPointerDown');
let hit = scene.pickWithRay(ray, (mesh: BABYLON.AbstractMesh) => {
// some filtering
});
function disposeBox() {
label?.dispose();
rect1?.dispose();
line?.dispose();
sphere?.dispose();
}
// TODO could do hit.pickedMesh?.metadata ??
if(hit.pickedMesh && hit.pickedMesh.metadata) {
console.log('we hit gold');
disposeBox();
sphere = BABYLON.MeshBuilder.CreateSphere("a", {}); // Create a sphere with the given name
sphere.visibility = 0; // Set visability 0 = no visable 1 = visable
sphere.position = hit.pickedPoint;
let advancedTexture = GUI.AdvancedDynamicTexture.CreateFullscreenUI("UI", true, this); // Enables GUI Elements for the scene
advancedTexture.renderAtIdealSize = true;
line = new GUI.Line("line");
line.lineWidth = 4; // Width of the line
line.color = "red"; // Color of the line
advancedTexture.addControl(line);
line.linkWithMesh(sphere); // Link with given Mesh
// Texbox
rect1 = new GUI.Rectangle("rect1"); // Create Textbox
rect1.adaptHeightToChildren = true;
rect1.adaptWidthToChildren = true;
rect1.cornerRadius = 20; // radius for the corners
rect1.color = "red"; // Border color
rect1.thickness = 4; // Thickness of the border
rect1.background = "white"; // Background color
advancedTexture.addControl(rect1);
rect1.linkWithMesh(sphere); // Link with given mesh
rect1.linkOffsetY = -150; // Let it float above
// Text
label = new GUI.TextBlock("label"); // Create Lable for text
label.text = hit.pickedMesh.metadata.split(';').join('\n'); // Set the displayed text
label.resizeToFit = true; // Reisize to content
label.color = "Black" // Fontcolor
label.paddingTop = 10; // Set the padding
label.paddingBottom = 10; // Set the padding
label.paddingRight= 10; // Set the padding
label.paddingLeft = 10; // Set the padding
rect1.addControl(label);
// Connect Line with TextBox
line.connectedControl = rect1; // Connect with TextBox
} else {
disposeBox();
}
}