[Sort-of Resolved] Z-index not updating on buttons

Questions
1

Hey folks!

I have a question regarding the Virtual Joystick and GUI buttons. As mentioned in the docs and in this post, you can set the z-index for for the canvas elements, so that buttons can be triggered.

I’m basically trying to replicate the the button as in this playground. I don’t need the function to reset the z-index of the joystick, basically just want the buttons to sit on top of the joystick canvas.

I tried to set the z index for the button but it does not seem to work. Any ideas on what I might be doing wrong?

Here is how I create the buttons:

export class ActionButtons {
    private ui: AdvancedDynamicTexture;
        // Button press states
        public isSprintPressed: boolean = false;
        public isJetpackPressed: boolean = false;
        public isKickPressed: boolean = false;

    constructor() {
        this.ui = AdvancedDynamicTexture.CreateFullscreenUI("UI");

      
        // Only create action buttons if the device is a mobile device
        if (isMobileDevice()) {
            this.createActionButtons();
        }
    }
    private createActionButtons() {
        const buttonSize = "60px";
        const buttonSpacing = "10px";

        // Create sprint button
        const sprintButton = Button.CreateImageOnlyButton("sprintButton", "sprinting.png");
        sprintButton.width = buttonSize;
        sprintButton.height = buttonSize;
        sprintButton.horizontalAlignment = Control.HORIZONTAL_ALIGNMENT_RIGHT;
        sprintButton.verticalAlignment = Control.VERTICAL_ALIGNMENT_BOTTOM;
        sprintButton.left = `-${buttonSpacing}`;
        sprintButton.top = `-${buttonSpacing}`;
        sprintButton.isPointerBlocker = true; // Prevents propagation to joystick
        sprintButton.zIndex = 10; // Set a higher z-index for the button
        this.ui.addControl(sprintButton);

        // Create jetpack button
        const jetpackButton = Button.CreateImageOnlyButton("jetpackButton", "jetpacking.png");
        jetpackButton.width = buttonSize;
        jetpackButton.height = buttonSize;
        jetpackButton.horizontalAlignment = Control.HORIZONTAL_ALIGNMENT_RIGHT;
        jetpackButton.verticalAlignment = Control.VERTICAL_ALIGNMENT_BOTTOM;
        jetpackButton.left = `-${parseInt(buttonSpacing) + parseInt(buttonSize) * 2}px`; // Adjust position to not overlap with sprint button
        jetpackButton.top = `-${buttonSpacing}`;
        jetpackButton.isPointerBlocker = true; // Prevents propagation to joystick
        jetpackButton.zIndex = 10; // Set a higher z-index for the button
        this.ui.addControl(jetpackButton);

        // Create kick button
        const kickButton = Button.CreateImageOnlyButton("kickButton", "kicking.png");
        kickButton.width = buttonSize;
        kickButton.height = buttonSize;
        kickButton.horizontalAlignment = Control.HORIZONTAL_ALIGNMENT_RIGHT;
        kickButton.verticalAlignment = Control.VERTICAL_ALIGNMENT_BOTTOM;
        kickButton.left = `-${parseInt(buttonSpacing) + parseInt(buttonSize) * 4}px`; // Adjust position to not overlap with other buttons
        kickButton.top = `-${buttonSpacing}`;
        kickButton.isPointerBlocker = true; // Prevents propagation to joystick
        kickButton.zIndex = 10; // Set a higher z-index for the button
        this.ui.addControl(kickButton);

        // Add event listeners for the buttons
        sprintButton.onPointerDownObservable.add(() => {
            console.log("Sprint Button Pressed");
            this.isSprintPressed = true;
            
        });
        sprintButton.onPointerUpObservable.add(() => {
            this.isSprintPressed = false;
        });

        jetpackButton.onPointerDownObservable.add(() => {
            this.isJetpackPressed = true;
        });
        jetpackButton.onPointerUpObservable.add(() => {
            this.isJetpackPressed = false;
        });

        kickButton.onPointerDownObservable.add(() => {
            this.isKickPressed = true;
        });
        kickButton.onPointerUpObservable.add(() => {
            this.isKickPressed = false;
        });
    
    }

}

export default ActionButtons;

And this is how I create the the joystick:

import { Vector3, VirtualJoystick } from '@babylonjs/core';

class CustomTouchStick extends VirtualJoystick {
    direction: Vector3 = Vector3.Zero();
    private directionMaxLength: number = 3;
    private directionSensitivity: number = 350;
    

    constructor(isLeftStick: boolean) {
        super(isLeftStick);
        this.setJoystickSensibility(1);
        this.setupListener();
    }

    private setupListener() {
        this.detect();
        this.direction = this.getDirection(this.deltaPosition.x, this.deltaPosition.y);
        requestAnimationFrame(() => this.setupListener());
    }

    private detect() {
        const { pressed, deltaPosition } = this;
        if (pressed) {
            this.direction = this.getDirection(deltaPosition.x, deltaPosition.y);
        } else {
            this.reset();
        }
    }

    private getDirection(deltaX: number, deltaY: number): Vector3 {
        const joystickVector = new Vector3(deltaX, 0, deltaY);
        if (joystickVector.length() === 0) {
            return Vector3.Zero();
        }
        const scaleFactor = 3;
        const scaledLength = Math.pow(joystickVector.length(), scaleFactor) * this.directionSensitivity;
        const cappedLength = Math.min(scaledLength, this.directionMaxLength);
        
        return joystickVector.normalize().scale(cappedLength);
    }

    private reset() {
        this.direction = Vector3.Zero();
        this.deltaPosition.x = 0;
        this.deltaPosition.y = 0;
    }
}

export default CustomTouchStick;

Thanks for all the help in advance :slight_smile:

2

The demo (and the reference in the post) are both referring to HTML elements - the button in the playground is a standard HTML element that can be set in front of the canvas thaat actually renders the scene AND the one that renders the virtual joystick. However, the GUI works a bit differently.

Would you be able to reprouce the entire use case in a (non working) playground? This way I will understand better what exactly youa re trying to achieve and will be able to help

3

Thanks Raanan!

That makes more sense now - I can’t directly set the z-index of the GUI buttons, as they are not html elements like in the examples. Should I be adding these buttons to a new canvas and set the z-index of the new canvas itself?

If I provide a repo will that help?

I create the buttons in playerUI.ts just in case you are looking for the snippet I provided.

1 Like
4

That is very helpful, yes :slight_smile:

What is the best way to reproduce it on the repo?

5

To run it you just need npm install, and npm run dev (but I assume you already knew this :smiley: )

To see the buttons not working I just switch to a mobile device in the dev tools. I have the code set up so that the buttons and joysticks only show when on mobile device.

When I remove this and have the buttons show on desktop, I can click them and they trigger the playerActions as intended.

Hope that makes sense?

Edit: After thinking about it should I just set up the buttons as HTML elements rather than using the GUI? I assume this will resolve my issue, and this will be the easiest way to go about this?

6

yep, this is what I was looking for. :slight_smile:

This is, of course, a solution. We do want people to use our GUI elements, but if HTML is your target (for example, if you don’t plan on using WebXR or native compilation), this will work for sure.

7

Hmm okay, I’m not planning on using them for anything other than triggering the player actions when on mobile device.

In that case, I’d like to use the GUI elements, just need to figure out how to resolve this z-index issue… but good to know there is a work around for the time being :slight_smile:

1 Like
8

Yeah, I see the issue. I assume the buttons are the little football buttons at the bottom, right?. This won’t work with the current architecture of both the GUI and the virtual joystick.

Something you can do is set VirtualJoystick.Canvas.style.pointerEvents = "none" when the touch happens at a certain place in the screen, but it might be an interesting task when on mobile because you don’t have mouse move to detect pointer position before a tap.

1 Like
9

Yep, that’s right!

Okay, I’ll take a look into this and see if I can figure it out. If not I’ll just go with the html element route.

Thanks for all the help, I appreciate it :v::heart:

1 Like
10

Just as an update for future reference for anyone having similar issues, I went with the html route for now for simplicty’s sake. This is how I have updated my ActionButtons class:

export class ActionButtons {
    // Button press states
    public isSprintPressed: boolean = false;
    public isJetpackPressed: boolean = false;
    public isKickPressed: boolean = false;

    constructor() {
        // Only create action buttons if the device is a mobile device
        if (isMobileDevice()) {
            this.createActionButtons();
        }
    }

    private createActionButtons() {
        this.createHtmlButton("Sprint", "sprinting.png", "60px", "10px", 10, () => {
            this.isSprintPressed = true;
            console.log("Sprint Button Pressed");
        }, () => {
            this.isSprintPressed = false;
        });

        this.createHtmlButton("Jetpack", "jetpacking.png", "60px", "10px", 80, () => {
            this.isJetpackPressed = true;
            console.log("Jetpack Button Pressed");
        }, () => {
            this.isJetpackPressed = false;
        });

        this.createHtmlButton("Kick", "kicking.png", "60px", "10px", 150, () => {
            this.isKickPressed = true;
            console.log("Kick Button Pressed");
        }, () => {
            this.isKickPressed = false;
        });
    }

    private createHtmlButton(text: string, imgSrc: string, size: string, margin: string, right: number, onPointerDown: () => void, onPointerUp: () => void) {
        const btn = document.createElement("button");
        btn.style.width = size;
        btn.style.height = size;
        btn.style.position = "absolute";
        btn.style.bottom = margin;
        btn.style.right = `${right}px`;
        btn.style.zIndex = "10";
        btn.style.backgroundImage = `url(${imgSrc})`;
        btn.style.backgroundSize = "contain";
        btn.style.backgroundRepeat = "no-repeat";
        btn.style.border = "none";
        btn.style.outline = "none";
        btn.style.cursor = "pointer";

        btn.addEventListener("pointerdown", () => {
            onPointerDown();
            const joystickCanvas = (VirtualJoystick as any).Canvas;
            if (joystickCanvas) {
                joystickCanvas.style.pointerEvents = "none";
            }
        });

        btn.addEventListener("pointerup", () => {
            onPointerUp();
            const joystickCanvas = (VirtualJoystick as any).Canvas;
            if (joystickCanvas) {
                joystickCanvas.style.pointerEvents = "auto";
            }
        });

        document.body.appendChild(btn);
    }
}

export default ActionButtons;
1 Like