VirtualJoystick with swipe, tap, hold detections

I created class which extends BABYLON.VirtualJoystick. I share it for you:
https://github.com/onekit/babylonjs-touchstick
Give a star if you like this idea.

BabylonJS TouchStick

This TypeScript class extends the functionality of the standard VirtualJoystick in BabylonJS. It provides smooth motion delta with movement filtering, and gesture information such as swipe, hold, hold center, and tap. Additionally, you can directly obtain Vector3 from the joystick for controlling the character.

Install

NPM

npm i babylonjs-touchstick

Yarn

yarn add babylonjs-touchstick

Parameters

Tap

  • tap: Represents a quick single touch.
  • doubleTap: Represents a quick double tap.

Swipe

  • swipe.up: Denotes quick movement upwards on the joystick.
  • swipe.down: Denotes quick movement downwards on the joystick.
  • swipe.left: Denotes quick movement to the left on the joystick.
  • swipe.right: Denotes quick movement to the right on the joystick.

Hold

  • hold: Indicates holding the joystick.
  • holdCenter: Indicates holding the joystick immediately after touching it in the center.

Direction

  • direction: Represents a BABYLON.Vector3 type for character direction. The direction.length() can be used as delta for movement.

Delta Smoothing

Original deltaPosition, but with smoothing.

  • deltaPositionSmoothed: Represents smoothed movements of delta using cube easing. You can use deltaPositionSmoothed.x and deltaPositionSmoothed.y instead of deltaPosition.x and deltaPosition.y.

Usage

TypeScript example:

import TouchStick from 'babylonjs-touchstick'
import { AbstractMesh } from 'babylonjs'

export class TouchInput {

    private stickLeft: TouchStick = new TouchStick(true);
    private stickRight: TouchStick = new TouchStick(false);

    constructor() {
        this.stickLeft.setDirectionMaxLength(3) // cap for maximum distance of BABYLON.Vector3 length
    }

    private handleEvents() {
        const {swipe, tap, doubleTap, hold, holdCenter} = this.stickRight;

        if (swipe.down) {
            console.log('Swiped down right stick')
        }

        if (doubleTap) {
            console.log('Double tap')
        }

        if (holdCenter) {
            console.log('Enter menu')
        }

        if (this.stickLeft.swipe.up || this.stickRight.swipe.up) {
            console.log('Swiped up both sticks')
        }

    }
    
    handleMovement(mesh: AbstractMesh) {
        mesh.position
            .add(this.stickLeft.direction)
        
        mesh.position
            .scale(this.stickLeft.direction.length())
    }

}

Read official documentation of original BABYLON.VirtualJoystick for more information: Babylon.js docs

Upd May 21, 2024: Fine-tuned swipes detection and restructured class TouchStick.ts:

import { Vector3, VirtualJoystick } from 'babylonjs'

class TouchStick extends VirtualJoystick {
    direction: Vector3 = Vector3.Zero()
    private directionMaxLength: number = 2.8
    private directionSensitivity: number = 12000
    deltaPositionSmoothed: {
        x: number
        y: number
    } = { x: 0, y: 0 }
    swipe: {
        up: boolean
        down: boolean
        right: boolean
        left: boolean
    } = {
        up: false,
        down: false,
        right: false,
        left: false,
    }
    tap: boolean = false
    doubleTap: boolean = false
    hold: boolean = false
    holdCenter: boolean = false
    threshold: number = 0
    private lastStartTouchTime: number = 0
    private lastEndTouchTime: number = 0
    private lastStartHoldTime: number = 0
    private lastStartHoldCenterTime: number = 0
    private lastStartTapTime: number = 0
    private lastEndTapTime: number = 0
    private lastEndSwipeTime: number = 0
    private lastEndHoldTime: number = 0
    private startTouch: number = 0
    private endTouch: number = 0
    private startHold: number = 0
    private startHoldCenter: number = 0
    private startTap: number = 0
    private endTap: number = 0
    private endSwipe: number = 0
    private endHold: number = 0

    getThreshold(): number {
        return this.threshold
    }

    setThreshold(threshold: number) {
        this.threshold = threshold
    }

    getDirectionSensitivity(): number {
        return this.directionSensitivity
    }

    setDirectionSensitivity(directionSensitivity: number) {
        this.directionSensitivity = directionSensitivity
    }

    getDirectionMaxLength() {
        return this.directionMaxLength
    }

    setDirectionMaxLength(directionMaxLength: number) {
        this.directionMaxLength = directionMaxLength
    }

    constructor(isLeftStick: boolean) {
        super(isLeftStick)
        this.setJoystickSensibility(1)
        this.setupListener()
    }

    private setupListener = () => {
        this.detect()
        this.deltaPositionSmoothed = this.smoothDeltaPosition()
        this.direction = this.getDirection(
            this.deltaPosition.x,
            this.deltaPosition.y,
        )
        requestAnimationFrame(this.setupListener)
    }

    detect() {
        const { pressed, deltaPosition } = this
        const currentTime = new Date().getTime()
        this.startTouch = currentTime - this.lastStartTouchTime
        this.endTouch = currentTime - this.lastEndTouchTime
        this.startHold = currentTime - this.lastStartHoldTime
        this.endHold = currentTime - this.lastEndHoldTime
        this.startHoldCenter = currentTime - this.lastStartHoldCenterTime
        this.startTap = currentTime - this.lastStartTapTime
        this.endTap = currentTime - this.lastEndTapTime
        this.endSwipe = currentTime - this.lastEndSwipeTime

        if (pressed) {
            this.lastStartTouchTime = currentTime
            this.detectSwipes(deltaPosition)
            this.detectHold(currentTime)
            this.detectHoldCenter(currentTime, deltaPosition)
            if (!this.tap) {
                this.lastStartTapTime = currentTime
            }
        } else {
            this.detectTap(currentTime)
            this.lastEndTouchTime = currentTime
            this.reset()
        }
    }

    private detectSwipes(deltaPosition: { x: number, y: number }) {
        const thresholdY = 0.1
        const thresholdX = 0.1
        const deltaY = deltaPosition.y
        const deltaX = deltaPosition.x

        if (this.endTouch < 400 && this.startTouch < 400) {
            if (Math.abs(deltaY) > thresholdY && Math.abs(deltaX) < thresholdX) {
                this.swipe.up = deltaY > 0
                this.swipe.down = deltaY < 0
            } else if (Math.abs(deltaX) > thresholdX && Math.abs(deltaY) < thresholdY) {
                this.swipe.right = deltaX > 0
                this.swipe.left = deltaX < 0
            }
            if (this.swipe.up || this.swipe.down || this.swipe.left || this.swipe.right) {
                this.lastEndSwipeTime = new Date().getTime()
            }
        }
    }

    private detectHold(currentTime: number) {
        if (this.endTouch > 600 && this.startTouch < 600) {
            if (!this.hold) {
                this.lastStartHoldTime = currentTime
            }
            this.hold = true
        } else {
            this.lastEndHoldTime = currentTime
        }
    }

    private detectHoldCenter(currentTime: number, deltaPosition: { x: number, y: number }) {
        const thresholdY = 0.01
        const thresholdX = 0.01
        const deltaY = deltaPosition.y
        const deltaX = deltaPosition.x

        if (this.startHold < 900 && this.hold && !this.holdCenter && Math.abs(deltaY) <= thresholdY && Math.abs(deltaX) <= thresholdX) {
            this.lastStartHoldCenterTime = currentTime
            this.holdCenter = true
        } else if (this.startHoldCenter >= 900) {
            this.holdCenter = false
        }
    }

    private detectTap(currentTime: number) {
        if (!this.doubleTap && !this.tap && !this.hold && this.startTap < 50 && this.endSwipe > 350) {
            this.tap = !(this.startTap < 50 && this.hold) && this.endTap > 150 && (!this.hold && this.endHold < 250)
            if (this.tap) {
                if (this.endTap < 450) {
                    this.doubleTap = true
                    this.tap = false
                }
                this.lastEndTapTime = currentTime
            }
        } else {
            this.tap = false
            this.doubleTap = false
        }
    }

    private smoothDeltaPosition() {
        return {
            x: this.filterAxisDelta(this.deltaPosition.x, this.threshold) * 4500,
            y: this.filterAxisDelta(this.deltaPosition.y, this.threshold) * 4500,
        }
    }

    private getDirection(deltaX: number, deltaY: number): Vector3 {
        const deltaPosition = { x: deltaX, y: deltaY }
        const joystickVector = new Vector3(deltaPosition.x, 0, deltaPosition.y)
        if (joystickVector.length() === 0) {
            return Vector3.Zero()
        }
        const scaleFactor = 3
        const scaledLength =
            Math.pow(joystickVector.length(), scaleFactor) * this.directionSensitivity
        const cappedLength = Math.min(scaledLength, this.directionMaxLength)
        return joystickVector.normalize().scale(cappedLength)
    }

    private filterAxisDelta(delta: number, threshold: number = 0.00002): number {
        delta = delta * delta * delta
        if (delta > threshold) {
            delta -= threshold
        } else if (delta < -threshold) {
            delta += threshold
        } else {
            delta = 0
        }
        return delta
    }

    private reset() {
        this.direction = Vector3.Zero()
        this.hold = false
        this.holdCenter = false
        this.swipe = { up: false, down: false, right: false, left: false }
        this.deltaPosition.y = 0
        this.deltaPosition.x = 0
    }
}

export default TouchStick
7 Likes

Hi,
First of all, thanks a lot for sharing your virtual control code. I really appreciate it.

I’ve downloaded the .ts file and integrated it into my game. However, during my initial test, I encountered a peculiar issue. The virtual control seems to only function properly on the left half of the screen. I’m wondering if you have any insights or some parameters to change into what might be causing this behavior? No erros in cosole.log.

Once again, thanks for your contribution!

Yes. Cause it in boundaries of stick (one side of screen). It’s correct if you use left stick, swipe need to work on left stick. On right stick you can use another object:

private stickLeft: TouchStick = new TouchStick(true);
private stickRight: TouchStick = new TouchStick(false);

In VirtualJoystick the parameter means “leftJoystick”.

However I admit need more tunes for sensitivity and detections.

To detect both sides for swipe we can use this check:

if (this.rightStick.swipe.up || this.leftStick.swipe.up) {
  console.log('swiped up both sides')
}
2 Likes

Thanks, i see now: constructor(isLeftStick: boolean) {

All right now, or left :sweat_smile:

1 Like

Yes. And today I added doubleTap feature. :slight_smile: Upgraded and decide to make it easy to use. Looks funny 10kb script as separate packet in package.json. Now I add it as dependency in my project file: "babylonjs-touchstick": "^0.0.3" and will support it. Yeah. :slightly_smiling_face: :grinning: I am almost real contributor of BabylonJS.

babylonjs-touchstick v.0.0.3

This TypeScript class extends the functionality of the standard VirtualJoystick in BabylonJS. It provides smooth motion delta with movement filtering, and gesture information such as swipe, hold, hold center, and tap. Additionally, you can directly obtain Vector3 from the joystick for controlling the character.

Install

NPM

npm i babylonjs-touchstick

Yarn

yarn add babylonjs-touchstick

Parameters

Tap

  • tap: Represents a quick single touch.
  • doubleTap: Represents a quick double tap.

Swipe

  • swipe.up: Denotes quick movement upwards on the joystick.
  • swipe.down: Denotes quick movement downwards on the joystick.
  • swipe.left: Denotes quick movement to the left on the joystick.
  • swipe.right: Denotes quick movement to the right on the joystick.

Hold

  • hold: Indicates holding the joystick.
  • holdCenter: Indicates holding the joystick immediately after touching it in the center.

Direction

  • direction: Represents a BABYLON.Vector3 type for character direction.

Delta Smoothing

Original deltaPosition, but with smoothing.

  • deltaPositionSmoothed: Represents smoothed movements of delta using cube easing. You can use deltaPositionSmoothed.x and deltaPositionSmoothed.y instead of deltaPosition.x and deltaPosition.y.

Usage

TypeScript example:

import TouchStick from 'babylonjs-touchstick'

export class TouchInput {

    private stickLeft: TouchStick = new TouchStick(true);
    private stickRight: TouchStick = new TouchStick(false);
    
    private handleEvents() {
        const { swipe, tap, doubleTap, hold, holdCenter } = this.stickRight;
        
        if (swipe.down) {
            console.log('Swiped down right stick')
        }
        
        if (doubleTap) {
            console.log('Double tap')
        }

        if (holdCenter) {
            console.log('Enter menu')
        }
        
        if (this.stickLeft.swipe.up || this.stickRight.swipe.up) {
            console.log('Swiped up both sticks')
        }
        
    }

}

Read official documentation of original BABYLON.VirtualJoystick for more information: Babylon.js docs

2 Likes

Have you ever thought about adding a parameter to give the option of showing or not showing the visual part of the control, in this case the circles?

1 Like

This feature already exists in base class of VirtualJoystick. I use transparent color for it.

stick = new TouchStick(true)
stick.setJoystickColor('rgba(0,0,0,0)')

or put your image instead of original:

stick.setContainerImage('/img/something.svg')

Looks like hack, I know. :slightly_smiling_face:

BTW: changing image of puck

stick.setPuckImage('/img/puck.svg')

sounds interesting, however I cover it with my finger. :grin:

1 Like

Great!

Thanks!