Navigation GUI for tablets & mobiles

Hi all

Has any developed or has a referance to a nice & neat Navigation bar using Babylon’s GUI as in the green circle in the attached image. cat sim

Basically I want to use it when my Babylon site runs on a tablet / mobile since I cannot use finger touch for both movement & rotation at the same time using the pep module.

Hi J, sorry for the slow replies. https://www.babylonjs-playground.com/#91I2RE#93

Yeah, it’s broken. Last I knew, something changed in Context2d object… BJS is delivering it’s own version of Context2d, instead of DOM version… and BJS version has no ‘save()’ or ‘restore()’… but I see no errors at console, about that. i once did.

ANYWAY, long ago, I started screwing-around with circular GUI 2d controls. I borrowed Adam’s ColorPicker control, and started hacking. I made a “DragPuck” and a “ButtonRing”… and they are sort-of meant to be laid atop each other. Once upon a time, somewhere in the #91I2RE series… they worked. They were never massaged/professionalized to become an official control. It was all just Wingnut, screwing-around with round controls (which are ALWAYS painted onto a little SQUARE canvas area).

The real issue with round clickable areas… (painted onto square canvai)… is picking zones. It’s all good fun. Overlaying controls atop other controls… is not recommended and comes with its own problems, but in some cases, it works ok. (watch out for isPointerBlocker and z-index stuff, of course) Round controls are ONLY painted round… but they are still square and have canvas in the middle of the doughnut. :slight_smile: (probably only pertinent if overlaying controls atop each other).

All in all, feel free to try to get that playground operational, if you choose-to. I have fought with it a bit, but got frustrated and played MudRunner instead. :slight_smile: You may wish to search old forum for ‘ipod’ or ‘ipod control’, too. I think I once called it that, too. DragPuck, ButtonRing, IpodControl, I tried all sorts of demented round-control experiments… using those terms. Might be worth screwing-with. Or maybe grab a copy of Adam’s ColorPicker, and hack it yourself… make yourself a circular controller control. :slight_smile:

1 Like


or Use Virtual Joysticks - Babylon.js Documentation
but i dont like babylon`s virtual joystick

2 Likes

I was just working with something similar. To be correct, I found a forum post with some working code, I adapted it and it works quite well. Of course, it could be improved in a few things but I simply did not find the time.

Layout side

<?xml version="1.0"?>

<root>
    <Ellipse id="leftJoystick" color="#ffffff" isVisible = "true" alpha = "0.4" zIndex="2" thickness = "3"  paddingLeft="0px" paddingRight="0px" paddingTop="0px" paddingBottom="0px" height="160px" width="160px" isPointerBlocker = "true" verticalAlignment = "Control.VERTICAL_ALIGNMENT_BOTTOM" horizontalAlignment = "Control.HORIZONTAL_ALIGNMENT_LEFT" onPointerDownObservable="onJoystickDown" onPointerUpObservable="onJoystickUp" onPointerMoveObservable="onJoystickPuckMove" >
        <Ellipse id="leftInnerJoystick" color="#ffffff" zIndex="2" thickness = "4" paddingLeft="0px" paddingRight="0px" paddingTop="0px" paddingBottom="0px" height="80px" width="80px" isPointerBlocker = "true" verticalAlignment = "Control.VERTICAL_ALIGNMENT_CENTER" horizontalAlignment = "Control.HORIZONTAL_ALIGNMENT_CENTER" /> 
        <Ellipse id="leftPuck" zIndex="2" thickness = "0" isVisible = "true" background="#c7c41c" paddingLeft="0px" paddingRight="0px" paddingTop="0px" paddingBottom="0px" height="60px" width="60px" isPointerBlocker = "true" verticalAlignment = "Control.VERTICAL_ALIGNMENT_CENTER" horizontalAlignment = "Control.HORIZONTAL_ALIGNMENT_CENTER" /> 
    </Ellipse>
</root>

Then JS side :

let advancedTexture = BABYLON.GUI.AdvancedDynamicTexture.CreateFullscreenUI("UI", undefined, undefined, BABYLON.Texture.NEAREST_NEAREST);
let joystickX = 0;
let joystickZ = 0;
let guiLoader = new BABYLON.GUI.XmlLoader();
guiLoader.loadLayout("joystick.xml", advancedTexture, null);

let onJoystickDown = function (coordinates) {
        guiLoader.getNodeById("leftPuck").isVisible = true;
        guiLoader.getNodeById("leftPuck").floatLeft = coordinates.x - (guiLoader.getNodeById("leftJoystick")._currentMeasure.width * .5);
        guiLoader.getNodeById("leftPuck").left = guiLoader.getNodeById("leftPuck").floatLeft;
        guiLoader.getNodeById("leftPuck").floatTop = advancedTexture._canvas.height - coordinates.y - (guiLoader.getNodeById("leftJoystick")._currentMeasure.height * .5);
        guiLoader.getNodeById("leftPuck").top = guiLoader.getNodeById("leftPuck").floatTop * -1;
        guiLoader.getNodeById("leftPuck").isDown = true;
        guiLoader.getNodeById("leftJoystick").alpha = 0.9;
    }

    let onJoystickUp = function () {
        joystickX = 0;
        joystickZ = 0;
        guiLoader.getNodeById("leftPuck").isDown = false;
        guiLoader.getNodeById("leftPuck").left = "0px";
        guiLoader.getNodeById("leftPuck").top = "0px";
        guiLoader.getNodeById("leftJoystick").alpha = 0.4;
    }
    let onJoystickPuckMove = function (coordinates) {
        if (guiLoader.getNodeById("leftPuck").isDown) {
            joystickX = coordinates.x - (guiLoader.getNodeById("leftJoystick")._currentMeasure.width * .5);
            joystickZ = advancedTexture._canvas.height - coordinates.y - (guiLoader.getNodeById("leftJoystick")._currentMeasure.height * .5);
            guiLoader.getNodeById("leftPuck").floatLeft = _this.joystickX;
            guiLoader.getNodeById("leftPuck").floatTop = _this.joystickZ * -1;
            guiLoader.getNodeById("leftPuck").left = guiLoader.getNodeById("leftPuck").floatLeft;
            guiLoader.getNodeById("leftPuck").top = guiLoader.getNodeById("leftPuck").floatTop;
        }
    }

I can not promise this will simply work out of the box. You would need to implement it in your own context. You can also not use XML and build the layout in JS, but, why would you want to do that :smiley: ?

Cheers!

1 Like

Why? How can we improve it?

Some feedback my side.

1 - It can not be made sticky.
2 - It can not be made always visible
3 - It can not be modified ( size, color, etc )

Let’s wait for @kvasss feedback and let’s create an issue to capture the need (@thomlucc)

1 Like

hi. I will be able to see in the coming days as soon as my hands reach mobile navigation in current project

1 Like

I applied nippleJS, Much superior than Babylon’s joystick, but I found out that it conflicts with pep polyfill on IPAD devices (Android works fine)

Hi @kvasss - I’ve created the issue #7398: feel free to add your feedback in the comments when you get a chance. Thanks!!!

1 Like

the biggest problem with joysticks is that they cannot be used in a fixed limited area but only in full-screen mode, because of which other logic of interaction with the scene is lost

1 Like

This is why I’m really interested in recreating them. So if we can use the issue to capture the needs that would be cool

hi sir, can you show me how to rotate my character when i drag the joystick?

Can’t remember where I fetched this code but works fine

let adt = BABYLON.GUI.AdvancedDynamicTexture.CreateFullscreenUI(“UI”);
var xAddPos = 0;
var yAddPos = 0;
var xAddRot = 0;
var yAddRot = 0;
var translateTransform;
function detectmob() {
if( navigator.userAgent.match(/Android/i)
|| navigator.userAgent.match(/webOS/i)
|| navigator.userAgent.match(/iPod/i)
|| navigator.userAgent.match(/BlackBerry/i)
|| navigator.userAgent.match(/Windows Phone/i)
)
{return “mobile” ; }
else if (navigator.userAgent.match(/iPhone/i)
|| navigator.userAgent.match(/iPad/i)
)
{ return “tablet”;}
else
{return “desktop”; }
}
var agent = detectmob();

		function makeThumbArea(name, thickness, color, background, curves){
		   var rect = new BABYLON.GUI.Ellipse();
		   rect.name = name;
		   rect.thickness = thickness;
		   rect.color = color;
		   rect.background = background;
		   rect.paddingLeft = "0px";
		   rect.paddingRight = "0px";
		   rect.paddingTop = "0px";
		   rect.paddingBottom = "0px";
	
			return rect;
		}
	
		let leftThumbContainer = makeThumbArea("leftThumb", 2, "blue", null);
			if (agent =="mobile")
			{
			leftThumbContainer.height = "80px";
			leftThumbContainer.width = "80px";
			}
			else if (agent =="tablet")
			{
			leftThumbContainer.height = "100px";
			leftThumbContainer.width = "100px";
			}
			else
			{
			leftThumbContainer.height = "160px";
			leftThumbContainer.width = "160px";
			}
			leftThumbContainer.isPointerBlocker = true;
			leftThumbContainer.horizontalAlignment = BABYLON.GUI.Control.HORIZONTAL_ALIGNMENT_LEFT;
			leftThumbContainer.verticalAlignment = BABYLON.GUI.Control.VERTICAL_ALIGNMENT_BOTTOM;
			leftThumbContainer.alpha = 0.4;
	
		let leftInnerThumbContainer = makeThumbArea("leftInnterThumb", 4, "blue", null);
			if (agent =="mobile")
			{
			leftInnerThumbContainer.height = "50px";
			leftInnerThumbContainer.width = "50px";
			}
			else if (agent =="tablet")
			{
			leftInnerThumbContainer.height = "50px";
			leftInnerThumbContainer.width = "50px";
			}
			else
			{
			leftInnerThumbContainer.height = "80px";
			leftInnerThumbContainer.width = "80px";
			}
			leftInnerThumbContainer.isPointerBlocker = true;
			leftInnerThumbContainer.horizontalAlignment = BABYLON.GUI.Control.HORIZONTAL_ALIGNMENT_CENTER;
			leftInnerThumbContainer.verticalAlignment = BABYLON.GUI.Control.VERTICAL_ALIGNMENT_CENTER;
	
	
		let leftPuck = makeThumbArea("leftPuck",0, "blue", "blue");
			if (agent =="mobile")
			{
			leftPuck.height = "60px";
			leftPuck.width = "60px";
			}
			else if (agent =="tablet")
			{
			leftPuck.height = "60px";
			leftPuck.width = "60px";
			}
			else
			{
			leftPuck.height = "60px";
			leftPuck.width = "60px";
			}
			leftPuck.isPointerBlocker = true;
			leftPuck.horizontalAlignment = BABYLON.GUI.Control.HORIZONTAL_ALIGNMENT_CENTER;
			leftPuck.verticalAlignment = BABYLON.GUI.Control.VERTICAL_ALIGNMENT_CENTER;
	
	
		 leftThumbContainer.onPointerDownObservable.add(function(coordinates) {
														
				leftPuck.isVisible = true;
				leftPuck.floatLeft = coordinates.x-(leftThumbContainer._currentMeasure.width*.5);
				leftPuck.left      = leftPuck.floatLeft;
				leftPuck.floatTop  = adt._canvas.height - coordinates.y-(leftThumbContainer._currentMeasure.height*.5);
				leftPuck.top       = leftPuck.floatTop*-1;
				leftPuck.isDown    = true;
				leftThumbContainer.alpha = 1;
			});
	
		 leftThumbContainer.onPointerUpObservable.add(function(coordinates) {
				xAddPos = 0;
				yAddPos = 0;
				leftPuck.isDown = false;
				leftPuck.isVisible = false;
				leftThumbContainer.alpha = 0.4;
			});
	
	
		 leftThumbContainer.onPointerMoveObservable.add(function(coordinates) {
				if (leftPuck.isDown) {
					xAddPos = coordinates.x-(leftThumbContainer._currentMeasure.width*.5);
					yAddPos = adt._canvas.height - coordinates.y-(leftThumbContainer._currentMeasure.height*.5);
					leftPuck.floatLeft = xAddPos;
					leftPuck.floatTop = yAddPos*-1;
					leftPuck.left = leftPuck.floatLeft;
					leftPuck.top = leftPuck.floatTop;
					}
			});
	
		 adt.addControl(leftThumbContainer);
		 leftThumbContainer.addControl(leftInnerThumbContainer);
		 leftThumbContainer.addControl(leftPuck);
		 leftPuck.isVisible = false;
		
		 
		 let rightThumbContainer = makeThumbArea("rightThumb", 2, "red", null);
		   if (agent =="mobile")
			{
			rightThumbContainer.height = "80px";
			rightThumbContainer.width = "80px";
			}
			else if (agent =="tablet")
			{
			rightThumbContainer.height = "100px";
			rightThumbContainer.width = "100px";
			}
			else
			{
			rightThumbContainer.height = "160px";
			rightThumbContainer.width = "160px";
			}
			 
			 
			
			 rightThumbContainer.isPointerBlocker = true;
			 rightThumbContainer.horizontalAlignment = BABYLON.GUI.Control.HORIZONTAL_ALIGNMENT_RIGHT;
			 rightThumbContainer.verticalAlignment = BABYLON.GUI.Control.VERTICAL_ALIGNMENT_BOTTOM;
			 rightThumbContainer.alpha = 0.4;
	
		 let rightInnerThumbContainer = makeThumbArea("rightInnterThumb", 4, "red", null);
			if (agent =="mobile")
			{
			rightInnerThumbContainer.height = "50px";
			rightInnerThumbContainer.width = "50px";
			}
			else if (agent =="tablet")
			{
			rightInnerThumbContainer.height = "50px";
			rightInnerThumbContainer.width = "50px";
			}
			else
			{
			rightInnerThumbContainer.height = "80px";
			rightInnerThumbContainer.width = "80px";
			}
			
			 rightInnerThumbContainer.isPointerBlocker = true;
			 rightInnerThumbContainer.horizontalAlignment = BABYLON.GUI.Control.HORIZONTAL_ALIGNMENT_CENTER;
			 rightInnerThumbContainer.verticalAlignment = BABYLON.GUI.Control.VERTICAL_ALIGNMENT_CENTER;
	
	
		 let rightPuck = makeThumbArea("rightPuck",0, "red", "red");
			  if (agent =="mobile")
				{
				rightPuck.height = "60px";
				rightPuck.width = "60px";
				}
				else if (agent =="tablet")
				{
				rightPuck.height = "60px";
				rightPuck.width = "60px";
				}
				else
				{
				rightPuck.height = "60px";
				rightPuck.width = "60px";
				}
				 rightPuck.isPointerBlocker = true;
				 rightPuck.horizontalAlignment = BABYLON.GUI.Control.HORIZONTAL_ALIGNMENT_CENTER;
				 rightPuck.verticalAlignment = BABYLON.GUI.Control.VERTICAL_ALIGNMENT_CENTER;
	
	
			 rightThumbContainer.onPointerDownObservable.add(function(coordinates) {
				 rightPuck.isVisible = true;
				 rightPuck.floatLeft = adt._canvas.width - coordinates.x-(rightThumbContainer._currentMeasure.width*.5);
				 rightPuck.left = rightPuck.floatLeft*-1;
				 rightPuck.floatTop = adt._canvas.height - coordinates.y-(rightThumbContainer._currentMeasure.height*.5);
				 rightPuck.top = rightPuck.floatTop*-1;
				 rightPuck.isDown = true;
				 rightThumbContainer.alpha = 0.9;
			 });
	
			 rightThumbContainer.onPointerUpObservable.add(function(coordinates) {
				 xAddRot = 0;
				 yAddRot = 0;
				 rightPuck.isDown = false;
				 rightPuck.isVisible = false;
				 rightThumbContainer.alpha = 0.4;
			 });
	
	
			 rightThumbContainer.onPointerMoveObservable.add(function(coordinates) {
				 if (rightPuck.isDown) {
					
					 xAddRot = adt._canvas.width - coordinates.x-(rightThumbContainer._currentMeasure.width*.5);
					 yAddRot = adt._canvas.height - coordinates.y-(rightThumbContainer._currentMeasure.height*.5);
					 rightPuck.floatLeft = xAddRot*-1;
					 rightPuck.floatTop = yAddRot*-1;
					 rightPuck.left = rightPuck.floatLeft;
					 rightPuck.top = rightPuck.floatTop;
					 }
		 });

		  adt.addControl(rightThumbContainer);
		  rightThumbContainer.addControl(rightInnerThumbContainer);
		  rightThumbContainer.addControl(rightPuck);
		  rightPuck.isVisible = false;
2 Likes

Finally you will have to add the following code in registerBeforeRender

translateTransform = BABYLON.Vector3.TransformCoordinates(new BABYLON.Vector3(xAddPos/120, 0, yAddPos/120), BABYLON.Matrix.RotationY(camera.rotation.y));
scene.activeCamera.cameraDirection.addInPlace(translateTransform);
scene.activeCamera.cameraRotation.y += xAddRot/40000 *-1;
scene.activeCamera.cameraRotation.x += yAddRot/40000 *-1;

The above is used for touch screens, but in case you want to use the joystick XML "plugin’, you can use the same concept utilizing coordinates

1 Like

I did a playground with this code and some code refactoring:

  • desktop: double click toggle fullscreen
  • mobile: click fullscreen
  • advancedDynamicTexture scale:

Outside the playground, sometimes we neet to reset scale after a “resize” event or the GUI will be stretched. to fix it:

window.addEventListener("resize", () => { 
   advancedDynamicTexture.scaleTo(canvas.width, canvas.height);
});
// or window.innerWidth, window.innerHeight
  • Padding left, right and bottom
  • Adding gap to allow moving the joystick without affecting the Pointer
  • fix camera move with gap
const gap = 30;
leftThumbContainer.left = gap;
leftThumbContainer.top = -gap;
rightThumbContainer.left = -gap;
rightThumbContainer.top = -gap;
3 Likes

Indeed, why they still not improve this?

Who is “They”? I hope you are not speaking about the joystick manufacturers because this will likely never happen :grinning: On the side of browsers, just don’t count on Apple to make any effort for webGL :grin: And for whatever WE can do on the side of BJS, this is all free and open contribution (and most people don’t have the time and resources to work it beyond a certain limit and over time). Of course, if you can help with it, I shall be the first to applause and praise you for your efforts :smiley:

1 Like

hi Is there a demo or playground based on nipple.js ?