XR camera rotation in first person walkthrough

Questions
1

Hi!
I’m a complete newbie when it comes to babylonjs and 3D in general.
I’m trying to build a simple walkthrough to be used in combination with an Oculus headset.
I have a very basic scene which is a single room with a door. Just something miniamlistic to be able to test movement with collisions and rotation.
What I’m trying to achieve is for the user to be able to move and rotate in the scene without phisically moving, just by moving left and right joysticks.

At the moment I have the following working fine:

  • Controlling movement (left, right, forwards and backwards) with right joystick
  • Movement does not depend on position of headset (where you look at). So if for example you look to the right and push the joystick forward you go forward while looking to the right. You do not move to the right.
  • Collision detection
  • Fixed y position to avoid “flying” and “diving” into the floor

The only thing I cannot get to work properly is rotation with the left joystick.
I’ve tried asking AI but it never gets it right. When it gives me the code to achieve rotation it messes everything up. I need the user to “spin in place” but all the code its giving me till now doesnt manage the rotation well and even messes up movement too.

I tried creating a Playground but couldnt get it to work. I was getting errors all the time. I guess I dont understand how it works. I’m so sorry about that. :smiling_face_with_tear:

This is my code at the moment (with no code managing rotation).

I would be so grateful if someone could help me out.
Thanks in advance!!

import {
	Engine,
	Scene,
	ArcRotateCamera,
	Vector3,
	HemisphericLight,
	MeshBuilder,
	Color3,
	StandardMaterial
} from "@babylonjs/core";
import "@babylonjs/loaders"; // In case we load assets later
import "@babylonjs/core/XR/features/WebXRControllerMovement";

const COLLIDER_HEIGHT = 1.7;
const FIXED_Y = COLLIDER_HEIGHT / 2;
const INITIAL_POSITION = new Vector3(0, FIXED_Y, 0);

const canvas = document.getElementById("renderCanvas");
const engine = new Engine(canvas, true);

const createScene = async () => {
	const scene = new Scene(engine);
	scene.clearColor = new Color3(0.8, 0.9, 1.0);
	scene.collisionsEnabled = true;

	// Light
	const light = new HemisphericLight("light", new Vector3(0, 1, 0), scene);
	light.intensity = 0.8;

	// Optional: non-XR camera for testing on desktop
	const camera = new ArcRotateCamera("camera", Math.PI / 2, Math.PI / 3, 10, new Vector3(0, 1, 0), scene);
	camera.attachControl(canvas, true);

	// Create room
	await createRoom(scene);

	// Add WebXR experience
	const xrHelper = await scene.createDefaultXRExperienceAsync({
		disableTeleportation: true,
	});

	const input = xrHelper.input;

	// Create collider
	const collider = getCollider(scene, {
		height: COLLIDER_HEIGHT,
		position: INITIAL_POSITION
	});

	scene.onBeforeRenderObservable.add(() => {
		const cam = scene.activeCamera;
		
		let move = new Vector3(0, 0, 0);

		if (!input.controllers || input.controllers.length === 0) {
			// No controllers detected
		}
		else {
			input.controllers.forEach(controller => {
				const handedness = controller.inputSource.handedness;
				let axes = null;

				// Find the thumbstick component (usually called "xr-standard-thumbstick")
				if (controller.motionController && controller.motionController.components) {
					for (const compId in controller.motionController.components) {
						const comp = controller.motionController.components[compId];

						if (comp.type === "thumbstick" && comp.axes) {
							axes = comp.axes;
							break;
						}
					}
				}

				if (!axes || typeof axes.x !== "number" || typeof axes.y !== "number") {
					return;
				};

				// Left controller: rotation
				if (handedness === "left") {
					const leftX = axes.x;
					
				  
				}

				// Right controller: movement
				if (handedness === "right") {
					const rightX = axes.x;
					const rightY = axes.y;
					
					if (Math.abs(rightX) > 0.01 || Math.abs(rightY) > 0.01) {
						const forward = new Vector3(0, 0, 1);
						const right = new Vector3(1, 0, 0);
						move.addInPlace(right.scale(-rightX * 0.1)); // left/right
						move.addInPlace(forward.scale(rightY * 0.1)); // forward/backward
					}
				}
			});
		}

		move.y = 0; // Only move horizontally

		collider.moveWithCollisions(move);
		collider.position.y = FIXED_Y;

		cam.position.copyFrom(collider.position);
	});

	return scene;
};

createScene().then(scene => {
	engine.runRenderLoop(() => {
		scene.render();
	});
});

window.addEventListener("resize", () => {
	engine.resize();
});

async function createRoom(scene) {
	// Floor
	const floor = MeshBuilder.CreateGround("floor", { width: 6, height: 6 }, scene);
	const floorMat = new StandardMaterial("floorMat", scene);
	floorMat.diffuseColor = new Color3(0.6, 0.8, 0.9);
	floor.material = floorMat;

	// Door parameters
	const doorWidth = 1.5;
	const doorHeight = 2.2;
	const wallHeight = 3;
	const wallThickness = 0.2;
	const roomSize = 6;

	const wallMat = new StandardMaterial("wallMat", scene);
	wallMat.diffuseColor = new Color3(1, 1, 1);

	const wall1 = MeshBuilder.CreateBox("wall1", {
		width: roomSize,
		height: wallHeight,
		depth: wallThickness
	}, scene);
	wall1.position.z = -roomSize / 2;
	wall1.position.y = wallHeight / 2;
	wall1.material = wallMat;

	// Wall 2 (left)
	const wall2 = MeshBuilder.CreateBox("wall2", {
		width: wallThickness,
		height: wallHeight,
		depth: roomSize
	}, scene);
	wall2.position.x = -roomSize / 2;
	wall2.position.y = wallHeight / 2;
	wall2.material = wallMat;

	// Wall 3 (right)
	const wall3 = MeshBuilder.CreateBox("wall3", {
		width: wallThickness,
		height: wallHeight,
		depth: roomSize
	}, scene);
	wall3.position.x = roomSize / 2;
	wall3.position.y = wallHeight / 2;
	wall3.material = wallMat;

	// Left part of front wall
	const frontWallLeft = MeshBuilder.CreateBox("frontWallLeft", {
		width: (roomSize - doorWidth) / 2,
		height: wallHeight,
		depth: wallThickness
	}, scene);
	frontWallLeft.position.z = roomSize / 2;
	frontWallLeft.position.y = wallHeight / 2;
	frontWallLeft.position.x = -((roomSize - doorWidth) / 4 + doorWidth / 2);
	frontWallLeft.material = wallMat;

	// Right part of front wall
	const frontWallRight = MeshBuilder.CreateBox("frontWallRight", {
		width: (roomSize - doorWidth) / 2,
		height: wallHeight,
		depth: wallThickness
	}, scene);
	frontWallRight.position.z = roomSize / 2;
	frontWallRight.position.y = wallHeight / 2;
	frontWallRight.position.x = ((roomSize - doorWidth) / 4 + doorWidth / 2);
	frontWallRight.material = wallMat;

	// Top part above the door
	const frontWallTop = MeshBuilder.CreateBox("frontWallTop", {
		width: doorWidth,
		height: wallHeight - doorHeight,
		depth: wallThickness
	}, scene);
	frontWallTop.position.z = roomSize / 2;
	frontWallTop.position.y = doorHeight + (wallHeight - doorHeight) / 2;
	frontWallTop.position.x = 0;
	frontWallTop.material = wallMat;

	// Enable collisions
	floor.checkCollisions = true;
	wall1.checkCollisions = true;
	wall2.checkCollisions = true;
	wall3.checkCollisions = true;
	frontWallLeft.checkCollisions = true;
	frontWallRight.checkCollisions = true;
	frontWallTop.checkCollisions = true;
}

function getCollider(scene, options) {
	const collider = MeshBuilder.CreateBox("xrCollider", {
		width: 0.25,
		height: options.height || 1.7,
		depth: 0.25
	},
	scene);

	collider.isVisible = false;
	collider.checkCollisions = true;
	const colliderMat = new StandardMaterial("colliderMat", scene);
	colliderMat.diffuseColor = new Color3(1, 0, 0);
	collider.material = colliderMat;
	collider.position = options.position || new Vector3(0, options.height / 2, 0);
	return collider;
}
2

Please take a look at xr-extension topic. I hope you will find it useful.

xr-extension

3

Thanks Blax.
I was hoping for help with some code regarding the rotation issue I’m facing.
I’m already struggling with this small example and reading your article and watching your videos has confused me even more, sorry. As I said, I have very little knowledge regarding babylonjs and 3D in general.
Thanks for your reply.

4

The main thing is to start diving in :grinning_face_with_smiling_eyes:

I also had issues with rotation — one approach you can try is creating a Quaternion with the desired rotation (for example, 0.1 degrees around the Y axis) and applying it to the camera on the relevant frames:

helper.baseExperience.camera.rotationQuaternion = myYRotQuaternion;

p.s.
I don’t know the specifics of your project, but in general, forcibly moving or rotating the user within the scene is considered bad practice and often leads to motion sickness.

The headset’s sensors are specifically designed to provide position and orientation data so that the user can feel grounded in the virtual space. Forced movement and rotation should be handled with care.

1 Like
5

Thanks for your suggestion. I will look into the rotationQuaternion you mention. Don’t know where to start though :sweat_smile:.

What I need is the user to be able to move around the scene but without phisically moving. Just when like you’re playing a game sitting on your couch. The player can turn around to move in any direction without having to phisically turn around. Imagine you’re standing and you want to walk in the direction opposite to where you’re looking at. You’d move the rotation joystick left or right until you turn 180deg and then push the movement joystick forward to move in that direction.
In fact I have it working this exact way in my desktop version.

Thanks

6

ok, now i see.
In this case just rotation of camera with use Quaternions should help.

Btw, if you will use xr-extension you may use navigate function of NavigateInSpaceAux for this purpose.

8

You can use WebXRCamera.cameraRotation to rotate the WebXR view using a controller. This is how it is done in the WebXRControllerMovement feature documented at WebXRMovement.

Here’s a playground that does this using your example:
https://playground.babylonjs.com/#W3TR8F

Note that I changed the cam variable to xrCamera to show it is using the current WebXRCamera instead of the active camera.

9

Thanks so much for the playground docEdub!

I tried your example with the Oculus joysticks. It works but there’s something not quite right:
If I’m outside the room and rotate, once rotation has finished movement with right joystick is still working fine, but if I go inside of the room and rotate, after that, movement with right joystick is messed up, basically movement is inverted (for example if I move joystick to one side I move to the opposite side in the scene).

Would you be so kind to help me out with this final tweak?

Thanks in advance!

10

Hi again.
AI was helpful this time.
Got it working with a small tweak.

Replaced:

const forward = new Vector3(0, 0, 1);
const right = new Vector3(1, 0, 0);
move.addInPlace(right.scale(-rightX * 0.1)); // left/right
move.addInPlace(forward.scale(rightY * 0.1)); // forward/backward

With:

const forward = new Vector3(0, 0, -1);
const right = new Vector3(1, 0, 0);
let localMove = right.scale(rightX * 0.1).add(forward.scale(rightY * 0.1));

// Rotate movement by camera's Y rotation
const yRotation = xrCamera.rotationQuaternion
	? xrCamera.rotationQuaternion.toEulerAngles().y
	: xrCamera.rotation.y;

localMove = Vector3.TransformCoordinates(localMove, Matrix.RotationY(yRotation));
move.addInPlace(localMove);

Thanks so much for everybody’s help, especially docEdub for setting me in on the right path.

Cheers!

2 Likes