Locked target camera and user interaction

[sinaeth]

Hello there,

I would try to explain what I am trying to achieve the best I can.
For the website of our web agency, we want to make a virtual tour of our workplace.

To do so, we create a FlyCamera following a path along a Curve3 which locked target is a small Mesh following another Curve3. Both objects move at the same time in an AnimationGroup. Everything here is working fine so far.

Fact is, we would want to have some kind of user interaction on the camera according to the position of the mouse on the screen. Let’s say if the mouse is in the center, the camera should directly point the target. All the way left, and the camera should rotate a bit to look left of the target, and so on with top, right and bottom.

First try was to rotate the camera according to the mouse position but this really did nothing as the locked target seems to prevent any other control on the camera. But as I am new to Babylon I might have done something wrong.

Second thought was to cheat a little by rendering a bigger canvas (150vh, 150vw) and setting up a bigger fov on the camera to move the canvas to simulate user interaction in the scene. This stutters and anyway, the result does not seem “natural” enough.

Now I am asking for your advices, how would you realize something like that ? Is it even possible with this set up?

Thank you in advance,

[Wingnut]

Hi S! I LOVE this topic. https://www.babylonjs-playground.com/#1YD970#120

There’s a testing playground… not like a building-with-rooms’n’hallways scene, but it’s still something to play-with. It has a “carBody” which travels the path, it can be set visible or not, it can be used as a camera parent or position plotter, but NOTE… this car does not pitch and yaw based-upon the path. It should be used for POSITION only, not direction-setting… so parenting camera to car, is likely not useful.

As you can see, all standard camera-navigation keys, pointer-drags, and mousewheel-rolls… fight-with the automated tour… same problems as sinaeth mentioned.

Scene is set-up for free cam or arc cam, and no .lockedTarget. Instead, lines 124-150… is an experimenter’s render-loop area… where demented people play.

Further down, lines 153-183… we have installed a mousewheel for the freeCamera, as wanted. I have it hotwired for Firefox… using event.deltaY. Other browsers may vary. Lines 159 and 163 are lines of concern.

I call this topic… “soft-guided tour”. This is where a “tourist” is taken on an automated guided tour, but can pause the tour at any “point”, then have near-full “investigate something” freedom, yet remind the user that they can return to the guided tour… (with a button?).

In a way, it’s like parenting with rubber bands. The “tourist” is allowed to wander-away from the group, yet have a rubber band to return them to the guided tour. The guided tour SHOULD pause… while one of its tourists wanders-off on a personal exploration/investigation. That way the tourist doesn’t miss anything on the tour. They are returned to the guided tour… at the point where they wandered-off.

Let’s pretend there IS a button/label called “Free-Fly Tour Here”… which, when clicked, stops the guided tour camera positioning/targeting system, changes its label to “Resume Guided Tour”, and then allows user standard FreeCam/UniversalCam.

The “Resume Guided Tour” button (“resume” event)… should do something fancy… at LEAST gently animate back-to guided tour current position. THAT could get a VR-gear wearer… motion sick. Perhaps better to fade-to-black, re-position/target cam, then fade-up again and re-activate the auto-tour with a gentle ease-up-to full tour-speed.

A bit more about rubber bands, magnetism, and guided tours that have automatic “feature-upon” targets. “Featuring-upon” a room or thing… as the guided tour “passes-by”… is a real nice thing. As a slow-moving guided tour travels a virtual hallway, it might be swinging left/right to look-into rooms thru their open doors. In a way, it is “target magnetism”. Even though the “tour guide” sort-of dictates that the camera look straight ahead as the tourist walks the halls, curiosity “pulls” the tourist’s eyes (aiming)… into the open doorways (or other ‘featured’ tour-items) as the tour passes-by. This is done automatically… part of the “featured” things along the soft-guided tour.

During the look-into-the-doorways… is when a user is likely to roll a mousewheel… to see some things inside the room… CLOSER. Let’s say THAT mousewheeling… is an indicator to stop the auto-tour, set camera to free-roam default, and display the “resume tour” button. The tourist has just shown indication that they want to investigate/explore. Detach from tour.

Ok, that’s me, talking way too much, as usual. Soft-targeting… magnetism-targeting… curiosity-based tour detach/resume… all things that COULD be pondered, here. LOVE IT! We could REALLY use camera magnets and magnetic field generators. Essentially, pulling a camera-craft (and/or its aiming target) around in outer space, using gravitational pulls from planetary bodies or invisible black holes. heh. YEE HA. FUN!

A bit more: I think… only the tour guide (or user-done camera navving) can position the camera itself (camBody magnetism/soft-control). Both tour guide and/or featured items… can position the camTarget (camTarget magnetism/soft-control). During (slow?) guided tour, camTarget positioning (to aim at featured items momentarily) needs to be done with gentle animation… to avoid VR head-gear vertigo issues, I suppose. Looking left/right repatedly, into room doors… could cause eyeball vomiting… for VR headgear users.

Let’s say the cameraTarget gets pulled toward an item in a side-room, as the tour passes down the hallway. The tour guide continues down the hallway, using ITS camTarget rubber band… to try to pull the camera’s “attention” back to “hallway-forward” aim. But the item within the side-room, has a strong magnetic pull… and tries to hold the camera’s aim-attention. As the tour guide gets further way, its pull actually increases (tour guide repeatedly insisting that you to stay with the group). Eventually, the tour guide’s increasing pull… out-pulls the featured-item magnetism, and the cameraTarget SHOULD SLOWLY swing back around to the tour guide/hallway-forward. DURING that tug-o-war, overlapping magnetic fields fight-over the cameraTarget position… based upon distances and field strengths. Phew… cool.

Ok, sorry if that’s too deep and heavy. Stay tuned for wiser and less-yappy folk.

[JohnK]

A slightly different way of doing the animation Babylon.js Playground

An empty mesh follows the yellow line.

An arc rotate camera is given the empty mesh as a parent.
NOTE the small radius for the arc rotate camera

A sphere follows the white line
NOTE both lines have identical number of points

The empty mesh is made to lookAt the sphere and after this limits are put on the rotation of the camera.

An onAfterRenderObservable is used for the animation and removed at the end of the animation.

EDIT more complicated version can be found

https://www.babylonjs-playground.com/#SQFG0Q#14

https://doc.babylonjs.com/snippets/track

[sinaeth]

@Wingnut
Well, first thanks a lot for your answer. This help is much appreciated, playground is super helpful.

I will take the time to process everything during the weekend and getting back at it on Monday. This subject is indeed interesting and has more depth than I’ve initially thought. This is not the first time I see posts from you while looking up some issues on this forum and I can tell you know your stuff.

To summarize a lot, what is advised is to separate tour-guide and user-in-control part as both would fight over the camera control otherwise ?

@JohnK
Well thanks also. I will take a look into that also.
I’m still a bit concerned about using ArcRotateCamera as I’m not sure it would fit my needs and to be fair, I doesn’t exactly get how it would behave (comparing to a classic camera, it is a bit harder to visualize), but I would dig into it also.
My concern is that I got a 3d model of a building and I’m afraid that the camera would clip into walls and other 3d objects.

Would get back with some ideas soon

Have a nice weekend.

[withADoveInOneHand]

You can really easily set the camera to collide with meshes for a nice AAA feeling. You might want a second ‘look at’ target that moves independently of the rails, and switch to it with user control then back for autofollow or whatever.

I’d love to see what you come up with - really any camera movement you can imagine can be accomplished.

[JohnK]

@sinaeth

My concern is that I got a 3d model of a building and I’m afraid that the camera would clip into walls and other 3d objects.

You have a set track for the camera position (really a track for the camera parent) and a small radius, r, for the camera orbit. In this case when the track distance from all objects >r the camera will never collide with your models.

The idea of the small radius (0.0001 in my example) is that the camera now seems to be one set on a fixed pivot point that looks outwards in all directions from that pivot rather than as a satellite looking down over a large sphere.

[sinaeth]

Well, arc rotate camera was indeed what I was missing.

For now, the example provided by JohnK is fitting my needs. This will be the first version, and are already thinking of VR and pause/resume tour as suggested by Wingnut for the future. Let’s go step by step

@withADoveInOneHand For now I don’t need interaction so I disabled mesh colliding as I hope it saves up some ressources. But setting the camera radius to a really low value seems to do the trick. I’m keeping your idea as well if needed in the future.

There is something that I don’t get yet. When I tried to use a flyCamera, setting target every frame caused stutter at some steps of the tour - especially at the beginning and when the camera is zoomed out. This was resolved when I switched to lockedTarget.
But the stutter is coming back with ArcRotateCamera. I don’t have a playground yet as I use Babylon with React and Typescript as well as external files but I would create a simple one if needed.

Maybe one of you already got this kind of issue. This should be visible at preprod.purjus.fr. Speed might be a bit high, but I get the issue whatever its value is.
The path is not in its final version

Thank you in advance,

[Pryme8]

Here is what I used in one of my recent projects to do what I think you are asking.

I would give the function a camera target, a point to go to and then a meshs target. What the function did was then generate a bezier curve from that information (this is what you will prolly need to adjust) and then animated the camera.

Not sure if this is what you want, but it did the dirty for me. It was kinda justed hacked together to “get it done” and could be quite refined.

const cameraMove = (target, cameraPoint, meshes, scene)=>{		
			function remove(observer){
					scene.onBeforeRenderObservable.remove(observer)
					scene.cameraIsAnimating = false
					delete scene.lastCameraObserver
					delete scene.lastCameraAnimation
					delete scene.lastCameraTargetAnimation
					scene.hiddenCameraTarget.dispose()
					delete scene.hiddenCameraTarget
			}	
		
			if(scene.cameraIsAnimating){
				scene.lastCameraAnimation.stop()
				scene.lastCameraTargetAnimation.stop()
				remove(scene.lastCameraObserver)							
			}
				scene.cameraIsAnimating = true
		
				var camera = scene.activeCamera			
				var trackedVertex = getPointByID(cameraPoint.trackedVertex, meshes[2])
				
				var pA = scene.activeCamera.position.clone()
				var tv = parseFloat(target.value)/10
				var pB = trackedVertex.add(cameraPoint.offset(tv))
	
				if(pA.x == pB.x &&
				   pA.y == pB.y &&
				   pA.z == pB.z){
					scene.cameraIsAnimating = false
					return false
				}
	
				var pM = (pB.clone().subtract(pA.clone())).scale(0.5).add(pA.clone())
				
				console.log('trackedVertex:'+trackedVertex, 'pA:'+pA, 'pB:'+pB, 'pM:'+pM)
				
				var displaceAmount = 5.2
				var detail = 64
				
				var projectionVector = pM.clone().subtract(meshes[2].position.clone()).normalize()	
				pM.addInPlace(projectionVector.scale(displaceAmount))
			
				var path = BABYLON.Curve3.CreateQuadraticBezier(pA, pM, pB, detail).getPoints()	
				
				
				
				var duration = 100
				var step = Math.floor(duration/path.length)
				var animation = new BABYLON.Animation("cameraSwoop",
				"position", 30,	BABYLON.Animation.ANIMATIONTYPE_VECTOR3)
				
				var keyFrames = []
				
				keyFrames.push({
						frame: 0,
						value: camera.position.clone()
					})
				
				for(var i=1; i<=path.length; i++){
					var ap = path[i-1]
					keyFrames.push({
						frame: step*i,
						value: ap
					})
				}
				
				
				animation.setKeys(keyFrames)
				camera.animations = [animation]	
				
				var et = trackedVertex.add(cameraPoint.targetOffset(tv))
				
				var ht = scene.hiddenCameraTarget = BABYLON.MeshBuilder.CreateBox("box", {size: 0.1}, scene);
				ht.visibility = 0
				
				ht.position = camera.getFrontPosition(3)
				
				
				var path2 = BABYLON.Curve3.CreateCatmullRomSpline(
				[ht.position, et], detail, false).getPoints()

				console.log('path2', path2)
			
				var targetAnimation = new BABYLON.Animation("cameraPan",
				"position", 30,	BABYLON.Animation.ANIMATIONTYPE_VECTOR3)
				
				var keyFrames2 = []
				
				keyFrames2.push({
						frame: 0,
						value: ht.position
					})
				
				for(var i=1; i<=path2.length; i++){
					var ap = path2[i-1]
					keyFrames2.push({
						frame: step*i,
						value: ap
					})
				}
				
				targetAnimation.setKeys(keyFrames2)
				ht.animations = [targetAnimation]
				
				scene.lastCameraObserver
				scene.lastCameraAnimation
				scene.lastCameraTargetAnimation			
				
					
				
				function moveing(){
					camera.setTarget(ht.position)								
				
					if(BABYLON.Vector3.Distance(camera.position, path[path.length-1])==0){
						console.log('STOP!')						
						camera.setTarget(path2[path2.length-1])
						scene.lastCameraAnimation.stop()
						scene.lastCameraTargetAnimation.stop()
						remove(scene.lastCameraObserver)
					}
				}
				
				scene.lastCameraAnimation = scene.beginAnimation(camera, 0, step*path.length, false, 1)	
				scene.lastCameraTargetAnimation = scene.beginAnimation(ht, 0, step*path2.length, false, 1)	
				scene.lastCameraObserver = scene.onBeforeRenderObservable.add(moveing)			
			
		}