This has been tested and does reproduce with Meta Quest 2 (OculusBrowser) and Meta immersive web emulator extension (chromium 122 desktop). Babylon 6.45.1. This is an issue with hands, not an issue with controllers.
Setup:
active immersive-vr session with xr-controller-teleportation “feature” enabled with defaults as per the “DefaultXRExperience”
scene has a designated floor mesh (for teleportation purposes), and another mesh
user uses hands to interact with the session (rather than controllers; this is important)
Steps:
user directs a hand’s ray such that it 1) hits a non-floor mesh first and 2) if continued after hitting that, also hits a floor mesh
user executes the “select” action (pinch thumb&index fingers) with that hand
EXPECTED: Teleportation UX (target&torus meshes, direct or parabolic ray pointing to them, etc) does not initialize
OBSERVED: Teleportation UX does initialize and becomes visible
Sample screenshot with playground Babylon.js Playground (taken from basic documentation on teleportation) and meta immersive web emulator:
On this screenshot we see a parabolic teleportation ray, but the issue reproduces with direct one as well.
the user did not first pinch “select” while pointing directly at a floor and later move the teleportation ray to look as it looks at the screenshot (which would have been the intended behavior in that case, if I understand it correctly from here ). Instead the user pinched “select” action while pointing directly at the “IcoSphere” mesh (just from such an angle that also hits the floor, if ray were to continue… the point is that the ray should not have continued at all beyond hitting the “IcoSphere” in this particular use case).
finally and importantly, if all the same actions are executed in the same exact setup but with a Quest 2 controller (executing “push thumbstick forward” action instead of “pinch fingers to activate webxr select” action) then the issue does not reproduce. Specifically, in that case “nothing happens” when user pushes the thumbstick forward, i.e. the teleportation UX does not initialize.
I suspect that the technical reason for the current difference in behavior between hands and controllers has something to do with different events generated in these two different situations (hand pinch generates the webxr default action aka “select”, but controller thumbstick push generates a completely different event). But in any case, there are two problems with the current hand behavior I described:
It is not consistent with controller behavior
It is annoying. Arguably, it’s bad UX (while the controller behavior is arguably natural, good UX)
Teleportation is fully configurable. There are many flags to set, if you want to change the default behavior.
What you are referring to is a main-component-only scenario, where a controller does not have a thumbstick, so there is nothing to “push” forward. This can be disabled. You can also define blocking elements that will block the teleportation ray from going through it. It’s all part of the options when enabling and configuring the teleportation ray.
As a developer, you have the freedom to do whatever you think is right to your use case. We provide many flags and configuration parameters for you to make it work as you expect it to. If you think we are missing something, I will be very (VERY) happy to add it. I will be happy to discuss what you think is a good UX scenario when trying to teleport with hands only. A different gesture? that might require educating your users. But it is still technically possible.
One thing I did notice I am missing from the teleportation is the ability to trigger the teleportation yourself and not using events. I will be happy to look into that and see if I can add that. This way, you can define your gestures/interaction and trigger the teleportation yourself. I still think that, for ease of use, using hotspots and direct ray is the better scenario. Or, not using the teleportation at all, and build your own movement in your scene, when using hands. Teleportation is wonderful when you have a controller, as you have all of those extra interaction possibilities, but with a hand and a single event (select, leaving squeeze out of it for now) you can have movement except for forcing teleportation that way we do it, or creating hot spots and moving there when the user selects them.
OH! one last thing - PLEASE don’t stop asking these questions I am really happy answering them, as they either validate things are working or give me the right push to think of a different solution, if and when presented. I love those discussions. Also, don’t take my answer as the absolute truth. I know I don’t know everything, and am always happy to learn
I appreciate that teleportation feature is configurable and there is pickBlockerMeshes setting among others.
The reason I posted this as a bug is that I argue that the default behavior is bad UX, and I propose an alternative default behavior that’s so much better in such a great majority of the use cases that it’s worth making a (technically) breaking change and fixing it.
Let me describe the default teleportation behavior I propose in more detail than I did initially:
Setup: webxr immersive session is active, babylon’s teleportation feature is enabled (without any pickBlockerMeshes specified), but isn’t active at the moment, i.e. the user doesn’t see the target meshes for selecting where to teleport.
User engages controller’s component that’s dedicated to teleportation, i.e. “pinch” aka “select” if the controller is a hand or “push thumbstick forward” if that’s a Quest controller or whatever action is configured to activate teleportation.
IF the pointer ray at that frame hits the floor directly (i.e. before it passes though any other mesh of the scene), THEN (1) teleportation should activate, i.e. the teleportation target meshes should apear on the floor, they should start following the pointer ray on the floor, etc
ELSE IF the pointer ray hits a non-floor mesh directly and, if continued, also hits a floor mesh THEN (2) teleportation should NOT activate
ELSE (3) teleportation should not activate
END IF
That’s the proposal.
Notice how it’s not specific to hands. I propose a consistent behavior regardless of the controller used.
On Quest, if a controller with a thumbstick is used, everything already works as I propose here.
However, if a hand is used, then (2) doesn’t currently behave as I propose: teleportation activates in that case.
Before I go on educating everybody how much better this default behavior is compared to current default and why you should seriously consider changing the default behavior… Is the proposal clear?
First, not a bug. We had this behavior since the happy WebVR days.
Diving into a bit of a technical explanation - Teleportation is in no way related to the pointer selection feature (i.e. pointer events and mesh selection). They are deliberately independent from one another, so you could disable/enable any feature without stopping the other one. They are semi-connected though - when a teleportation is triggered, the pointer selction feature is being disabled so it wouldn’t disturb the teleportation from working.
What you are describing can be achieved with very little configuration. Adding blocker meshes and disabling the parabolic ray.
I do partly agree with you on one thing - perhaps pickable meshes can disable the teleportation instead of just setting the all in the blocker node array. It was done this way for perf reasons - no need to run through all of the meshes in the scene to filter which nodes can be picked - floor meshes and blocked meshes are the only one participation in the pickWithRay call. I don’t mind adding a flag to use the scene’s default pointer down predicate on top of the existing predicate. I would even consider making it the default behavior, though it will hurt performance in scenes with many complex meshes. I’m still thinking about that. Anyhow - enabling the scene predicate will allow any mesh that is pickable, visible and enabled to block the teleportation ray.
That’s simply wrong. In fact the issue reproduces with quest 2 controllers (pushing thumbstick forward to teleportation) just as well as with hands. Not sure how I was testing previously. Regardless…
What you are describing can be achieved with very little configuration. Adding blocker meshes and disabling the parabolic ray.
Well, it’s close but not quite. Let me explain. Please be patient with me. There is a clear difference between the behavior that I proposed and the behavior that’s achieved by Adding blocker meshes and disabling the parabolic ray in BJS 6.45.1. You may (or may not) say that the difference is minor and/or it may be hard to implement (as default or non-default behavior) or you may not prefer what I had described. You can make your decisions on what to implement and how, but let’s first be clear on the behavior that I propose.
First I will describe the behavior of bjs 6.45.1 after adding a non-floor mesh to pickBlockerMeshes. Let’s say we have two meshes: A (in the pickBlockerMeshes) and FLOOR (in floorMeshes). User points the ray of their hand/controller to A from such an angle that if the ray is continued through A it also hits FLOOR. User executes the “teleportation” action (push thumbstick forward or pinch fingers). Here is what happens: teleportation is triggered, the pointer selction feature is being disabled so it wouldn’t disturb the teleportation from working, the ray turns red (to indicate that where the ray is pointing is not a valid teleportation destination), the user does not see the teleportation target meshes (but we are in the “teleportation mode” nevertheless). If the user keeps holding the teleportation “mudra” (i.e. thumbstick pushed forward or thumb&index fingers together) and turns their hand/controller/ray such that it does not hit A but hits FLOOR directly, the ray becomes white, the teleportation target meshes appear, etc: we are in the teleportation mode and will remain in there until the user releases “mudra” of their hand/controller (or timeout occurs).
Now let me describe the behavior I proposed. We have the same two meshes, A and FLOOR. User points the ray of their hand/controller to A from such an angle that if the ray is continued through A it also hits FLOOR. User executes the “teleportation” action (push thumbstick forward or pinch fingers). Here is what I propose to happen: teleportation isn’t triggered; the ray does not become red; if the user now keeps the “mudra” and moves hand/controller/ray such that it hits FLOOR directly the teleportation target does not appear (because we aren’t in the “teleportation mode”).
Do you see the difference?
If you decide not to implement it (as the new default or an option) for whatever reason, that’s fine with me, as long as you understand the proposal. If this isn’t implemented, I’ll configure tepeportation the way that’s the closest possible to that.
I now noticed that “teleportation mode” also activates when the teleportation action on the hand/controller performed while the ray points away from all floors and doesn’t hit a floor at all (not even through other meshes). If the user brings the ray down to the floor while keeping the “mudra”, they see the teleportation target. I think that’s also unexpected and awkward. I think it shouldn’t activate. I don’t even think it should be configurable. Just don’t activate “teleportation mode” when the ray is pointing away from floors, seems logical to me, and I don’t see any users depend on current behavior (so changing this, although technically a breaking change, may be safe). Consider changing that, too?
Can you please elaborate a bit more? I don’t know what scene’s default pointer down predicate is, what the existing predicate is, what it means to use one on top of the other, and what kind of flag do you mean here specifically…
Can you please explain why the parabolic ray is relevant here and how disabling it can help? I didn’t notice a relevant difference in behavior between direct and parabolic teleportation rays
Of course - in the steps you have described you have mentioned that it should work only when directly pointing at the floor. This is why I mentioned disabling the parabolic ray
Sure - the scene’s default picking predicate decides what meshes are pickable per default (mouse/touch) in any scene. The general gist is - if a mesh is enabled, visible and pickable, select it as part of the pointer down process. This can be used in the teleportation as well. Teleportation now takes all pickable meshes, if you ask it to (using the block all pickable flag).
Sure makes sense to me. In this case it is only a problem when using hands, but can be solved quite easily.
Oh, and - on that we will agree to disagree, i guess
Teleportation should tell you - no, you can’t teleport, if you asked it to start teleporting. This is proper UX (when using controllers). I agree that, in case of main controller only, there is no reason to show the red teleportation line. But in a controller mode? The user has explicitly asked to be teleported, they should be notified whether or not it is being triggered.
I am really happy answering them, as they either validate things are working or give me the right push to think of a different solution, if and when presented. I love those discussions. Also, don’t take my answer as the absolute truth. I know I don’t know everything, and am always happy to learn