# BabylonNative and OnPointerDown (or how to select a mesh)

**URL:** <https://forum.babylonjs.com/t/babylonnative-and-onpointerdown-or-how-to-select-a-mesh/17983>\
**Category:** Questions\
**Tags:** native, input\
**Created:** [January 29, 2021, 12:18pm UTC](https://forum.babylonjs.com/t/babylonnative-and-onpointerdown-or-how-to-select-a-mesh/17983 "2021-01-29T12:18:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Peppe.Belgenio](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/peppe.belgenio/32/14042_2.png) [@Peppe.Belgenio](https://forum.babylonjs.com/u/Peppe.Belgenio)\
**Post date:** [January 29, 2021, 12:18pm UTC](https://forum.babylonjs.com/t/babylonnative-and-onpointerdown-or-how-to-select-a-mesh/17983/1 "2021-01-29T12:18:00Z")

</div>

Hello All,  
I am trying to pick an object using Babylon Native on iOS.  
My goal is to pick the object in an AR environment but also without AR I am not able to make this work.

Have anyone succeded to pick meshes under touch in babylon native and AR?

Thank you very much.

Here’s is everything I’ve tried in the past days but nothing seems to works on Native.  
None of the logs is ever written.

```
scene.onPointerObservable.add((pointerInfo) => {
            switch (pointerInfo.type) {
                case BABYLON.PointerEventTypes.POINTERDOWN:
                    BABYLON.Tools.Log("onPointerDown");
                    break;
                case BABYLON.PointerEventTypes.POINTERUP:
                    BABYLON.Tools.Log("onPointerUo");
                    break;
                case BABYLON.PointerEventTypes.POINTERMOVE:
                    BABYLON.Tools.Log("onPointerMove");
                    break;
            }
        });

        scene.onPointerDown = (evt, pickInfo, type) =>
        {
            BABYLON.Tools.Log("onPointerDown");

            if ( anchors && pickInfo) {
                                    
                anchors.addAnchorPointUsingHitTestResultAsync(xrHitTestModule);
            }
        };
        
        scene.onPointerPick = (evt, pickInfo) =>
        {
            BABYLON.Tools.Log("onPointerPick");

            if ( anchors && pickInfo) {
                                    
                anchors.addAnchorPointUsingHitTestResultAsync(xrHitTestModule);
            }
        };
        
        scene.onPointerObservable.add( pointerInfo =>
        {
            BABYLON.Tools.Log("onPointerObservable");
        });
```

---

<div class="post-metadata">

**Author:** ![syntheticmagus](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/syntheticmagus/32/395_2.png) [@syntheticmagus](https://forum.babylonjs.com/u/syntheticmagus)\
**Post date:** [January 29, 2021, 5:42pm UTC](https://forum.babylonjs.com/t/babylonnative-and-onpointerdown-or-how-to-select-a-mesh/17983/2 "2021-01-29T17:42:09Z")

</div>

Hi Peppe.Belgenio,

Welcome to Babylon! If you’re using Babylon Native by itself (as opposed to Babylon React Native), input may not be hooked up to work the way you’re expecting yet. (This is a work-in-progress.) Are you using the Babylon Native Playground app? If so, experience.js provides [an example](https://github.com/BabylonJS/BabylonNative/blob/master/Apps/Playground/Scripts/experience.js#L34-L52) of how to manually work around rudimentary input handling short-term. Longer-term, more seamless and sophisticated input handling is coming; it just hasn’t been implemented quite yet. Will the `InputManager` approach demo’d in experience.js be enough to unblock you for your current scenario?

---

<div class="post-metadata">

**Author:** ![Peppe.Belgenio](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/peppe.belgenio/32/14042_2.png) [@Peppe.Belgenio](https://forum.babylonjs.com/u/Peppe.Belgenio)\
**Post date:** [February 1, 2021, 8:13am UTC](https://forum.babylonjs.com/t/babylonnative-and-onpointerdown-or-how-to-select-a-mesh/17983/3 "2021-02-01T08:13:55Z")

</div>

Thank you syntheticmagus for the quick answer.  
Yes, I tried to workaround using a custom implementation of the InputManager but there are  
still two big problems at the current stage:  
The click is only detected after moving a little the finger on the screen and furthermore I would also need the clicked object under the touch .

This is what I tried but the “click” is not detected before a small drag on the screen(I guess a couple of renders) and no object is picked up nor added with “ **addAnchorPointUsingHitTestResultAsync** ”:

```
function CreatePickHandling(scene,anchors,xrHitTestModule)
{
    var inputManager = new InputManager();
    scene.onBeforeRenderObservable.add(function() {
    
        if(anchors && xrHitTestModule && inputManager.isPointerDown)
        {
            anchors.addAnchorPointUsingHitTestResultAsync(xrHitTestModule);
        }
    });  
}

```

Anchors and xrmodule are defined as follow:

```
const xrHitTestModule = xr.baseExperience.featuresManager.enableFeature(
     BABYLON.WebXRFeatureName.HIT_TEST, "latest", {
         offsetRay: { origin: { x: 0, y: 0, z: 0} , direction: { x: 0, y: 0, z: -1}}, 
         entityTypes: ["mesh","plane"] });

...

const anchors = xr.baseExperience.featuresManager.enableFeature(BABYLON.WebXRFeatureName.ANCHOR_SYSTEM, "latest");

...

if (anchors) {
    console.log('anchors attached');
    anchors.onAnchorAddedObservable.add(anchor => {
                
         model.isVisible = true;
         anchor.attachedNode = model.clone("clone");
         model.isVisible = false;
   });

   anchors.onAnchorRemovedObservable.add(anchor => {
       console.log('disposing', anchor);
       if (anchor) {
            anchor.attachedNode.isVisible = false;
            anchor.attachedNode.dispose();
       }
    });
 }
```

---

<div class="post-metadata">

**Author:** ![syntheticmagus](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/syntheticmagus/32/395_2.png) [@syntheticmagus](https://forum.babylonjs.com/u/syntheticmagus)\
**Post date:** [February 1, 2021, 6:24pm UTC](https://forum.babylonjs.com/t/babylonnative-and-onpointerdown-or-how-to-select-a-mesh/17983/4 "2021-02-01T18:24:14Z")

</div>

I don’t have a complete answer for both issues, but at least the first one I should be able to help with. A delay between input and reaction is expected with `InputManager`'s current implementation because it’s quite naive (really only built for debugging, we didn’t expect it to get real use). In order for input to be processed by JavaScript, information about the input must move from where the program receives it on the main thread into the JavaScript thread. The `InputManager` does this by naively `Dispatch`ing a setter to the `JsRuntime`; this setter will apply the new value once it’s run, but because it goes to the back of the work queue (on which all the rest of the JavaScript work must also run), that’s usually not for a frame or two. This delay becomes more noticeable when framerate is low.

In theory, if you’re seeing multi-frame delays, this behavior could be changed by modifying the `InputManager`'s behavior to not use `Dispatch` anymore. Instead of `Dispatch`ing from the main thread, just set the `InputManager`'s state directly, using a mutex to make the operation thread-safe. I put up [a branch](https://github.com/syntheticmagus/BabylonNative/commit/0d6e7b9111e1e7364e4f8f0f7f6e7c56102bfa61) with a quick implementation of this which I tested locally (to commit this would require more changes, but this is sufficient to try). On Windows, I did not observe a meaningful difference between this approach and the prior one, presumably because my framerate was already quite good; but if you’re experiencing multi-frame input delays, you might try this approach out and see if it helps. If this improves things for you, I can likely turn this into a change that we could check in.

As for XR hit tests on iOS, that’s unfortunately not something I have any direct knowledge about. @ryantrem might know more. Ryan, does the code pasted above look like it should work with Babylon Native as it’s implemented today? Thanks!

---

<div class="post-metadata">

**Author:** ![ryantrem](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/ryantrem/32/34010_2.png) [@ryantrem](https://forum.babylonjs.com/u/ryantrem)\
**Post date:** [February 1, 2021, 10:34pm UTC](https://forum.babylonjs.com/t/babylonnative-and-onpointerdown-or-how-to-select-a-mesh/17983/5 "2021-02-01T22:34:14Z")

</div>

If you can’t use Babylon React Native, but you are on iOS, then you should use Babylon Native’s `NativeInput` class. This is not used in the Playground app right now because we currently only have support for iOS, Android, and UWP (not other platforms). The `InputManager` you see in Playground is just for quick testing - it is a completely separate “input manager” from the Babylon.js API. `NativeInput` on the other hand fully routes through the Babylon.js input system so everything should work as expected, including XR input and anchors. To use `NativeInput` in your own custom Babylon Native app, there are just a couple steps:

1. Initialize `NativeInput` when other Babylon Native components are initialized, and store the `NativeInput` instance. For an example, here is how we do it in Babylon React Native: [BabylonReactNative/BabylonNative.cpp at master · BabylonJS/BabylonReactNative (github.com)](https://github.com/BabylonJS/BabylonReactNative/blob/master/Modules/%40babylonjs/react-native/shared/BabylonNative.cpp#L50)
2. Handle raw input (e.g. touch input) from the `MTKView` you are creating in your app and forward it to the `NativeInput` instance. For an example, here is how we do it in Babylon React Native: [BabylonReactNative/EngineViewManager.mm at master · BabylonJS/BabylonReactNative (github.com)](https://github.com/BabylonJS/BabylonReactNative/blob/master/Modules/%40babylonjs/react-native/ios/EngineViewManager.mm#L37-L51). Note this goes through an intermediate layer (`BabylonNativeInterop`) which in your case you could skip (go directly to your `NativeInput` instance).

Again, if you use Babylon React Native, this is all set up for you automatically, but if you need to use Babylon Native directly then you’ll need to set something like this up. Later we will replace the `InputManager` test code with the actual `NativeInput` class, but probably not until `NativeInput` supports all the platforms generally supported by Babylon Native.