# Babylonjs native and asset loading **URL:** \ **Category:** Questions\ **Created:** [March 23, 2020, 5:31pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403 "2020-03-23T17:31:22Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![brianzinn](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/brianzinn/32/28013_2.png) [@brianzinn](https://forum.babylonjs.com/u/brianzinn)\ **Post date:** [March 23, 2020, 5:31pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/1 "2020-03-23T17:31:22Z")
I need to do a different kind of asset loading for a react native application. What I propose is adding extensibility to BabylonJS to allow developers to add their own or augment existing asset loaders. In current state, I would need to hoist the FileTools static convenience methods. What I would like to do is add extension (open closed) to allow developers to substitute. It seems at least for where DOM Image is used to load assets that Scene is available everywhere (except for 2D Image) and we could make scene an optional constructor param perhaps. So, my proposal is to add to SceneOptions an optional IAssetLoader to override asset loading. I haven’t fully investigated, but I think IAssetLoader could basically follow FileTools method signatures, unless you are ready to switch to Promises (all the native code uses async/await, so I need to wrap as sync). I’ve seen how videos and sounds are loaded and that will require some extra thinking. I did some tests on the weekend getting images loaded to textures from local app disk (ie: bundled assets/CameraRoll) using react-native utilities. I realize we have overrides to load “data:…” and I can work around and add custom code. I want to add this though not for a custom application, but for a library - otherwise I could just hard-code things and use existing static methods like “Texture.LoadFromDataString”, whereas I would like to use `new Texture(require('./localAsset/file.jpg'), ...)` etc. So, just want to start a discussion to see if this would be a welcome addition before working on a PR and also interested to learn more how BabylonJS Native project intends to allow Textures/Sounds/Videos to load from native filesystem/camera/etc. in case I have missed out the proper way to integrate native files with BabylonJS. ---
**Author:** ![Deltakosh](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/deltakosh/32/26635_2.png) [@Deltakosh](https://forum.babylonjs.com/u/Deltakosh)\ **Post date:** [March 23, 2020, 7:02pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/2 "2020-03-23T19:02:27Z")
Pinging @bghgary and @syntheticmagus ---
**Author:** ![bghgary](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/bghgary/32/48532_2.png) [@bghgary](https://forum.babylonjs.com/u/bghgary)\ **Post date:** [March 23, 2020, 7:13pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/3 "2020-03-23T19:13:34Z")
Our current plan for loading files is to override BABYLON.WebRequest. Accessing local files via `file:///` syntax and access app package files using a TBD url scheme (e.g. `app:///`). As for React Native, we are already actively working on this integration. Once all of this is done, all file access behaves like normal (i.e. FileTools will work as is without modification). The only difference is use `file:///` to access local files by absolute or relative path and `app:///` or something similar to access app package files. Does that help? ---
**Author:** ![brianzinn](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/brianzinn/32/28013_2.png) [@brianzinn](https://forum.babylonjs.com/u/brianzinn)\ **Post date:** [March 23, 2020, 7:32pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/4 "2020-03-23T19:32:54Z")
@bghgary that does help and glad to hear it is being actively worked on. The main site lists react-native as not supported or implemented. Are you using the GLView from `expo-gl` or something else for canvas? I have similar questions related to how to move pan and gesture events as BabylonJS input. If you can share a branch or anything would be appreciated. Thanks. ---
**Author:** ![bghgary](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/bghgary/32/48532_2.png) [@bghgary](https://forum.babylonjs.com/u/bghgary)\ **Post date:** [March 23, 2020, 8:51pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/5 "2020-03-23T20:51:02Z")
> [@brianzinn](#): > > Are you using the GLView from `expo-gl` or something else for canvas? No, Babylon Native doesn’t use WebGL. Everything is shimmed at the Babylon.Engine layer. You can see the code in the [nativeEngine.ts](https://github.com/BabylonJS/Babylon.js/blob/master/src/Engines/nativeEngine.ts). > [@brianzinn](#): > > how to move pan and gesture events as BabylonJS input Input abstraction is also being worked on. @PolygonalSun wrote an article on this recently: [Providing Input on… Input. When I’m looking at the Babylon.js… | by Babylon.js | Medium](https://medium.com/@babylonjs/providing-input-on-input-e908d53dd69e). Once this is done, you will be able to use Babylon.js input system as is and it should work in Babylon Native also. Note that Babylon Native will not have an HTML DOM and thus anything that interacts with the DOM will not work. ---
**Author:** ![brianzinn](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/brianzinn/32/28013_2.png) [@brianzinn](https://forum.babylonjs.com/u/brianzinn)\ **Post date:** [March 23, 2020, 10:21pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/6 "2020-03-23T22:21:36Z")
@bghgary It feels like we have gone down 2 different paths and I still need the customization from the original comment 🙂 Right now I am using the WebGLRenderingContext provided by GLView. From expo site: “GLView that acts as OpenGL ES render target and gives GL context object implementing WebGL 2.0 specification”. If I want to continue going that way then I would still need the extensibility from my original comment - although I think there are a lot of edge cases that I will have issues with. I am just beginning my investigation recently, but that component looks standard for even for non-Expo react-native projects. I read the Inputs article thanks for the link. I added the initial support for 2 of the VR devices and did some work on the GUI keyboard, so have already worked with the existing code somewhat. I am really interested to see where what you are building goes - the Profile part looks like what the spec for webxr does. I just want to send through events like the PEP polyfill - nothing with other devices currently and from that article was not possible to see if that could be done? In other words, I don’t need so much another level of abstraction as a way that I can send through pan/gesture events directly. If you look at the existing camera inputs they are “attached” instead of being sent pointer events. I feel like in both cases they are missing inversion of control to allow the extensibility that I am discussing. I want to intercept asset loading (similar to existing CorsBehavior) and flow through pointer events using my library of choice. ---
**Author:** ![syntheticmagus](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/syntheticmagus/32/395_2.png) [@syntheticmagus](https://forum.babylonjs.com/u/syntheticmagus)\ **Post date:** [March 23, 2020, 10:51pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/7 "2020-03-23T22:51:39Z")
Hi brianzinn, Regarding customizability, have you checked out the “plugin” architecture we introduced to the project a few weeks ago? With that architecture, virtually all functionality is split into modular plugin-style components that can be easily swapped out and customized independently. For example, Babylon Native’s provided [`Console`](https://github.com/BabylonJS/BabylonNative/tree/master/Plugins/Console) polyfill isn’t always necessary, and in some cases it’s harmful (React Native provides its own of these already), so it’s very easy to [include/exclude](https://github.com/BabylonJS/BabylonNative/blob/master/Apps/Playground/CMakeLists.txt#L122) or to swap out with something else. The kind of customization I think you’re talking about, which I believe boils down to polyfilling new capabilities in addition to the ones Babylon Native has already targeted, should be especially easy to add. All you should have to do is make a new folder following the pattern in the [Plugins](https://github.com/BabylonJS/BabylonNative/tree/master/Plugins) directory, make sure that subdirectory is added at the correct point during CMake configuration so that the names of its dependencies are defined (for something that pretty much only depends on `JsRuntime` and `napi`, you could just add it to the Plugins directory itself), then link to it and [initialize it](https://github.com/BabylonJS/BabylonNative/blob/master/Apps/Playground/Win32/App.cpp#L109-L110) within your consuming app to make the new types and behaviors available in JavaScript. Will that work for the customization you have in mind? (Note: The plugins change is still new, and not everything has been adapted to use the new paradigm correctly yet. 😄 Please excuse the plugins with weird initialization patterns [like `Console`] and things like that; those should be fixed soon.) As for whether or not your customization belongs in Babylon Native itself, I think that’ll depend on how broadly applicable it is. If you decide to go down the plugin route, then since plugins are quite modular and can be pretty easily transplanted within the file structure, I don’t think it matters much where you put it at first. You could just put it in the Plugins folder directly for simplicity’s sake, and then it could be pretty easily moved elsewhere later (even to a separate project) if that turns out to be the right thing to do. So, in sum, our new plugin-based architecture is specifically designed to make customization, especially of the sort that I think you’re talking about, as easy as possible. 😃 We haven’t had a chance to implement or write down all the patterns we’ve been thinking about on that front yet, but let me know if you’d like to discuss further! ---
**Author:** ![brianzinn](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/brianzinn/32/28013_2.png) [@brianzinn](https://forum.babylonjs.com/u/brianzinn)\ **Post date:** [March 23, 2020, 11:24pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/8 "2020-03-23T23:24:12Z")
@syntheticmagus the work you have done looks very promising. I want to extend BabylonJS to react-native cross platform with javascript only and not through the low level calls. It now makes sense how you are overriding the Request, while I am hoping to extend with purely javascript (and even non-native). I haven’t done any c++ for a while, so I think I’ll just sit back for a bit and see where this goes. I have written a lot of hardware device drivers, so can see what you are building, but I want to stick for this project with javascript, so I can easily integrate and swap out existing NPM libraries - like the react-native Pan/Gesture and asset libraries. ---
**Author:** ![JCPalmer](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/jcpalmer/32/42_2.png) [@JCPalmer](https://forum.babylonjs.com/u/JCPalmer)\ **Post date:** [March 24, 2020, 2:19pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/9 "2020-03-24T14:19:10Z")
Thought I would round this topic with my assumptions, questions. I had already assumed that a default path, or explicit file url would grab from somewhere in the app structure. Am also assuming that you could grab files whether they be assets, or say a 3kb JSON file with the directions to perform something using http or https. Anything but a `.js` file, right? If this is running on the Android platform, there are 3 types of storage App, Data, & Cache. Data can be deleted with the `CLEAR DATA` button. While this also deletes Cache, Cache also has a separate `CLEAR CACHE` button. I would hope that any export / textures / JS files which are bundled could be placed in the App area of the distribution, so that they could not be deleted. What url pattern would be used to get files stored in each of the 3 types? If the answer to my from http assumption is Yes, is there a way or could there be to put the stuff into storage? Actually, even stuff generated like preferences would be great to store. Is it possible to make a plug-in which puts it somewhere, preferably in Data? ---
**Author:** ![brianzinn](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/brianzinn/32/28013_2.png) [@brianzinn](https://forum.babylonjs.com/u/brianzinn)\ **Post date:** [March 24, 2020, 7:15pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/10 "2020-03-24T19:15:11Z")
@JCPalmer for react-native ‘expo-asset’ takes care of that. You can `require('./asset/texture/file.jpg')` files that are deployed with your app and access it directly, but for BabylonJS you currently need to take that asset (download and resolve it) and only then was I able to get it to load with Texture.LoadFromDataString(). It is that part of the process that I want to hijack. I did not do a lot of testing, but did run into some webgl errors about origin when applying textures. On the other hand any external files like `new Texture('https://playground..../', ...)` load automatically. BabylonJS creates a DOM image attaches a src and corsBehavior and then when loaded attaches to the texture. That does work right now in Expo with no changes. From what I understand the BabylonNative project does not have DOM access and will override WebRequest behavior instead and use their own scheme convention. The device OS itself does the caching, so what happens is the asset itself knows if it is downloaded already and will check the device cache (to avoid multiple downloads and see if cache is cleared), but the babylonjs engine also maintains a texture cache. I am still in learning mode, so not 100% positive about all those comments. ---
**Author:** ![MarianG](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/mariang/32/138_2.png) [@MarianG](https://forum.babylonjs.com/u/MarianG)\ **Post date:** [March 25, 2020, 3:36pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/11 "2020-03-25T15:36:40Z")
Hey @brianzinn, you can take a look at what I did here, few months ago. > **[Bulisor/BabylonjsDemo](https://github.com/Bulisor/BabylonjsDemo)** > > Babylonjs & ReactNative(Expo) . Contribute to Bulisor/BabylonjsDemo development by creating an account on GitHub. Unfotunatelly I lost the main project, but atleast I learned a lot about RN and babylonjs too This is still on my todo list, and I’m planning a 2.0 version on which I’m already working on my sparetime. It has import for textures and gltf files and interaction for rotation and zoom. Maybe it is not the best and the clearest code,… I’m sorry for this 😉 Cheers! ---
**Author:** ![brianzinn](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/brianzinn/32/28013_2.png) [@brianzinn](https://forum.babylonjs.com/u/brianzinn)\ **Post date:** [March 25, 2020, 8:04pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/12 "2020-03-25T20:04:00Z")
That’s great to see others interested @MarianG !! I see you ended up basically with the same issues as me. You needed to resort to static methods for loading app textures and had to hand-code Device inputs over to specific camera properties. What I would prefer is an inversion of control where we can say that a pointer event occurred and publish an event Without coupling. For loading assets to have ability to handle asset (URL) loading requests ourselves instead of static hard coded in framework Tools. Those two changes would allow me to integrate native easier. I’m not writing the code for my own app, but instead trying to make it easier for developers to integrate BabylonJS and also declaratively when possible - otherwise I could follow some of your code patterns. I am interested to work with you on your v2 and maybe we can share progress and tricks. I have an iPhone and Pixel for testing. I added custom props handling yesterday, so maybe can get local assets loading, but it’s a hack. Would like to work towards a clean native integration 🙂 I need to do more testing!! ---
**Author:** ![MarianG](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/mariang/32/138_2.png) [@MarianG](https://forum.babylonjs.com/u/MarianG)\ **Post date:** [March 27, 2020, 7:22am UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/13 "2020-03-27T07:22:30Z")
Great, I’ll come back with updates soon as well ---
**Author:** ![JRuiseco](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/jruiseco/32/8318_2.png) [@JRuiseco](https://forum.babylonjs.com/u/JRuiseco)\ **Post date:** [April 1, 2020, 7:52pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/14 "2020-04-01T19:52:49Z")
One of the best parts of a system like Babylon is the ability to write UI with modern webdev frameworks. Which are arguably the most efficient ways to write extensive UI. I really think there is a lot of value in either embedding a virtual dom based ui system compatible with modern MVC frameworks, or having a Babylon plugin for react native. ---
**Author:** ![waverider404](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/waverider404/32/17464_2.png) [@waverider404](https://forum.babylonjs.com/u/waverider404)\ **Post date:** [April 3, 2020, 2:48am UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/15 "2020-04-03T02:48:36Z")
I work on mac so can i use G++ to compile instead of Xcode (Not a fan of xcode, it crashes too often) ---
**Author:** ![bghgary](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/bghgary/32/48532_2.png) [@bghgary](https://forum.babylonjs.com/u/bghgary)\ **Post date:** [April 3, 2020, 7:17pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/16 "2020-04-03T19:17:42Z")
@waverider404 I’m not totally sure you can use G++ to compile. Perhaps @Cedric will know. I’m not a big fan of Xcode either, but it doesn’t crash for me. ---
**Author:** ![Cedric](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/cedric/32/2742_2.png) [@Cedric](https://forum.babylonjs.com/u/Cedric)\ **Post date:** [April 4, 2020, 11:42am UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/17 "2020-04-04T11:42:25Z")
@waverider404 You should be able to compile using g++ and make: ```auto cmake .. make ``` The issue will be with storyboard and resources. For iOS, we use swift. I don’t know if gcc compiles swift. ---
**Author:** ![waverider404](https://sea2.discourse-cdn.com/flex024/user_avatar/forum.babylonjs.com/waverider404/32/17464_2.png) [@waverider404](https://forum.babylonjs.com/u/waverider404)\ **Post date:** [April 4, 2020, 4:14pm UTC](https://forum.babylonjs.com/t/babylonjs-native-and-asset-loading/9403/18 "2020-04-04T16:14:11Z")
Just researched online and results say it’s possible > Please note that you also need to install Clang if you want to create native executables with Swift. Clang is not needed to run the swift command interactively or to run a .swift file. Interestingly, installing GCC (including g++) and creating symlink clang++ to g++ does allow swiftrc to build an executable" >