Babylon Unity Toolkit and React

I’ve recently been exploring a Babylon, React, Next.js stack as a new workflow, in addition to using the Babylon unity toolkit for its amazing unity usage for environments, light baking and scene placements. So far things have been good with all of the documentation and info here on the forum for guidance, I have been able to set up what looks like will be a nice stack and workflow. My only issue I really cant seem to overcome on my own is trying to implement exported content from the Babylon toolkit into the react/next.js app. My issue stems from 1) The scene being rendered properly with textures and lightmaps is tied to the scenemanager and many other scripts, which would not be an issue on a standard web app because you can just source it on the index. 2) I cant import the script into the react component unless its a node module because all babylon scripts need to be transpiled by node-transpiled-module npm pkg in order to work on next.js, and it can only look in the node modules directory. 3) I tried to create a local npm package out of the babylon.scenemanager script and the exported screen script, but its not clear what the dependencies are needed so the are a lot of NOT DEFINED errors. There’s 10,202 lines of code in just the scenemanager alone and its getting really hard to go error by error and cross reference what @babylon/core definitions are needed. 4) I cant just import babylon core as BABYLON on the scenemanger because it declares a var BABYLON on multiple lines so I keep getting this error:

It seems to me (in my very limited understanding) that Babylon toolkit exports are not compatible with component base apps unless the creator creates some sort of module package that can be installed like the rest of Babylon es6 packages, so my question(s) are; Is this the case? Has someone else come across this problem and created a workaround? Has anyone created a workflow for toolkit using react and would like to share?

I really thought the local npm pkg was going to be it but that proved difficult as well. The scenemanager and other dependencies make it really hard to manually implement and I’m not sure how to get a 1 to 1 render of the export, in next.js without it.

Please help…

I haven’t tried the Unity toolkit or seen what it generates - is that why you need a BABYLON defined? Assuming you don’t need what is generated for ssr - you could try a dynamic import to get around the transpiling:

import dynamic from 'next/dynamic'
...
const YourComponent = dynamic(() => import('../components/YourComponent'), {
  ssr: false
})

Sounds tricky to work with and you will still have the same issue if you are missing imports from ES6. There is a legacy ES6 export - it will be a big bundle, but maybe that’s a way to get started at least for testing and not so much for production?:

import * as BABYLON from '@babylonjs/core/Legacy/legacy';
1 Like

@brianzinn Thank you for your quick response!

The toolkit has two options for export output, 1) A web project 2) Just the babylon scene (in my case glb), the scene javasript file and a folder with textures.

In the case of (1), it comes with an html index along with source folders for scripts and scenes. The html index sources all scripts, runs a lot of supplemental code then loads the babylon scene on the page and this works well for standard website because you can just add your content to the boilerplate html .

in the case of (2) If you manually try to load the scene in either a standard web app or component you get multiple errors for different issues that basically boils down to the fact that the scene.js file requires the babylon.scenemanager.js file which is only exported in case (1). Here is a thread from an issue pertaining to just that:

@MackeyK24, can you provide some insight here?
I think ultimately I may try an alternative exporter (GitHub - Plattar/gltf-exporter: Unity3D GLTF2 importer and exporter toolchain) and compare outputs, I’m just not sure if it exports lightmaps as well as the Toolkit :slight_smile:

@brianzinn I think you are spot on with the next/dynamic solution for dealing with importing and after further reading, I believe that can work in delivering the content; But the ultimate goal with this project and using Next.js was to test different efficiencies of SSR with babylon/webgl content as there is very limited test/use cases online. From what I understand Next.js would render the page and initial canvas on server then serve to client, and I want to see the different results using this method. Either way thank you because solution or not, I will try this as a means of comparing performance between CSR and SSR.

1 Like

If you use GLTF what requires any extensions (it will give info warnings about CVTOOLS extensions) then you will need the babylon.manager.js loaded on the page. I do not support (or know how) to make npm packages for the toolkit or what i would have to do differently to support npm packages.

If you are using my Babylon Toolkit API … like script components and my universal materials that support things like HDR lightmaps and HDR reflection textures, HDR skyboxes and HDR environments (IBL) then you are going to need babylon.manager.js

Noy if you cant or wont use the scene manager, you can export just raw gltf models (No baked lightmaps etc…I make that stuff happen via the gltf extensions…so you can have ZERO touch baked lighting from Unity to Babylon… Gotta have an extension for that… and its in babylon.manager.js)

take a look at this thread to see the options

@MackeyK24 Thanks for the response. I have looked at that thread and referenced it above in my previous post. I am aware of the necessary scene manager and I mention in detail how I tried different methods of trying to get this to work. The toolkit is amazing and the feature I am most interested in using is the lightmaps, skybox, etc. The issue I am having is implementing it in react/next.js, due to the nature of the way node.js can export and import scripts. My suggestion of a npm package was just me throwing out what I think is a solution or work around but I don’t know if that is what it will take. Do you have any suggestions on implementing with react?

I actually don’t use BabylonJS libs in my react front end. I use a iframe in react and load the engine.html page in the frame and use quest string to pass the scene file I want to use

Basically the iframe is just a scene loader… I use window state or post message for any interaction with the top host page

The toolkit has a simple front end option that creates the index page and the actual engine page is loaded in the iframe

This approach has been working very well for some of my larger and commercial projects

1 Like

That’s a cool approach, I’ll look into using iframe. Are you (or @brianzinn) aware of any performance differences using this method in react versus loading the babylon scene directly as a component?

Loading an iframe is a good option for embedding content into your page, but I have only done so for third party content. My experience with react directly integrates the scene with React components with builds that bundle babylonjs, but that philosophy does not look to combine well with the generated files you are working with. If you just need to load iframes then hopefully that works for you. I will defer performance difference answers to @MackeyK24 that has proposed that solution.

1 Like

I have not any performance issues … As a matter a fact I get great performance and switching scenes really reload the engine page in the iframe … so we can store sockets in the parent page and stay connected to networks and chats in the front end app

Also each scene get a fresh DOM vs tearing down a scene and trying to load another in the same dom

1 Like

Hi MackeyK24,
I am interested in the iframe approach with react(typescript); would it be possible for you to share a “hello world” github project that uses iframe + react + typescript + babylonjs. I like this approach since I can keep my drawing code separate from my react application that is doing a bunch of webapi calls to get the data for the scene.
Any pointers would help; also basically I want to surround my iframe with react based “mesh selectors” which I will push into the iframe / babylonjs scene using messages – I felt your approach sounds a lot cleaner, not sure how to setup the entire workflow.

@dhiru I am assuming you are using the Babylon Unity Toolkit, since you posted here. Using iframes sounds like the easiest way to go as the pages are already generated for you. I don’t have a messaging solution to offer as proposed above for iframes, but perhaps postMessage is being used:
Window API: postMessage | Can I use… Support tables for HTML5, CSS3, etc

I don’t want to discourage away from using iframes or any other way of working and if that design is working for you then great. If you aren’t using the Babylon Unity Toolkit then I personally would not choose iframes and instead rely on the react router to control navigation and control tearing down and creating new scenes/engines as well as API communication to update scenes. A state management library or in-app messaging can be used where the DOM can communicate directly with your scene and vice-versa. HTH :slight_smile:

The toolkit itself already does an IFRAME implementation when using the Front End Index Page option. Does not use react in the main index.html (but you can use whatever you want in the front end index page… I use bootstrap by default)

This approach i use a front end index page to select the scene i want to load… and call the engine.html page in the iframe

As far CROSS WINDOW COMMUNICATION

Technincally, if you on the same domain… you can directly access the parent react window and dom using:

window.parent

and from react you can directly call:

frame.contentWindow

But i actually use the iframe content from different domains sometimes. So i setup a little bi directional POSTMESSAGE bridge to SAFELY communicate between the parent react page and the babylon framed content…

WORKS WELL :slight_smile:

1 Like

Many thanks, for the responses; I was planning on calling my engine.html in an iframe but was getting stuck when packagin / building the typescripts / javascripts included in the engine.html (created for babylonjs) – I am sure I am doing something silly – so went back-n-forth between doing babylonjs completely in react; vs the iframe way; since I aready have an existing react website.

I will experiment both ways and see if I can push my (“sub-par” / “silly” :slight_smile: ) attempt on github – perhaps it might help us see what I am doing or help someone else … in how not to do things :).

Thanks again

1 Like

@dhiru It sounds like you are building and calling iframe index component from within the same project, if that is the case, that will not work. As mentioned above, at any point you try to compile the unity toolkit exported content, it will fail.

What I ended up doing was creating code that extracted metadata from the exported scene and just recreated it in react. I don’t think that’s necessary for you because as mentioned by @MackeyK24 the toolkit provides you with a index.html file, all you need to do is host that file separate from your react project (maybe create a github page) and then ref that url in a iframe component in the react project.

hope that helps

Hey, maybe helpful idk …
.GitHub - jeremy-coleman/next-tsc-proposal: better way to use typescript / postcss / etc with nextjs

This approach you transpile everything into a new build folder , and treat that as the root of your next app. Basically eliminates the need for next/webpack specific tooling, including transpile modules. The gulp-preact branch also uses the iframe strategy similar to what mackey uses, but it is actually putting the next pages into an iframe instead of the other way around. This is like 2 years old though. I have conjured up some other trickery recently but nothing that i feel great about

1 Like