Interop with react / angular / vue

I’m building different 3d visualization products and have recently fallen in love with babylon.

I’ve tried different setups of how to communicate between BJS and react / angular, but I have not yet found a way which feels good.

So my question to you guys and girls is: If you are building apps which are a hybrid between a framework and babylon, how do you do the communicating between them?

Examples:
From react tell BJS to animate camera to position, rotation?
From react loading items from an api, to BJS spawning the items in the scene?
From BJS when clicking a mesh how to trigger a redux action or rxjs Subject emit?

I’d love to hear any suggestions or if you have any demos :slight_smile:

I’ve been doing this in two ways.

  1. creating an api-ish object (either on the bjs side or the react side) and sticking it to the window. This object has all of the functions for triggering something to happen in bjs, or for applying store mutations
  2. full decouple with events (also through the window)

Despite having done both, I have no real preference. Both let the functional code stay nice and functional… both also introduce a global variable (or window events, which are similar in my book) and a concern about the order in which things load (the ui first? or the game?).

here’s some pseudocode for the window event version

// in ui layer (some button onclick =>)
window.dispatchEvent(new CustomEvent('animateCamera', { detail: { position, rotation })

// in bjs code
window.addEventListener('animateCamera', event => { /* do bjs stuff with e.detail.position , etc */)

And here might be going the other way into some redux/vuex type of framework

// on mesh clicked...
window.dispatchEvent(new CustomEvent('meshClickInfo', { detail: { stuff }))

window.addEventListener('meshClickInfo', event => { 
     store.commit('meshName', event.detail.something)
     // presumably this state change causes something to change in the ui
})

It may also worth mentioning that webpack has some fancy export options, such as exporting a single object or function. I had one game where the whole game was bundled together and stuck into window.startGameAndConnectToServer(url) which was a function that the react ui would call when the player chose a server from the server browser (all react for that part).

been doing this at work with a custom wrapper for what is basically a canvas component and a code api for accessing both engine and functional code babylon side. I’m using pure typescript react, so it might be a bit overkill for what you’re looking for, but it helps separate both environments (that is UI and engine code) without having to send browser messages all over the place. I’m also using react-responsive-canvas to deal with resizing, a classic react ref to get the component when my wrapper renders to trigger engine init on its HTMLCanvasElement and all of my functional code behind. The wrapper just instantiates non react classes like, say, Renderer and Scene custom classes with logic. These classes expose public methods like moveCamera, selectMesh, etc… and are called directly on the instances on react events : state changing, routine rerenders, and, depending on the situation, also messages. The point is to keep it simple with our architecture, but it implies your babylon code is a bit structured. Our internal reason is that the same engine is used as a plain website for dev purposes, a react website for other stuff, and a weird webview setup in native code. This allows us to have one functional codebase for all these entry points. Dunno if this can point you in the right direction, but it keeps us from having headaches due to radically different environments :smile:

1 Like

Thanks for the answers.
It sounds like I’m on the right path.

As you say, it’s probably best to keep the BJS stuff totally separate from the react stuff.

I’m going to try to wrap a babylon canvas as a web component that exposes the different attributes and events.

That way it will be easy for any on of our web developers to interact with it either from react or angular or vue :slight_smile:

2 Likes

I’ve recently been experimenting more heavily w/ redux+react, clojurescript + reframe, and vuex+vuejs. I’ve been trying to mix it with games at layer deeper than the ui. For people who like the SSOT/store/immutability I think an interesting place to mix in a game is right at the store. I’m only at the start of this project but here’s some of the prototype code…

The trick was how to make the store pattern relevant to a game – the unidirectional flow of data in those frameworks typically results in some sort of reactive dom rendering (but doesn’t have to).

The stores in all of these things are basically subscribe(mutation, state) and commit('someMutation', someValue) which is already about as safe and decoupled as the event stuff I had suggested above. I’ve tried directly tapping into this in reframe and vuex outside of the usual use case.

Vuex - using the store and changing something webgl-ish in babylon:

const state = {
    forwards: 87,
    backwards: 83,   
    fov: 0.8,
    etc: '...',
}
const getters = { ... }

const actions = {
    openKeybindModal(...),
    handleKeybindInput(...),
    changeFov({ commit }, newFov) {
        commit('updateFov', newFov)
    }
}

const mutations = {
    ...
   updateFov(state, payload) {
      state.fov = payload
   }
}

I’ll skip the html parts but there is a ui where the fov can be changed, and saved to local storage and reloaded etc etc – and all of that works the typical web programming way. And now here’s how it plugs in to the game:

// game plugin
    store.subscribe((mutation, state) => {
          console.log(mutation) // { type: "settings/misc", payload: { fov: 0.8 }
          if (mutation.type === 'settings/misc') {
              if(mutation.payload.fov) {
                  camera.fov = mutation.payload.fov
              }
          }
    })

So there we have a store mutation where the reactivity A) updates the settings menu + localstorage, and B) changes the camera fov which is essentially reactivity all the way over in webgl land. The parsing of mutations is a bit tedious (usually automagic in those frameworks).

And here it is going in the opposite direction (game => store) with the added complexity of the game getting its own state from a server (multiplayer, websocket):

networkStuff.messages.forEach(message => {
    if (message.protocol.name === 'Scores') {
         // format - message.scores: [ { playerId: 123, kills: 5, deaths: 2 }, etc ]
         store.commit('game-ui/leaderboard',  { scores: message.scores })
         // -> causes the k/d and score table to change, which is html for me
         // but could also have been babylon-gui via the above pattern
    }
})

I’m refactoring a game to get a good portion of its state into a store to see how it goes. Hopefully the experiment works. There’s a lot of game-related state that I would never try to treat immutably… but at least so far a lot of the annoying code (keybinds, in-game ui, player equipment loadout, error modals, chat ui) is coming out better in the flux pattern.

2 Likes

That will deserve a blog :slight_smile: We accept community author on BJS blog by the way :slight_smile:

1 Like

Great Question!

I’m a bit late to see this. I came from a world of events via ESB, BackboneJS events bus, event stores, etc. It’s a different paradigm with redux reducers - more like CQRS. You have events going in and a state coming out. The state changes coming out are not visible in BabylonJS (React gets those changes via ReactDOM.render(…)). I have built a React Renderer on top of the ‘react-reconciler’ that is able to flow prop changes directly to the scene and BayblonJS components. You could use custom hosts to wire directly into that mechanism provided by the React framework, but let’s try to go a simpler way.

First I would recommend using redux-saga to easily do API calls and keep your reducers pure and you can easily flow from there the data into your BabylonsJS scene.

What I did was register in my BabylonJS scene as a listener to all of the redux actions (so like another listener on a message bus) to intercept the actions I was interested in. The code is very simple, I have just added my own middleware and I can tap into listening for redux actions whenever I create a scene. I have my React App and BabylonJS scene synchronized and you can, for example, click a button on canvas or DOM to trigger actions in one or both worlds. Here is how the middleware works (GitHub - brianzinn/redux-wiretap: add/remove listeners to monitor redux actions) - it has an NPM, but you can just add a file to your project.

Triggering a redux action from BabylonJS, you can just pass connected action creators (that dispatch automatically) as props like regular react to trigger actions.

It’s not super clean because React will listen to props and update, while in your scene, you will need to make sure you are listening to the right actions and updating accordingly. The nice thing about a saga is that you can add listeners there that also create events and have a cleaner separation with more domain specific events perhaps. Or publish ‘past tense’ events separately after updating store - that’s how I went for many events.

I have been experimenting with a few different ways - the trick is synchronizing and trying not to duplicate your efforts. RxJS and vue reactivity go towards getting changes automatically. Interested in any updates you have. Cheers.

2 Likes

There is a little on React with Babylon.js and in the docs How to use Babylon.js with ReactJS - Babylon.js Documentation
Anybody who wants to expand on it is welcome

Hello, I have built a project accroding this doc, but how to inject the earcut into my project, I used [React-script-tag] but it does’t work!

You will be better to ask this as a new topic.

Hey Tuxic, and thanks for this information. Do you know if there are type definitions for this module?

I use Babylon with Svelte.

There’s a few ways to communicate between the two but I personally just use Svelte stores. Any code, like your app’s main Babylon scene, can subscribe to these global state stores and when they change, react to those changes. It’s dead simple really.

I’ve only done a small amount of stuff with React and Vue, but in comparison, Svelte is less code, less boilerplate bloat, no framework (compiles to plain JS), faster performance (no shadow DOM) and has a lot built-in like state management and animation for instance, that you’d normally have to reach for a third-party library to do in React or Vue.