WebXR Medical Simulation Proof of concept

Hey guys, I wanted to finally share a project I have been working on independently since the beginning of the year. It is my attempt at a Full Stack WebXR developer portfolio project piece, that’s one part React/Next.js Server Side Rendered webapp and second part babylon.js powered WebXR hospital simulation scene (nonVR desktop and mobile controls included as well). Details of the process below links

LINKS
Direct Link to Babylon Scene
Load speed ranges from 2 seconds to 1 min depending on internet speed

Link to Full Webapp Demo

For the Webapp:
I designed the dashboard wireframe and prototype in figma, developed all react components, and implemented API functionality to store and receive backend data, track course progress and social engagement.

For Babylonjs (The good stuff :slight_smile: ) :
I got the environment assets on the Unity asset store and sketchfab and the characters were created on Character Creator 3. I initially staged and designed the environment within Unity with the intention of using the “Babylon Toolkit” but in early testing I found it difficult trying to export the scene as a component into a react app, also found it hard to customize the scripts after the export, which is needed to track user progress to share with backend. I ended up using the native unity FBX exporter and exporting the scene into blender. Before moving to Blender, in a separate Unity project I did import the characters and added the desired mixamo animations, then used The Babylon Toolkit feature that exports selection only and exported each character separately, which allowed me to import into blender as GLB with animation stored. In Blender, I imported the env, then characters, baked lightmaps, decimated and optimized the meshes down to 20% of originally polycounts, merged and packed UVs, baked shaders into babylon friendly PBR shaders, merged meshes, optimize character blend shapes (which I found significantly increases gltf file size) and created texture atlases to reduce draw calls and improve framerates. After that the fun part, implement the actual code to load and render in react and create input options for demo purposes.
Simple, I know /s

End to end, It was QUITE THE PROCESS, and not to mention hours spent watching tutorials, reading the docs and checking the forum for the amazing community guidance and insight. It was worth every moment as I learned so much, but I know I only scratched the surface of the amazing things that could be done with babylonjs.

Upcoming Improvements:
Compress textures to decrease bundle size (KHT? )
Add faux contact shadows for characters (any advice on this?)
Create some sort of blinking function to eliminate the blank stares that throw off Immersion in VR

ALL feedback is welcomed and don’t hold back :smile:

12 Likes

Woot this is quite impressive!

cc @PirateJC

1 Like

Omg that’s amazing work! I love that you documented your workflow. Can I tweet about this project? @liquid_electron is my handle on that app

1 Like

Thank you, and I would be honored if you did!! Im not on twitter but could you maybe tag me via a link to my linkedin?

Btw, looking forward to the release of the book!

1 Like

GORGEOUS WORK! :heart_eyes: I love both the 3D scene and the webapp components, they look so well designed and pleasing to the eye. Love love how everything is pretty easy to understand and absorb and there is absolutely no clutter or anything out of place. You have an incredible eye for design!

1 Like

What’s your linked in url?

Thinking My website should suffice, https://www.iyanlaylor.dev/

Very humbled, Thanks again!!!

Thank you so much for the kind words. My background was 3d environment art before I decided to switch over to programming, so I try to pull from my artistic side as much as possible.

2 Likes

@Lay84 - SUPER cool! I really like it!

Hope it’s ok, we tweeted it from the Babylon account as well!

3 Likes

When I entered the hospital room, it really felt and looked like a hospital room! (I could even smell the antiseptic. :joy:) Everything seemed authentic. As part of my job, I take care of one of the owners of the business I work for, often accompanying her to a hospital and doctor’s offices–so I know what it’s like to be the guy sitting in the row of chairs…

2 Likes