Babylon Viewer (v2)

[Dad72]

It says in the doc that you can add custom functionality. For example if I want to add a button that takes a screenshot of the canvas and generates the image file, is it possible? is there an example for this?

[ryantrem]

In general, “custom functionality” is what we consider an “advanced scenario” and there is some documentation and an example (more coming) in the docs here: Viewer Advanced Usage | Babylon.js Documentation

Specifically about taking a screenshot, you can definitely do that, but only when using the npm packages and building your own bundle. There are two reasons for this:

1. The current Viewer ESM bundle includes the pieces of @babylonjs/core and @babylonjs/loaders that are used by the Viewer itself. This is because these two packages are effectively not compatible with direct ESM usage currently.
2. The current Viewer ESM bundle does not re-export the underlying BJS libs. This will help with compatibility when core/loaders are eventually fully ESM compatible.

That said, assuming you install the @babylonjs/viewer@preview package in your own project along with @babylonjs/core, then you can add your own UI on top of (or within) the Babylon Viewer, access the viewerDetails to get access to the engine, scene, and camera, and then use Tools.CreateScreenshot.

Here is a quick example using CodeSandbox (which allows package installation and custom bundling, unlike CodePen): https://codesandbox.io/p/sandbox/optional-chaining-forked-6g7mtj?workspaceId=ws_Cy8iztiBhRZG7hdumcf2vw

[labris]

@ryantrem

have the new Viewer package fully replace the old Viewer package

I hope it will not break all sites which use old Viewer like <script src="https://cdn.babylonjs.com/viewer/babylon.viewer.js"></script>

[ryantrem]

The Babylon CDN is generally not meant for production use, but regardless we won’t break this. The existing Viewer url on the Babylon CDN will still resolve to the legacy Viewer.

[ryantrem]

@ryantrem

@labris

Also, I wonder what is the best way to have the environment without skybox and with transparent clearColor

It’s not possible with the Viewer API right now, but I think it should be. I will add this to the feature list.

This is now possible with the latest release. On the custom element, you can still set the environment attribute if you want to use the same texture for both IBL and the skybox, but you can also separately set environment-lighting and environment-skybox. Effectively environment is a shorthand for setting both to the same value.

[ryantrem]

@ryantrem

Pending any other community feedback, we plan to flip the switch to remove the -alpha suffix, the @preview tag, and have the new Viewer package fully replace the old Viewer package on npmjs.org right after the new year.

Heads up, as of 7.43.0, the new Viewer package has replaced the legacy one, so npm install @babylonjs/viewer will now get the new Viewer. The legacy viewer is still available via npm using version numbers prior to 7.43.0 (both ES6 and UMD), and the legacy viewer is still available through the same Babylon CDN. Right now, the plan is to not publish any more updates to the legacy viewer (NPM or CDN).

[labris]

What are the plans for Hotspot/UI Configurator?

[ryantrem]

There is another effort I have under way (it’s been lower priority for a while) to introduce a new “extensible sandbox.” There is a brief blurb about it at the bottom of this article: Formally Adopted

The direction I’m going right now is to make the Viewer configurator an extension (plugin) for this new extensible sandbox. I’d love to squeeze this in for the Babylon 8.0 release, but it just depends on other 8.0 priorities.

Here is a sneak peak of what the work in progress looks like:

For hotspots specifically, the workflow is:

1. Add a new hotspot
2. Select a target point on the model surface
3. Position the camera around that target point and snapshot the camera state

[roland]

Guys, could we boost the Babylon viewer’s (including the assets viewed) load speed by leveraging the Speculation Rules API on the page?

@labris maybe something you would like to try

[labris]

@roland Do you have any good examples of Speculation Rules API with Babylon?
I also thing they should or may be implemented at a page level or only at some viewer’s use cases…

[roland]

@labris

Do you have any good examples of Speculation Rules API with Babylon?

Hello! I don’t have any. That’s why I tried to pass the research to you It’s on my list to prerender multiple scenes (each on separate page) in our project and fast switch them using the Speculation API (maybe in a month) but for now I use multiple webviews in the parent native Flutter app. It’s far from perfect - you have to deal with memory management yourself. The Speculation API is supposed to hide the implementation details of prerendering a html page.

[ryantrem]

The Viewer now supports custom environment rotation and intensity via environment-rotation and environment-intensity. Thanks for this contribution @alexandremottet!

[ryantrem]

As of 7.47.0, by default the Viewer now automatically suspends the render loop when nothing would otherwise visually change frame to frame. This is intended to minimize resource pressure when many Viewers are used on a page, as well as reduce power usage on battery constrained devices (e.g. mobile).

It’s tricky to get the logic correct around this, so if you see any issues where something doesn’t seem to render correctly, but then does render correctly when you do something like move the camera, then it’s a bug and please let us know!

This feature can also be disabled by adding the render-when-idle attribute: Viewer HTML Element Interface | Babylon.js Documentation

[brianzinn]

@ryantrem

As of 7.47.0, by default the Viewer now automatically suspends the render loop when nothing would otherwise visually change frame to frame.

Wow - this is a really great insight for the community. The various checkpoints where you flag the scene as having mutated. Just to be clear, something like mesh.position.y += 3 - that will not mark the scene as mutated, right? I do see that you capture regular animations and animation groups and even skybox/environment changes. I wonder if something that useful belongs upstream - like you could register with flags on an observable. I’ve previously added observables to obvious existing ones like onViewMatrixChangedObservable. Anyway, really great to see something more thought out.

I ended up adding an “isPaused” to my project ( Pause Renderer - React Babylonjs), because I just wasn’t able to easily hook into all the changes you mentioned, but wanted to provide that capability to consumers. I did add an observable in GUI add ValueAndUnit change tracking · BabylonJS/Babylon.js@516c900 and my thought was to make a sceneChanged observable tying into those and then you could use an observer mask to sign up for certain events (like those) that you care about by using Babylon.js/packages/dev/core/src/Misc/bitArray.ts at master · BabylonJS/Babylon.js. I don’t think your viewer is concerned about GUI events, though, but something supported in the framework could be great.

so, it could end up being something like:

// this will map to flag via BitArray of an observable maybe - each option is a mask
scene.onMutationObserved.add({onAnimationChanged: true, onViewMatrixChanged: true, onMeshPositionChanged: true})

Not sure how heavyweight that is, but useful as an opt-in. That’s what I was looking to implement. On the other hand, this can easily be added to any existing scene. Like an observable for when meshes are added, etc.

Anyway, I’m really excited to see that even just in the viewer - I’ll be keeping any eye on that and I think brining in (copying) some of that in to at least the react-babylonjs examples. Hope you don’t mind! Maybe something re-usable…

Thanks for the update. This is something really useful for low end devices and pages showing 3D scenes trying to lower their footprint.

[Deltakosh]

@brianzinn

mesh.position.y

Well it could as the vector3.y is a property that will flag the position as dirty

[j-te]

I don’t want my Viewer to render the ‘enable audio button’ as it interferes with the UX/UI. For now, I’m happy with removing the audio altogether for my unique project. Most of my previous projects haven’t actually utilised Audio.

I have found a solution during initialization but it’s a bit dirty:

const engine = viewerResult.scene.getEngine();
const engineConstructor = Object.getPrototypeOf(engine)?.constructor;
engineConstructor.audioEngine.useCustomUnlockedButton = true;

Posting here for others/discuss as a feature.

[Anupam_Das]

@ryantrem really appreciate all the work you’ve been doing on the viewer. We’ve been lurking in the dark but frequent your blog posts and now about to jump head first into babylon-viewer this month. We’re really looking forward to working with this as our customers were looking for a light-weight solution to use across multiple endpoints across their business. Very interested in evaluating this as a potential solution.

[ryantrem]

@brianzinn

Just to be clear, something like mesh.position.y += 3 - that will not mark the scene as mutated, right? I do see that you capture regular animations and animation groups and even skybox/environment changes. I wonder if something that useful belongs upstream - like you could register with flags on an observable.

Correct, with the Viewer’s current implementation, it will not automatically see something like this as a mutation. Current thinking is that for “advanced” Viewer scenarios like this (where you are writing code to manipulate the scene yourself), you can either set the render-when-idle attribute to disable this feature, or you can use the viewerDetails to get access to a markSceneMutated function. I talked with other people on the team about trying to do this kind of thing at a lower level, but it currently seems impractical to do so in a way that is performant and useful in the general case. It works for the Viewer because by default, things are very constrained and we know the relatively small set of things that can case the scene to be mutated and require a re-render.

@brianzinn

Anyway, I’m really excited to see that even just in the viewer - I’ll be keeping any eye on that and I think brining in (copying) some of that in to at least the react-babylonjs examples. Hope you don’t mind! Maybe something re-usable…

Thanks for the feedback! Feel free to re-use any code from the Viewer that might be useful to you. Also for your awareness, I’ve recently done a bit of work in the Viewer to enable inheritance as a route for customization. If you wanted to, it would definitely be possible for react-babylonjs to build on top of Viewer or ViewerElement to get some of this functionality largely for free. If you are interested, I’d be happy to chat with you more about it.

@j-te

I don’t want my Viewer to render the ‘enable audio button’ as it interferes with the UX/UI

I believe the only time audio comes into play with the Viewer (unless you are doing something custom) is when you load a glTF that uses the MSFT_audio_emitter extension, which I assumed would be very rare. What is your scenario where audio gets loaded? We could definitely add an attribute to disable audio (or have audio disabled by default and require an attribute to be set to enable it) if this would be useful to the Babylon community.

@Anupam_Das

We’ve been lurking in the dark but frequent your blog posts and now about to jump head first into babylon-viewer this month. We’re really looking forward to working with this as our customers were looking for a light-weight solution to use across multiple endpoints across their business. Very interested in evaluating this as a potential solution.

Great, thanks for the heads up! As you start to evaluate it, please share your feedback on this thread, or DM me on the forum. If there are any shortcomings for your scenario, we’re definitely interested in learning more!

[j-te]

@ryantrem

What is your scenario where audio gets loaded?

I used the UFO model so definitely embedded sound. So I load GLB assets dynamically inside a custom Asset Library, and I don’t want any embedded sounds to be read by the Viewer.

An optional property inside CanvasViewerOptions like disableAudio would suffice.
Babylon.js docs

The above core API audioEngine?: boolean should maybe change the default value: Engine.useCustomUnlockedButton = !audioEngine (but currently doesn’t).

So it would be nice if the Viewer could incorporate the above as a start.

Edit: maybe exposing audioEngine along with its type would be the best bet.

[brianzinn]

@ryantrem

If you wanted to, it would definitely be possible for react-babylonjs to build on top of Viewer or ViewerElement to get some of this functionality largely for free. If you are interested, I’d be happy to chat with you more about it.

hi, sorry I wasn’t trying to hijack your thread. I just think that detecting scene changes is a useful feature for downstream products that use babylon.js. It’s ok if the team feels it’s impractical. It definitely would hit performance and be opt-in. As you mentioned, the viewer is very constrained by default and I think the viewer and react-babylonjs serve different audiences. As with most things in computer science - the easiest one to get going with tends to be the best choice until you want more fine-grained control or to get closer to metal and then there is a wall. I don’t know how the viewer could be built on top of, since it uses it’s own scene and engine (I saw that the engine option is just a string). Maybe I don’t understand the ViewerElement - I think it wants to control the scene and camera creation. I do think a lot of the viewer functionality would be very useful to get as an extension. How is the viewer designed to be built on top of?