Hello folks!
I’ve been having a real struggle trying to understand why rendering the scene to an image produces inconsistent results. Specifically when I call ScreenshotTools.CreateScreenshotAsync() sometimes (e.g. ~50%) the generated images don’t contain the whole scene due to what looks like a race condition, or drawing the image before the scene has fully rendered. This means that the generated image may be either blank, or have the meshes with their materials rendered but without their edges rendered (which should always be on) or be missing all UIObjects expected on the scene (length measurement labels).
The issue has never been reproduced on any linux or MacOS machine we’ve tried (some hundreds of times). I’ve only seen it on Windows machines even if they have latest updates and dedicated GPU (but not only). It seems particularly prominent on a client’s Surface Book for whatever reason.
The fact that there are no console errors produced and that this happens sometimes makes it really frustrating to understand and debug. I’ve found that doing this messy rendering after every scene change decreases the likelihood of the bug’s appearance (but doesn’t kill it)…
Below I’m pasting the screenshot.ts file
import { Engine } from '@babylonjs/core/Engines/engine'
import { ScreenshotTools } from '@babylonjs/core/Misc/screenshotTools'
import { CameraController } from './camera'
import { ALL_ELEMENT_TYPES } from './constants'
import { SceneProvider, createScene } from './scene'
import { Blueprint, ElementType, TactElement, isPristineTactElement } from './types'
import { UIProvider } from './ui'
import { UnreachableCaseError } from './utils/UnreachableCaseError'
const SCREENSHOT_IMG_WIDTH_IN_PX = 3000
const SCREENSHOT_IMG_HEIGHT_IN_PX = SCREENSHOT_IMG_WIDTH_IN_PX
/**
* Generates a png screenshot of the blueprint provided with the tactElements provided
* visible at full opacity, including lenght measurements for those elements. The
* first element's type is used as the active type and all other types are visible.
*
* @param blueprint blueprint to render
* @param tactElements elements to be shown as planned
* @param format preferred format of the returned image. Formats correspond to:
* - [`<canvas>.toBlob()`](https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toBlob)
* - [`<canvas>.toDataURL()`](https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toDataURL)
*
* @returns screenshot depending on the provided `format`:
* - a string of base64-encoded characters which can be assigned to the src
* attribute of an <img> element to display it
* - a binary blob
*/
export async function createScreenshot<F extends 'dataUrl' | 'blob'>({
blueprint,
tactElements,
format,
}: {
blueprint: Blueprint
tactElements: TactElement[]
format: F
}): Promise<F extends 'dataUrl' ? string : F extends 'blob' ? Blob : never>
export async function createScreenshot({
blueprint,
tactElements,
format,
}: {
blueprint: Blueprint
tactElements: TactElement[]
format: 'dataUrl' | 'blob'
}): Promise<string | Blob> {
const canvas = document.createElement('canvas')
canvas.width = SCREENSHOT_IMG_HEIGHT_IN_PX
canvas.height = SCREENSHOT_IMG_HEIGHT_IN_PX
canvas.style.visibility = 'hidden'
document.body.appendChild(canvas)
const engine = new Engine(canvas, false, {
useHighPrecisionFloats: false,
preserveDrawingBuffer: true,
stencil: true,
alpha: true,
desynchronized: true,
powerPreference: 'high-performance',
})
const scene = createScene(engine)
const cameraController = new CameraController(scene, false)
const sceneProvider = new SceneProvider(
scene,
new UIProvider(cameraController.orthoCamera),
cameraController,
)
sceneProvider.addBlueprint({ blueprint, renderEdges: 'sync' })
scene.render()
cameraController.resetZoom(sceneProvider.rootMesh, false, true)
scene.render()
sceneProvider.activeElementType = getActiveElementType(tactElements, sceneProvider)
sceneProvider.shownElementTypes = ALL_ELEMENT_TYPES
tactElements && sceneProvider.setTactElements(tactElements)
sceneProvider.setSelectedElements(
tactElements
.flatMap(tactElement => {
return isPristineTactElement(tactElement)
? tactElement.id
: tactElement.parts.map(part => part.color && part.guid)
})
.filter(Boolean) as string[], // we do `as sting[]` because TS fails to infer it
)
scene.render()
const screenshotAsDataUrl = await ScreenshotTools.CreateScreenshotAsync(
engine,
cameraController.orthoCamera,
{
width: SCREENSHOT_IMG_WIDTH_IN_PX,
height: SCREENSHOT_IMG_HEIGHT_IN_PX,
},
'image/png',
)
canvas.remove()
engine.dispose()
switch (format) {
case 'dataUrl':
return screenshotAsDataUrl
case 'blob': {
const screenshotWithoutDataUrlPrefix = screenshotAsDataUrl.replace(
'data:image/png;base64,',
'',
)
const screenshotAsBufferedBinaryInBase64 = Buffer.from(
screenshotWithoutDataUrlPrefix,
'base64',
)
const screenshotAsBlob = new Blob([screenshotAsBufferedBinaryInBase64])
return screenshotAsBlob
}
default:
throw new UnreachableCaseError(format)
}
}
function getActiveElementType(
tactElements: TactElement[],
sceneProvider: SceneProvider,
): ElementType {
const firstElement = tactElements[0]
if (firstElement) {
const sceneElement = sceneProvider.findElementWithGUID(
firstElement.id || (firstElement.parentId as string),
)
if (sceneElement) {
return sceneElement.type
}
}
return ElementType.Wall
}
Here’s a link with some generated images showcasing the different ways the generation fails and the successful expected result:
And an attached image of the expected result (as a new user I can only attach max one image here):
Please ask me to share additional files imported here which I obviously skipped to not spam since this is non-trivial industrial application using Babylon.js.
Thank you for this amazing library, your insightful blog posts, and for the (hopefully!) help in killing this nasty bug I’ve been fighting on and off for months!
I think this should be somewhere in the docs.