Inspector CLI (for AI agents)

Announcements
,
1

Hey everyone - I’d like to introduce you to a new experimental API and tool, the Inspector CLI. Conceptually, it mostly serves the same purpose as Inspector, but is designed more for AI agents than humans.

Inspector CLI is built on much of the same foundation as Inspector v2. It ships with a default set of commands (querying scene entities, reading properties on entities, getting scene stats, etc.), and is extensible so new commands can be added to expose app specific state.

A concrete example of this is the Babylon.js Playground, which now uses the Inspector CLI but also adds Playground specific commands for reading and writing code, running the playground, etc.

Since this is experimental, there are no docs yet, but we’d love to have the community try it out and give us feedback!

Playground

The easiest way to try it out is in Playground.

  1. Open Babylon.js Playground in a new browser tab (or don’t include the inspectable query parameter and instead click the CLI connection button in the bottom right of the Inspector UI).
  2. Open your favorite dev AI agent chat interface and enter a prompt like: Create a Babylon.js scene with a bouncing ball. Use "npx @babylonjs/inspector --help" to read, write, and test the code.

Your App

You can also use it in your own app as a diagnostic tool for AI agents.

  1. If you already use Inspector v2 in your app, then you can open it and click the Inspector CLI button in the bottom right corner. Alternatively, you can add a call to the new StartInspectable API (Inspector v2 UI is not required), and then just run your app.
  2. Open your favorite dev AI agent chat interface and enter a prompt like: Look at the source for my app in path/to/source and help me understand why the cube isn't visible. Use "npx @babylonjs/inspector --help" to inspect scene state.

Custom Commands

Inspector CLI uses the same extensibility model as Inspector v2. New commands are added by creating a service that adds commands to the IBridgeCommandRegistry. For example:

const CustomCommandsServiceDefinition: ServiceDefinition<[], [IBridgeCommandRegistry, ISceneContext]> = {
    friendlyName: "Some Custom Commands",
    consumes: [BridgeCommandRegistryIdentity, SceneContextIdentity],
    factory: (commandRegistry, sceneContext) => {
        const commandRegistration = commandRegistry.addCommand({
            id: "ray-cast",
            description: "Casts a ray and returns the mesh uniqueId for a hit, otherwise -1",
            args: [
                {
                    name: "origin",
                    description: "Comma separated x,y,z for the ray origin",
                },
                {
                    name: "direction",
                    description: "Comma separated x,y,z for the ray direction",
                },
            ],
            executeAsync: (args) => {
                // 1. Parse the origin and direction args
                // 2. Call sceneContext.current?.pickWithRay
                // 3. Return the picked mesh uniqueId or -1 if no mesh was picked
            },
        });
    },
};

StartInspectable(scene, {
    serviceDefinitions: [CustomCommandsServiceDefinition],
});

Please let us know:

  • Was it useful for you?
  • If so, how did you use it and what did you like?
  • If not, what would make it more useful?

We also plan to experiment with an MCP front end (alongside the actual CLI), but wanted to get what we have so far in front of the community asap to get some initial feedback.

9 Likes
Introducing Inspector v2
2

I believe it could be very useful but I don’t understand what I need to do.

The short video would be very helpful.

1 Like
3

I already have my mcp server running into Babylon. The best usage is about analytic yet…