Low-Level Havok Debug Lines Visualization

By “for free”, I mean that someone already developed the C++ debug visualization in Bullet, PhysX, and Havok :slight_smile: In the video above, I did not derive any of the debug lines. Instead, PhysX (and also Bullet) gave me all the lines (and colors) for free, and I just rendered them.

Although Havok must have its own built-in debug lines visualizer, it’s currently not exposed in the WASM for us external users to use

Does Havok on any other framework provide debug lines?

Havok’s official website mentions Havok Visual Debugger (HVD). My wish is for Babylon and Havok to expose these visual debugger APIs (specifically lines) in their WASM for us to use :slight_smile:

Looks like Unity uses HVD

And Havok provide HVD for Unreal Engine

Havok 2013 source displaySegment() function, which may be a parallel of Bullet’s and PhysX’s debug visualization lines

1 Like

I guess that’s where I’m stuck. Does Havok actually draw anything? I would have thought it only modifies transforms on babylon objects. If it provides some ArrayBuffer of line/color data, it’d be most beneficial in a form easily usable by BABYLON as raw VertexData or Matrices array. If so, that’d be great. Maybe that’s what you’ve been asking for all along. Until we get better support along those “lines,” maybe my code here is useful.

That’s exactly it :slight_smile: Bullet and PhysX both offer ArrayBuffers of line/color data that we can render with Babylon. I’m confident that Havok also has an array of this line/color data, but its getter function is not exposed in the WASM and TypeScript types

True, so thank you for all of the code you’ve shared!

Toward the goal of Havok providing low level data on lines ancolors, it would be great to know how Havok provides that data over its debug interface. Until we have that, it might be useful to speculate.

On the other “side” of the interface, what BABYLON features can we take advantage of for visualization? I suggest the best candidate is a LinesSystem. In particular, the VertexData for a LinesSystem is likely the lowest level BABYLON data structure that would be useful. VertexData could be seamlessly visualized with vertexdata.applyToMesh(). One level above that would be Vector3 data as parameters to CreateLineSystemVertexData() which is of the form

{ colors?: Nullable<Color4[][]>;
   lines: Vector3[][];
}

Knowing that low-level havok uses heaps and consecutive floating point values, conversion to the above lines/colors structure might involve copying data around. It also might be inefficient for Havok to repeat the color information implied by the above structure, which is an indicator (at least to me) on whether Havok might supply the color information that way.

The most efficient way that I can think of is Havok to supply a Float32Array for lines to be interpreted as Float32Array[3][2]. And a color array as either Float32Array[3][1] or (slightly less space efficient) Float32Array[3][2], the latter at least matching the size of the line array, and matching color index to line vertex index wherein each color index is repeated for each of two line vetices.

It’s unclear to me now what the output of CreateLineSystemVertexData() exactly is. Given that mesh vertex data is generally in triangles, I’m not sure what VertexData from lines actually looks like. Is it possible Havok might produce that directly? Maybe, but I (without evidence) doubt it.

I’ll dig more into that when I have time.

For contact points implemented with thinInstance, the most useful low level data structure is a Float32Array containing 4x4 matrices, one for each translation point. One step above that (and much more likely, IMO) is as a Float32Array interpreted as Float32Array[3] or, above that, Vector3.

Edit: good news! CreateLineSystemVertexData() generates a linear array of numbers, sort of ignoring the “array of arrays” in the specified lines parameter. Additionally, VertexData allows number[ ] or Float32Array. My primary concern of somehow a mesh being triangles is now not a concern. I have confidence that if Havok produced a Float32Array of line segments each with two points, it can be used directly in VertexData and subsequently .applyToMesh.

CreateLineSystemVertexData() additionally creates a consecutive list of indices, which I think could be ignored (or easily created). The last bit is a list of colors in a Color4 array of similar construction, thus requiring a pair of Color4 objects per line segment. If Havok creates this natively, great! If not, creating it from a list of single colors or Color3 objects per line segment would be straightforward at some performance cost.

1 Like

Yup :slightly_smiling_face: I’ve been using BABYLON.MeshBuilder.CreateLineSystem() for Bullet and PhysX lines.

I wouldn’t worry about triangles. If Havok is like Bullet and PhysX, they both call the lines function (3 times - once per triangle line) inside of the triangle function. So the mesh triangles are already included in the debug lines, thus we don’t even need to deal with triangles - only lines.


I will create a minimal Playground showing Ammo’s Debug Drawer, which should make it clearer how powerful and simple to use these lines are. Please give me a few days for this

In the meantime, there’s @arcman7 's PG that uses the Ammo Debug Drawer. Please note in this PG, the lines are rendered as points (instead of lines)

Maybe you could use GreasedLine. There is a GreasedLineTools class also available with helper functions.

1 Like

I’ll give GreasedLine a shot, too. A drawback of the LineSystem is the lack of line width. Recreating a LineSystem with a hundred or so lines still performs well. Will be interesting comparing performance to GreaseLines. I wonder if thinInstances could help, but the lines would have to be separated by color for thinInstances. One advantage of thinInstances is the minimal javascript objects needed to support them. I think thinInstances only really need a matrix table in a Float32Buffer even though for my current implementation, I have a Map of Vector3 arrays. One array per color and one color per PhysicsEventType. My expectation is that Havok would create an all-in-one array and include color information in an array parallel to the line vectors. Can GreasedLineBuilder or GreasedLineMesh be coerced into accepting that format of lines/colors?

The points property of GreasedLineBuilder looks promising as a Vector3 [ ][ ] as does the colors property of IGreasedLineMaterial. It looks like it requires an array parallel to line vertices that holds color indeces (as opposed to colors directly).

I also wonder if the Float32Buffer[ ] supplied to points can be interpreted as an array of individual lines as opposed to successive line segments?

I’ll run some experiments and try it out!. Thanks for the pointer!

2 Likes

You have a couple of properties to control the colors of a GreasedLine. If none of them suits your needs you can still create the color texture yourself. There are examples for each scenario available in the docs.

Keep in mind that:

Maximum number of colors supported for one line instance depends on the maximum texture width (we use 1D textures here for maximum performance) your GPU can support. Minimum for all WebGL systems is 4k. On most modern desktop GPUs it is 16k.

Any flat array is interpreted as one successive line. Here is an example how can you deal with it when you want to use colors and widths as well:

1 Like

I start with an array of Vector3 arrays. Each line is only two vertices. Each line is a single color and a constant width.

With CreateLineSystem, I can use

    mesh = BABYLON.CreateLineSystem("debug collision",{ lines:lines },scene)
    mesh.color = colorsByPhysicsEventType.get(event)

This works well, but because the set of lines in a LineSystem can’t grow, a new set of lines (which could variously be more or fewer lines) needs a new LineSystem. I end up dispose() and regenerate each frame.

With GreasedLine I can generate using

mesh = BABYLON.CreateGreasedLine("debug collision", 
    { 
        points: lines,
        updatable:true,
    },{
        width:.0050,
        color: colorsByPhysicsEventType.get(event),
        materialType: BABYLON.GreasedLineMeshMaterialType.MATERIAL_TYPE_SIMPLE,
    });

And I get a similar result. Works great!

However, when I use setPoints, i have to convert the array of Vector3 arrays into arrays of float arrays. Each float array has six elements the xyz coordinates of each line’s endpoints. I also don’t see any lines beyond the number of lines in the initial GreasedLine. Do I need to also provide a new widths array and/or colors array?

const points = lines.map((a)=>[...a[0].asArray(),...a[1].asArray()])
mesh.setPoints(points,{updatable:true})

With an eye on this thread about optimizing setPoints (and after I get updating working at all), I’d like to find the most efficient mechanism of updating a GreasedLineMesh that doesn’t create a lot of objects that get garbage collected every frame.

2 Likes

Here’s a video of the GreasedLine implementation. It’s 18 bouncy dice that more than occasionally fall through the floor (those are rerolled). The small white dots are the collision points (they stay on the screen for 1 second). The size of the impulse generated by the collisions, indicated by line length, are shown in red for “START_COLLISION” and yellow for “CONTINUE_COLLISION”.

1 Like

Here is a topic answering your question.

There is an option parameter called offsets. I’m thinking about precreating a pool of lines and modifying the offsets.

https://doc.babylonjs.com/features/featuresDeepDive/mesh/creation/param/greased_line#offsets

Since this is a debug tool I wouldn’t care that much about performance. On contrary it’s nice to have smooth rendering even with the debugging enabled.

If you’ll need something specific, or come up with a idea how can we make GreasedLine more performant/robust/easier to use, I can implement the underlying code.

2 Likes

It’d be great if Havok’s Visual Debugger (HVD) would be exposed in the wasm file.

1 Like

IMHO it doesn’t make sense to put there.

Thinking about it yes, you are right. But it would still be great to have access to it in other way other than the main havok wasm file.

Let’s get surprised with what is @regna / @HiGreg cooking for us :wink:

1 Like

Give this a try. This very minimally tested. Only use the constructor and enable()!

It implements visualization of Havok physics collision points as spheres that appear for one second then disappear. Specify the duration in milliseconds as well as the sphere diameter.

Be gentle, this is my first attempt at defining a class in JavaScript. Seems to work on Android in Chrome and Kiwi browsers.

hk is the havokPlugin:

    const havokInstance = await HavokPhysics();
    hk = new BABYLON.HavokPlugin(true, havokInstance);

/**
 * @example var contactsDebug = new ContactPoints(1000,0.01,hk,scene).enable()
 * This is minimally tested. enable() seems to work but no other method is tested.
 * There very well may be syntax errors in some methods.
 */
class ContactPoints {
    #duration;
    #contactSphere;
    #contactPoints;
    #hk;
    #scene;
    #onBeforeRender
    #onAfterRender;
    #onCollision;
    constructor(duration,diameter,hk,scene) {
        this.#hk = hk;
        this.#scene = scene;
        this.#duration = duration;
        this.#contactSphere = BABYLON.MeshBuilder.CreateSphere("c",{ diameter:diameter, segments:16}, this.#scene);
        this.#contactSphere.isVisible = false;
    }
    enable() {
        this.#contactPoints = new ArrayDuration(this.#duration);
        this.#onBeforeRender = this.#scene.onBeforeRenderObservable.add(this.perFrame.bind(this));
        this.#onAfterRender = this.#scene.onAfterRenderObservable.add(this.afterFrame.bind(this));
        this.#onCollision = this.#hk.onCollisionObservable.add(this.collision.bind(this));
    }
    disable() {
        this.#hk.onCollisionObservable.remove(this.#onCollision);
        this.#scene.onBeforeRenderObservable.remove(this.#onBeforeRender);
        this.#scene.onAfterRenderObservable.remove(this.#onAfterRender);
        this.#contactPoints.clear();
        this.#contactSphere.thinInstanceCount = 0;
        this.#contactSphere.isVisible = this.#contactSphere.hasThinInstances;
    }
    collision(collisionEvent){
        this.#contactPoints.add(collisionEvent.point.clone())
    }
    perFrame(edata,estate) {
        console.log( "contactSphere", this.#contactSphere)
        this.#contactSphere.thinInstanceCount = 0;
        this.#contactPoints.forEach(
            (p)=>this.#contactSphere.thinInstanceAdd(BABYLON.Matrix.Translation(p.x,p.y,p.z))
        )
        this.#contactSphere.isVisible = this.#contactSphere.hasThinInstances;
    }
    afterFrame(edata,estate) {
        this.#contactPoints.elapse(edata.deltaTime);
    }
}

/** 
 * ArrayDuration encapsulates an Array() and internally tracks the lifetime of each
 * element. Only a few methods are currently exposed to simplify storage and access.
 * 
 * This class tracks the lifetime of all elements by using a parallel array of durations 
 * and total accumulated duration. Only two subtractions are nominally needed
 * for each elapse(), i.e. not requiring a full traversal of the duration array.
 * However, elements are only added with the underlying Array()'s .push() method and
 * removed with .shift(), trading the efficiency of elapse() calculation for the
 * inefficiencies of push/shift. This tradeoff is worth it if duration is many times
 * the value of each elapsed time.
 */
class ArrayDuration {
    
    // #array and #delta are parallel arrays where #delta[i] is the
    // difference in remaining duration between #array[i] and #array[i-1].
    // #delta[0] is the remaining duration of #array[0].
    // #array is ordered from shortest remaining lifetime to longest.
    #array = Array();
    #delta = Array();
    #total = 0; // the remaining lifetime of the last element
    #duration; // the lifetime of each element added
    
    /**
     * @constructor
     * @param {number} duration - the lifetime of each element, in arbitrary units
     * such as seconds, milliseconds or frames.
     */
    constructor(duration) {
        this.#duration = duration;
    }

    /**
     * Provides access to the forEach() method of the underlying Array().
     */
    forEach(...args) { return this.#array.forEach(...args);}
    
    // these don't work:
    //f2 = this.#array.forEach;
    //get f1() { return this.#array.forEach; }
    
    /**
     * Modify the duration of future added elements. 
     * If reducing the duration, all existing element's lifetimes are also reduced.
     * @param {number} value - the new duration, in arbitrary units
     * such as seconds, milliseconds or frames.
     * 
     * This function is currently untested.
     */
    set duration(value) {
        if (value < this.#duration ) this.elapse(this.#duration-value);
        this.#duration = value;
    }
    
    /**
     * Access the underlying array. This is for read only purposes. Do not modify
     * the array order or number of elements, otherwise the parallel array of lifetimes
     * will be invalid and could cause errors. You can, however, modify or replace 
     * elements that will retain the lifetime of the original element.
     */
    get array() {
        return this.#array;
    }

    /**
     * Add a new element to the underlying Array()
     * @param {any} element - the element to add
     */
    add(element) {
        this.#array.push(element);
        
        // The remaining duration after the previous element is calculated.
        // Because the duration of the new element is #duration and
        // the previous element's remaining duration is #total,
        // #duration-#total is pushed onto #delta.
        // #total is then updated to #duration.
        this.#delta.push(this.#duration-this.#total);
        this.#total = this.#duration;
    }
    
    /**
     * elapse time and remove elements with lifetimes that have exceeded duration.
     * @param {number} value - the amount of time to elapse, in the same units as duration
     */
    elapse(value) {
        // When time elapses, only #total and #delta[0] need to be
        // reduced by the elapse value.
        // After this reduction, if #delta[0] is <= 0, then
        // that element is removed, and any negative duration is carried forward
        // to subsequent deltas with each element removed if its resulting delta is <= 0.
        // If all elements are removed, #total is then reset to 0.
        if (this.#delta.length) {
            this.#total -= value;
            this.#delta[0] -= value;

            while (this.#delta.length && this.#delta[0] <=0) {
                this.#array.shift();
                const remain = this.#delta.shift();
                if (this.#delta.length) this.#delta[0] += remain;
            }
            if (!this.#delta.length) this.#total = 0;
        }
    }
    
    /**
     * clear the stored array with optional callback when removed.
     * @param {function(element)} callback - the function called with each element
     * upon its removal. Useful for dispose() or other cleanup function.
     */
    clear(callback) {
        if (callback) this.#array.forEach((e)=>callback(e))
        this.#array.length = 0;
        this.#delta.length = 0;
        this.#total = 0;
    }
}
3 Likes

You should write it in TypeScript so it could be added to the official babylon.js repo.

// Import BabylonJS types
import * as BABYLON from 'babylonjs';

interface CollisionEvent {
    point: BABYLON.Vector3;
}

// HavokPlugin Interface - define it as per HavokPlugin's actual structure
interface HavokPlugin {
    onCollisionObservable: BABYLON.Observable<CollisionEvent>;
}

class ContactPoints {
    private duration: number;
    private contactSphere: BABYLON.Mesh;
    private contactPoints: ArrayDuration;
    private hk: HavokPlugin;
    private scene: BABYLON.Scene;
    private onBeforeRender: BABYLON.Nullable<BABYLON.Observer<any>>;
    private onAfterRender: BABYLON.Nullable<BABYLON.Observer<any>>;
    private onCollision: BABYLON.Nullable<BABYLON.Observer<CollisionEvent>>;

    constructor(duration: number, diameter: number, hk: HavokPlugin, scene: BABYLON.Scene) {
        this.hk = hk;
        this.scene = scene;
        this.duration = duration;
        this.contactSphere = BABYLON.MeshBuilder.CreateSphere("c", { diameter: diameter, segments: 16 }, this.scene);
        this.contactSphere.isVisible = false;
        this.onBeforeRender = null;
        this.onAfterRender = null;
        this.onCollision = null;
    }

    enable(): void {
        this.contactPoints = new ArrayDuration(this.duration);
        this.onBeforeRender = this.scene.onBeforeRenderObservable.add(this.perFrame.bind(this));
        this.onAfterRender = this.scene.onAfterRenderObservable.add(this.afterFrame.bind(this));
        this.onCollision = this.hk.onCollisionObservable.add(this.collision.bind(this));
    }

    disable(): void {
        if (this.onCollision) {
            this.hk.onCollisionObservable.remove(this.onCollision);
        }
        if (this.onBeforeRender) {
            this.scene.onBeforeRenderObservable.remove(this.onBeforeRender);
        }
        if (this.onAfterRender) {
            this.scene.onAfterRenderObservable.remove(this.onAfterRender);
        }
        this.contactPoints.clear();
        this.contactSphere.thinInstanceCount = 0;
        this.contactSphere.isVisible = !!this.contactSphere.hasThinInstances;
    }

    private collision(collisionEvent: CollisionEvent): void {
        this.contactPoints.add(collisionEvent.point.clone());
    }

    private perFrame(edata: any, estate: any): void {
        console.log("contactSphere", this.contactSphere);
        this.contactSphere.thinInstanceCount = 0;
        this.contactPoints.forEach((p: BABYLON.Vector3) => {
            this.contactSphere.thinInstanceAdd(BABYLON.Matrix.Translation(p.x, p.y, p.z));
        });
        this.contactSphere.isVisible = !!this.contactSphere.hasThinInstances;
    }

    private afterFrame(edata: any, estate: any): void {
        this.contactPoints.elapse(edata.deltaTime);
    }
}

class ArrayDuration<T> {
    private array: T[] = [];
    private delta: number[] = [];
    private total: number = 0;
    private duration: number;

    constructor(duration: number) {
        this.duration = duration;
    }

    forEach(callback: (element: T) => void): void {
        this.array.forEach(callback);
    }

    set durationValue(value: number) {
        if (value < this.duration) this.elapse(this.duration - value);
        this.duration = value;
    }

    get arrayValue(): T[] {
        return this.array;
    }

    add(element: T): void {
        this.array.push(element);
        this.delta.push(this.duration - this.total);
        this.total = this.duration;
    }

    elapse(value: number): void {
        if (this.delta.length) {
            this.total -= value;
            this.delta[0] -= value;

            while (this.delta.length && this.delta[0] <= 0) {
                this.array.shift();
                const remain = this.delta.shift();
                if (this.delta.length) this.delta[0] += remain!;
            }
            if (!this.delta.length) this.total = 0;
        }
    }

    clear(callback?: (element: T) => void): void {
        if (callback) {
            this.array.forEach((e) => callback(e));
        }
        this.array.length = 0;
        this.delta.length = 0;
        this.total = 0;
    }
}

or even better with ES6 imports:

// Importing necessary components from Babylon.js
// TODO: fix imports
import { MeshBuilder, Scene, Observable, Observer, Vector3, Matrix, Mesh } from 'babylonjs';

// Defining the interface for the collision event
interface CollisionEvent {
    point: Vector3;
}

// Defining the interface for HavokPlugin
interface HavokPlugin {
    onCollisionObservable: Observable<CollisionEvent>;
}

class ContactPoints {
    private duration: number;
    private contactSphere: Mesh;
    private contactPoints: ArrayDuration<Vector3>;
    private hk: HavokPlugin;
    private scene: Scene;
    private onBeforeRender: Nullable<Observer<any>>;
    private onAfterRender: Nullable<Observer<any>>;
    private onCollision: Nullable<Observer<CollisionEvent>>;

    constructor(duration: number, diameter: number, hk: HavokPlugin, scene: Scene) {
        this.hk = hk;
        this.scene = scene;
        this.duration = duration;
        this.contactSphere = MeshBuilder.CreateSphere("c", { diameter: diameter, segments: 16 }, this.scene);
        this.contactSphere.isVisible = false;
        this.onBeforeRender = null;
        this.onAfterRender = null;
        this.onCollision = null;
    }

    enable(): void {
        this.contactPoints = new ArrayDuration(this.duration);
        this.onBeforeRender = this.scene.onBeforeRenderObservable.add(this.perFrame.bind(this));
        this.onAfterRender = this.scene.onAfterRenderObservable.add(this.afterFrame.bind(this));
        this.onCollision = this.hk.onCollisionObservable.add(this.collision.bind(this));
    }

    disable(): void {
        if (this.onCollision) {
            this.hk.onCollisionObservable.remove(this.onCollision);
        }
        if (this.onBeforeRender) {
            this.scene.onBeforeRenderObservable.remove(this.onBeforeRender);
        }
        if (this.onAfterRender) {
            this.scene.onAfterRenderObservable.remove(this.onAfterRender);
        }
        this.contactPoints.clear();
        this.contactSphere.thinInstanceCount = 0;
        this.contactSphere.isVisible = !!this.contactSphere.hasThinInstances;
    }

    private collision(collisionEvent: CollisionEvent): void {
        this.contactPoints.add(collisionEvent.point.clone());
    }

    private perFrame(edata: any, estate: any): void {
        console.log("contactSphere", this.contactSphere);
        this.contactSphere.thinInstanceCount = 0;
        this.contactPoints.forEach((p: Vector3) => {
            this.contactSphere.thinInstanceAdd(Matrix.Translation(p.x, p.y, p.z));
        });
        this.contactSphere.isVisible = !!this.contactSphere.hasThinInstances;
    }

    private afterFrame(edata: any, estate: any): void {
        this.contactPoints.elapse(edata.deltaTime);
    }
}

class ArrayDuration<T> {
    private array: T[] = [];
    private delta: number[] = [];
    private total: number = 0;
    private duration: number;

    constructor(duration: number) {
        this.duration = duration;
    }

    forEach(callback: (element: T) => void): void {
        this.array.forEach(callback);
    }

    set durationValue(value: number) {
        if (value < this.duration) this.elapse(this.duration - value);
        this.duration = value;
    }

    get arrayValue(): T[] {
        return this.array;
    }

    add(element: T): void {
        this.array.push(element);
        this.delta.push(this.duration - this.total);
        this.total = this.duration;
    }

    elapse(value: number): void {
        if (this.delta.length) {
            this.total -= value;
            this.delta[0] -= value;

            while (this.delta.length && this.delta[0] <= 0) {
                this.array.shift();
                const remain = this.delta.shift();
                if (this.delta.length) this.delta[0] += remain!;
            }
            if (!this.delta.length) this.total = 0;
        }
    }

    clear(callback?: (element: T) => void): void {
        if (callback) {
            this.array.forEach((e) => callback(e));
        }
        this.array.length = 0;
        this.delta.length = 0;
        this.total = 0;
    }
}

Courtesy of ChatGPT

2 Likes