"Hybrid Graph" GLSL shader editor tech preview for Babylon

[andyray]

Hi folks, I’ve been building a new type of GLSL shader graph editor as a nights and weekends project.

I’m calling this type of graph editor a “hybrid graph” editor. It’s “hybrid” because you can write raw GLSL in nodes, including full shaders, and then you can compose nodes together arbitrarily in the graph.

I recently made enough updates to the BabylonJS implementation of the graph editor to feel comfortable sharing it here. Here’s the demo link: Home - Featured Shaders - Shaderfrog 2.0

Try changing the example dropdown on the top middle to see some cool effects made with this editor. Here’s some screenshots:

There’s a lot to explain in how this demo works, but I’ll only call out a few things in this early preview.

1. To get an idea of the “hybrid graph”, try selecting the “Living Diamond” example, then find the green “purple metal” node in the graph, and double click on it. You can see that this is a complete raw GLSL shader program. The hybrid graph engine automatically parsed inputs and outputs from this node, making it composable in a graph.
2. Note that the graph is using the full “purple metal” shader node as the bumpTexture of a Babylon PBRMaterial. The hybrid graph basically lets you replace any source code of any shader with the output of any other GLSL node. So in this case, it’s hijacking the Babylon PBRMaterial source code, replacing the bumpTexture lookup with another shader function.
3. You can see the final compiled GLSL output in the “Compiled Source” tab. You can also edit this source code inline and see the updates. It’s useful to see what the engine (Babylon) is doing when compiling shaders.
4. This demo is early, and there’s plenty of ways to break it! Lots of edge cases, missing features, etc.

This is a GLSL editor, it’s not specifically a BabylonJS editor. BabylonJS is implemented as a plugin to the hybrid graph editor. You can also change the engine from BabylonJS to ThreeJS, and the exact same graph editor works for ThreeJS.

Right now in the demo you can’t save anything, it’s in-memory only.

This is the next evolution of the tool I built, Shaderfrog (three.js only). This new tool is open source: GitHub - AndrewRayCode/shaderfrog-2.0-hybrid-graph-demo: Experimental Shaderfrog 2.0 Hybrid Graph Shader Composer

To create the Hybrid Graph, I wrote a full GLSL compiler, which is already being used by lots of other projects: GitHub - ShaderFrog/glsl-parser: A GLSL ES 1.0 and 3.0 parser that can preserve whitespace and comments

Feedback welcome! There is a lot more I could add, so questions welcome as well.

[mawa]

Would need some time to check it out but it looks really amazing. I’m gonna keep this reference for later and will return to it later on. So far the work that has been done as well as the look’n feel is really engaging and is also pretty intuitive. I’m usually not a big fan of ‘editors’ but I think this one deserves a special mention. GJ,

[gabrieljbaker]

Hi @andyray this is absolutely nuts. Will be digging in on this all week.

[andyray]

@mawa

I’m usually not a big fan of ‘editors’

Understandable. I’m focusing on the UI layer right now (the graph editor), and it’s built on top of a core system which is decoupled from the UI. In future iterations I hope to have this API more cleanly defined so folks can import it into their projects to compose shaders, without any of the graph editor / UI components

[carolhmj]

Wow this is AMAZING! The diamond example is fantastic, and being able to easily combine GLSL code with a graph like approach is a super neat idea I’m not sure if I missed any instructions, but I wasn’t able to delete a node? Tried selecting it and pressing “del” on my keyboard but it didn’t seem to do anything

[andyray]

It’s possible you hit a Javascript error, there are still plenty of edge case bugs that can break the tech demo.

Also what operating system are you on? I’m able to delete a node with my Mac keyboard pressing “delete,” it may be a different keycode on unix/windows. I’ll test on my windows machine as well.

[andyray]

being able to easily combine GLSL code with a graph like approach is a super neat idea

Thank you! That’s the promise of this kind of editing system.

You can also create “expression” nodes by checking the “expression only” checkbox in the source code editor pane, below the code editor. What this means is that instead of needing hundreds of standard nodes for things like time, sin, viewMatrix, etc, you can just write that in an expression node however you want, and then plug in that expression to the source code of any shader. And instead of needing to manually wire together dozens or even hundreds of nodes, you can just write a few lines of code in a single node, and compose it arbitrarily.

[carolhmj]

Hmmm I’m not seeing any error on the console, so it may be a different key code. I’m using Windows 11 with Edge on Version 110.0.1587.57 Tried on Firefox too with the same result. It seems that the key code for my delete is 46, might be different on Mac indeed?

[mawa]

@carolhmj

delete is 46, might be different on Mac indeed

Nah, keyDown and Up is the same on mac. keyCode for ‘DEL’ is 46.

[Mostafa_Salloum]

@andyray is there any updates on the editor ? i am trying to use the fragment and the vertex in the cyos.babylon but i am getting a lot of errors, is there something i have to do extra because i am just copying the vertex and put it in the vertex at cyos same for the fragment

[carolhmj]

What kind of errors are you getting?

[andyray]

Hi, unfortunately there are not major updates to the Babylon side yet. It probably won’t be easy to use these shaders in your own project any time soon. You probably can use them with lots of work, but there’s no easy export-to-your-project right now.

The premise of this tool with Babylon is:

• Shaderfrog creates a BABYLON.PBRMaterial (or whatever built in material type you want)
• Shaderfrog reads the Babylon generated GLSL, and then modifies it to add new effects by injecting new GLSL
• Shaderfrog dumps the new compiled GLSL back into the BABYLON.PBRMaterial, and updates the material to add a new uniforms.

I made a Babylon playground example which lets you edit the raw GLSL of a Babylon shader, which can be helpful for debugging the overall workflow: https://playground.babylonjs.com/#YPXI4C#19

If you really wanted to, you could take the output from the “Compiled GLSL” tab in the Shaderfrog demo, and paste it into the text box in the playground, and then start hacking on it until it works, but this probably isn’t worth the effort. Also note the playground link above only lets you edit the fragment shader, you’d need to add another textarea to edit the vertex shader as well.

[andyray]

I’ve updated the demo to persist shaders to a database now, and gave it a new URL, so it’s easier to share examples

This demo shows off mixing Babylon default shader(s) with custom ones. A Babylon PBRMaterial is simply a node in the graph, it can be composed arbitrarily like any other node.

Andy Ray

(In case the tweet doesn’t load), running live for Babylon: Striped Glass Fireball - Shaderfrog 2.0 Hybrid Graph Demo

And running live for Three.js(!): Glass Striped Fireball - Shaderfrog 2.0 Hybrid Graph Demo

The point of these two examples is to show this same algorithm of the hybrid graph can work for multiple engines (Babylon, Three, and theoretically any other webgl engine)

Still no export capabilities yet

[andyray]

Now this same Babylon graph editor supports Three.js, and I recently got Playcanvas working (still pretty broken): https://twitter.com/andrewray/status/1687864488417255424?s=20

Imagine the same graph editing tool letting you port shaders/materials between engines. I’m still working towards getting the striped glass fireball effect shown above in Babylon working in Playcanvas. Most of the work is getting the engine plugin for Playcanvas built properly.

[andyray]

Shaderblog blog post explaining more about the hybrid graph alpha, and current state: Introducing the Shaderfrog 2.0 Hybrid Graph Editor Alpha

I also improved the load time of the Babylon editor by importing only the Javascript modules needed, and not import * from 'babylon'

[andyray]

Another good example of cross-engine capabilities. Babylonjs shader: Glass Water - Shaderfrog 2.0 Hybrid Graph Demo

And same graph in Three.js: Water Glass - Shaderfrog 2.0 Hybrid Graph Demo

[carolhmj]

I’m seeing an error in the console

[andyray]

Oh nice! It’s rare to find a browser specific bug these days. Looks like it’s Safari specific. At least I can reproduce in Safari on Mac, but not in Chrome. It also doesn’t load on iOS Safari.

[andyray]

Fixed the babylon shader. On Safari, Babylon generates different GLSL than Chrome, specifically it adds uniform Material { reflectionMatrix } while on Chrome it just adds uniform mat4 reflectionMatrix . Is because Safari supports UBOs and Chrome doesn’t?

[Evgeni_Popov]

It’s other way around, we must disable uniform buffers support on iOS because of some bugs of the OS. You should be able to replicate the problem on Chrome by setting engine.disableUniformBuffers = true;.