Can we have a p5.js sketch as a texture in babylon?

Questions
1

Hello everyone, this is my first time posting on this forum, first time posting on any forum as a matter of fact.

My goto library for artistic programs or anything graphics (2d) was (and still for quick sketches) is p5.js
Hence i was using webgl from p5.js, which is ok for a quick sketch or two, but not powerful and does not give the performance i desire.

I recently made this thing where you can using nothing but p5.js , you can have a look here : https://workingbuild.herokuapp.com/

you can have a look at a video of how the site is like roughly

I wish to recreate it (if possible) in babylon, since it i don’t think i can target low end devices this way, or even expand what I’m trying to do.

In a nutshell, i made it so that with a few small changes, a p5 sketch can be used as a texture on any plane and support mouse events, beautiful things can be done this way like playing chess on a wall, real time with another player moving around and such. The site i linked does somewhat the same, i have a flocking simulation put in there for demonstration

using it is very easy since we can just pass a PGraphic object to the texture() function before drawing a plane, this PGraphic can be a object of its own, and we can figure out mouse collisions using others algos ( i am using aabb for now but i wish to make this better too ) .

I was wondering if it is possible to make it so that i can have a p5 PGrahpic object as a texture on babylon planes. I wish to be able to have the freedom so that every wall is a program of its own and can be however complex, I recently demonstrated this which is not on the internet but it used p5.js as well and pretty much the same concept, imagine walking down a lobby which is like a gallery, one wall has game of life running in it, the other has a cloth simulation, another is the Mandelbrot set. It truly is a beautiful site and shows the beauty of computer graphics.

Thank you for reading this, I would be very happy if this is possible!

2

Hi Johnny,

This seems very cool, but pretty ambitious. At a surface level, assuming (as I’m unfamiliar with P5.js) the PGraphic object you mentioned is underpinned by this texture abstraction, it theoretically shouldn’t be too difficult to get Babylon to use this texture; all you should have to do is insert the underlying GL texture into a Babylon.js texture. Babylon doesn’t really support this, per se, but it’s not particularly hard to do if you’re willing to reach into some internals.

So, if that’s easy for you to try, I’d give that a shot, but there may be other issues with having two rendering engines trying to coexist in the same space; they may fight over the stateful GL context, for example. Moreover, I’ll be interested to learn if using Babylon.js for the 3D aspects meaningfully improves your performance here. Babylon is very fast, but I’m not sure the 3D part it’d be taking over was the slow part in the first place: there’s not much 3D going on, and depending on how P5 does the 2D stuff, that might be where the slowness is. You can do all the same 2D effects and things you’re doing in P5 in pure Babylon, of course, and that would almost certainly be faster and simpler. However, that would remove P5 from the equation altogether, which might defeat the purpose of what you’re trying to achieve.

Hope this helps, and best of luck!

1 Like
3

Thank you for replying! Yes it seems it is possible to tinker with the internals a little bit to mold the library to my liking, it would be a tough challenge too! I would surely spend a week or two learning how to do it though :v

I would like to ask another small thing, is it possible for us to get a pixel array as a texture? P5’s PGraphic object holds within an array which in a nutshell is all what it “needed”, the rest can be thought of as additional features. I think this should also be quite doable since its like an image after all, more like a video since this image can update causing the texture in turn to update as well.

I used Babylon and p5 and there is no competition! babylon takes the win by a mile ( in performance its far far better), at least with programs which i very much like, stuff like particle simulations and large maps. However there is a beauty in p5 and the simplicity it offers users to do! For that i shall try to glue this solution together

I wish the GL to be controlled by babylon at all times, i just wish to have a draw loop where i can update the p5 texture sketch every frame! That should be a piece of cake.

Thank you so much for answering! I will have to learn a lot about babylon and webgl to get this working, off i go to make something beautiful!