Looking for advice on implementing a custom scripting language

Off topic
1

Ola all,

I want to make an easy to use browser-based game editor, which simply allows you to import some resources (models/audio/etc) and lets you make scripts to make things happen. Databases, collaboration, anti-cheat and server hosting all get automatically managed by the editor. I want this engine to be as easily maintainable/upgradable as possible, so I can always stay on edge performance wise.

As a part of this plan, I want a simple to learn scripting language which has library binds to babylon (which will be doing most of the hard work). This means I only have to maintain the language and some server stuff, the content gets created on top of the hood using the custom language itself.

I want to use WASM for my custom language, primarily because of security concerns of making a javascript-based interpreter (which would mean extra maintenance). I’ve done some research into this field, but on top of all the other research I have to do, its starting to get overwhelming.

What I’ve found so far:

  • Lua (my most likely option, but lacks flexibility)
  • Typescript/C/C#/etc to WASM compiler (I want to be able to customize the language for simplicity)
  • Eclipse xText (not sure if I can compile this to wasm, but seems ideal)
  • LLVM (way too ambitious)

I am wondering if anyone else has ever tackled the problem of making a custom (in-browser) scripting language, or if anyone knows any alternatives to the ones mentioned above. Any advice or feedback is welcome, thanks for any help!

2

Why not simply using javascript?

1 Like
3

Hey Swat,

Sounds interesting! Curious though, what are your security concerns with a JS interpreter? The BJS PG does essentially that same task and AFAIK there aren’t security issues with it.

Have you considered going with a custom DSL (domain specific language) instead of full blown scripting? A DSL has the advantage of being totally defined by you to do exactly what you want (which is also the problem with it!)

With regards to scripting, the Roslyn C# compilers and source generators make it very easy to do what you’re describing. If you’re looking in that direction, it’s worth checking out.

I think you’ll need to decide what balances ease of development with granularity of control for your users; should users have to know or think programming or is a Roblox style of development what you’re aiming at?

HTH

1 Like
4

My main assumption comes from another browser game called Krunker, which had a custom script that would compile down to JavaScript. It created a ton of security issues for them and slowed down features massively because of it. Security in this case also means anti-cheat, and thus compatability with anti-cheat. (sometimes they’d update their scripting language but the anti-cheat would block features)

1 Like
5

Ola, in regards to my security concerns, see this quote;

The BJS compiler just executes javascript, which is not a problem, unless you have a browser game which needs to prevent people from hacking client-side code. On top of this, scripts would be downloaded when you join a server with scripts, meaning you could do xss attacks if it was regular javascript.

I think DSL is what I am aiming for. The main one I found for this was xText, Roslyn C# seems interesting, and I’ll make sure to check it out. In the end it should just be a relatively barebones language (think Lua), but that has enough control to modify/extend it when needed. (I also kind of dream of having it be type-based for a better workflow)

Aside from making the language, it has to work in a browser though, thats also a hard part of using a custom language :confused:

1 Like
6

Hi,

I tried something similar a few years ago. At that time I wanted to build an editor with which you can automatically build a 3d configurator including GUI. I quickly moved away from webassembly, more about that in a moment. To build a browser based editor I used Retejs and Blockly from Google. After a while I dropped this project, because it doesn’t fits my customer needs.

Now something about the Wasm stuff. First of all, webassembly or more precisely binary code does not prevent someone from stealing your code or manipulating it. Most games are not written in javascript and still people cheat. With Babylonjs there is a second problem. In order for Webassembly to interact with it, you have to develop a Javascript interface, which is where I would go if I wanted to cheat. In addition, the browser still shows you all the network requests that cheaters manipulate, hoping the server will accept them anyway.

Performance is also a thing. Javascript is highly optimized nowadays and it depends on the use case if you are really faster in the end. If you don’t use Rust or C++, you won’t gain much. The best example is Go, where the whole garbage collector is shipped with the wasm binary.

To build a DSL I recommend Jetbrains MPS. We once had two consultants in our company who presented it to us. It was pretty impressive, but my boss at the time decided that we didn’t need it :slight_smile:

If you need some further information, feel free to contact me :wink:

2 Likes
7

Good points on the Wasm stuff. Depending on the nature of your needs, you may want to look into running server-side authoritative physics or other simulations.

8

Thank you for all this insight! I’ve actually just started following a free course about MPS, it looks like it has a lot of potential. I really hope I can make an interpreter with it though, as I realized that compiling a language server-side is slow.

The main reason I want to use WASM is to have the scripting language in a seperate enviournment which does not have as much control over the browser. My main concern being people somehow using exploits in a Javascript based langauge to xss token grab or keylog. Using WASM hopefully also makes the barrier of entry for cheating higher, as WASM can’t yet be modified with script injecting extensions (as far as I know), which is the primary workflow browser game cheaters are used to. I also want to avoid having to make an entire language from scratch in JS to maintain, and would prefer to use some kind of a library/dsl that does most of the heavy lifting, and I think using WASM gives me more options for tools to use.

Performance wise, I totally believe JavaScript can be a plenty fast language. From my research it seems like WASM is only faster the more you make it do, but because entire games are built on top of this custom scripting langauge, I think it will end up balancing it out to be about equally fast. Hopefuly as time passes WASM will get faster and faster, making optimising the game/engine an automatic experience for me :stuck_out_tongue_winking_eye:

1 Like
9

Okay, you are webassambly fan :slight_smile: But before you start a project that you never get finished within 3 months (this is the time when most people quit, because it starts to be get boring ^^), look how others solved the problem to create a game engine or something like that. Unity 3d i.e. took Monogame as their environment and used C# including the toolchain as scripting language (with just a lexer, a parser and a compiler you are not finished - you also need an editor, a bundler and some other things). This had two advantages:

  1. They were able to completely focus on building new features for the game developer
  2. C# was a widely used and accessible language, which made the entry easier for new developers (no one really wants to learn godot script :wink: )

The success proves them right :wink: So I recommend you to use an existing language with toolchain, which can be compiled to webassembly. May be here is something for you, which you can use:

1 Like
10

Regarding your concern about running code in a separate environment, have you considered using Service Workers to sandbox your code?

Security-wise, they’re served from same origin only and only if they context is secure. True though, it doesn’t eliminate script injection hacks and such, it does make them more difficult to employ

11

I love C#! As I mentioned in the post, I really want to do as little work as possible and don’t mind using existing solutions. That said, I havn’t found a language yet that quite hits the right spot and being easy for non-programmers to pick up/teach. I really do need strong control over this language as the entire game is built around it. “They were able to completely focus on building new features for the game developer” in my case, the programming language is what offers most of the features, so im quite positive the work put into it is worth it.

That said, I do agree with everything you’re saying, ill definitely be looking around more for existing solutions, this is such a complex topic :sweat_smile:

12

Hmm I have heard of this but didn’t occur to me, I’ll have a look at it, thanks!

1 Like
13

I would be interested to understand this in more detail. Have you got an example of a feature you think would be better to design into a language instead of providing an API for someone to use that leverages an existing language, lexer/editor, parser & compiler toolchain?

14

This is great. Is any of it public? And did you have JavaScript that interpreted that data structure (which contained the data for the nodes and connections) and performed the necessary calculations on the client or did you send it to a server to be compiled and then put a new script tag into the HTML to fetch the newly compiled code or something?

15

No, nothing public yet. Just a proof of concept.

Yes everything is running client side in javascript. The idea was to save the structure as a json file and create a bundle without the editor stuff around. So that you deliver only the 3d view, the generated 2d ui, a css stylesheet for customizing and in the background babylonjs + the calculation engine.

1 Like
16

Okay guys, I am aware my opinion on this topic has been kind of all over the place, but hey, thats what this post was for :wink:

I have a relatively inexperienced scripting user base, and making a simple custom scripting language for them seemed like a fun challenge. That said, it was way more work than expected sadly, and I still had the rest of the project to still focus on.

I ended up using (sort of) vanilla TypeScript that gets validated and transpiled by the server. The TypeScript gets split into server and client code, it gets uploaded to the server, and the validated/transpiled client code gets sent back to the client as a service worker (Ty for jelster for the idea). There is a couple reasons why I think TypeScript is the best way to do this:

  • It can be compiled to WASM to make life more difficult for cheaters. This is great for “published” maps, as its slow to do.
  • It can be transpiled into JS very fast using esbuild. This is important as I transpile on the server. (As opposed to using a compiled language).
  • Like many here said, it’s an already finished and functioning product, saving time, effort and maintenance. (which is my goal with this project). Some people also have experience with it already.
  • Typescript already supports types, and it is relatively easy to transpile to it. TS and ESLint both give you a lot of tooling to modify the language, which allows me to make the syntax changes I want down the line. I’ve already got some proof of concepts down, but I’ll first focus on getting the game to work with normal TS.

The reason I send the code to the server for compiling:

  • I have to run the server-side scripts on the server anyways.
  • Prevents xss and hacks. Default methods can be taken out on the server side, like this:

Client:

Reflect.defineProperty(Object.prototype, "health", {value: 100, writable: false})

Server:

Reflect = {};
Object = {};

// Uploaded code
Reflect.defineProperty(Object.prototype, "health", {value: 100, writable: false}); // Uncaught TypeError: Reflect.defineProperty is not a function

With this setup, I have:

  • access to existing content from a blooming ecosystem
  • tools for obfuscation, minification, optimization, treeshaking and more
  • an entire stack using the same language (TS)
  • optional modification of the language
  • speed in runtime and transpilation time
  • many options in terms of security
  • easy direct access to babylon
  • very little work on my end :slight_smile:
3 Likes