[r]evolution Summer 2022: Wasm Space Program

For some time now I have wanted to build a universe simulation. The idea of millions of spaceships each making their own real-time decisions in a massive universe sounds compelling.

This idea isn't new — many video games and AI competitions have explored the same topic. But to my knowledge, none of them have tried to run the entire simulation within a database. So, I made this my goal and started building.

To complete this project, I would need a unique database. To start with, it needs to support large volumes of transactions and analytics at the same time on the same data. This is known in the database community as "HTAP" (Hybrid transactional/analytical processing). Next, it must be able to handle a highly concurrent read and write workload to support thousands of clients. Finally, the database needs to be extensible with custom logic to run the custom AI powering each spaceship.

Luckily for me, SingleStoreDB 7.9 satisfies all of these requirements and more. A bit of elbow grease and some long nights later, I put the finishing touches on what I call the Wasm Space Program.

Simulation Walkthrough

The Wasm Space Program is available online, so I suggest opening it up in another tab to follow along while reading the rest of this blog post. You can access it here: Wasm Space Program

After clicking to enter, you will see a screen full of solar systems.

Each of the orange/yellow circles on your screen is a unique solar system. You can expand the "Universe Stats" view in the top-left corner to see how many solar systems there are, as well as other details about the simulation. Let's click on a solar system to warp to it and see what is inside.

Within each solar system there are spaceships and energy nodes. Energy nodes look like asteroids covered in green crystals. You can think of energy nodes like phone chargers — spaceships need to sit on top of them to absorb energy. Since spaceships need energy to survive, energy nodes are a critical resource in the universe.

You will also see spaceships moving around. Each spaceship is driven by a program written in Rust which implements that spaceship's behavior. There are a number of different behaviors so far, ranging in aggression and movement strategies. Spaceships can choose to fight by occupying the same square as another ship. This will lead to combat, which results in the losing side being destroyed — leaving all of its remaining energy on the board for the victor to absorb.

I encourage you to click on different entities and inspect their statistics in the info panel which pops up. You can also open up the information screen by clicking "Information" in the top left corner.

So, how does this work?

The Wasm Space Program takes advantage of three major features in SingleStore DB 7.9 and SingleStore Cloud: Code Engine — Powered by Wasm, Workspaces and the Data API.

Code Engine — Powered by Wasm

To avoid needing a separate backend service to run each spaceship's AI, we use SingleStoreDB Code Engine. This feature supports creating functions using code compiled to WebAssembly (Wasm). In the case of the Wasm Space Program, each spaceship's AI is written in Rust.

This allows me to take advantage of the developer ergonomics and library ecosystem of Rust, while ending up with a function that I can run directly within my SQL queries. In addition to powering the spaceships, I also am using Wasm for other utility functions which are easier to write in Rust than in SQL. All of the Rust code used in this simulation is available here.

See more: Code Engine — Powered by Wasm

Workspaces

In SingleStore 7.9, we have released the ability for multiple compute clusters (called Workspaces) to share access to the same database. One of the workspaces can read and write, while the rest of the workspaces can only read from the database. This feature is extremely useful when you want to isolate portions of your workload from one another.

For the Wasm Space Program, it's extremely important that each turn takes less than one second to ensure the game runs in real time. To achieve this, we run the write workload in a dedicated workspace. Meanwhile, we need to power thousands of game clients running read queries against the universe — so we use multiple read only workspaces to ensure we can handle any scale.

See more: Workspaces

Data API

The final piece of the puzzle is how to connect directly from the game client running in the browser (Javascript) to SingleStoreDB. Historically, SingleStoreDB only supported the MySQL protocol which sadly can not be used directly from a browser. So, you would need to build a backend API service to act as the middleman between browser and SingleStore. This added complexity to application architecture as well as a lot of boilerplate code which didn't provide much business value.

Well, I'm happy to say that the Data API provides a solution to this problem. With the Data API, you can connect directly from the browser to SingleStore over normal HTTP(s) connections. In addition, query results come back in JSON which makes them trivially easy and fast to handle in Javascript.

The Wasm Space Program uses the Data API to power the game client. Everything you see on the screen is the result of running SQL queries directly from your browser to SingleStore with no backend service or API in the middle. This results in performance improvements, development agility and deployment simplicity for the application.

See more: Data API

Looking forward

I have worked at SingleStore for over nine years and can't emphasize enough how revolutionary SingleStore DB version 7.9 is. The ability to extend the database with custom logic and run low latency transactional and analytical workloads on the same storage completely changes how I think about building applications.

The Wasm Space Program will be undergoing some changes in the upcoming weeks, culminating in an open competition to see who can develop the best spaceship AI. Stay tuned for rules and instructions on how to join in on this experience.

In the meantime, please check out the code on Github and try running it on your machine. You can even build your own spaceship behavior by adding strategies here. Take a look at how the other strategies are organized and injected into SingleStore by searching for their names throughout the codebase. If you encounter any issues or want to send a pull request, all contributions are welcome.

Thanks for playing along, and have a fantastic day!

Missed [r]evolution 2022? Check out all of the sessions on demand.