We're at the Game Developers Conference (GDC) this week in San Francisco. Stop by our booth if you're reading this on the cable car on the way to the exhibition hall.

Somewhat surprisingly, Dolt's biggest vertical is video game development. That's weird. Why? What do game developers do with the world's first version controlled SQL database? This article will answer that question.

Game Configuration

Modern video games have gigabytes of configuration traditionally stored in YAML, JSON, or even XML files. Those files are stored in Git, or a competing version control system, so developers, designers, and artists can follow standard software development practices to change, review, and deploy configuration.

Configuration in games is growing in size proportional to the complexity of games. Every level, item, environment object, character, and piece of dialogue requires configuration. When talking to game developers, often much of the iterative portion of game design is configuration changes. Configuration adjusts the difficulty of a challenge. Configuration changes a game's look and feel. These changes must be isolated from other changes, generally using branches. Version control is a necessity to keep all these changes organized.

To put this in perspective, we talked to one game developer whose game series has been around since the late 1990s. Their game was home to 400 gigabytes of XML configuration files. The configuration was stored in version control, Perforce in this case. Another developer of an open world role playing game had 200GB of YAML even before their game was even released.

So Many Files...

Let's calculate how many words are in 100 Gigabytes. A letter in modern encoding (UTF8 or Unicode) is approximately 2 bytes. An average word has 5 letters plus a space. That's 12 bytes per word. A binary gigabyte has 1,073,741,824 bytes.

1,073,741,824/12 =  89,478,485
89,478,485 x 100 = 8,947,848,500

That's right. Approximately 9 billon words, 4.5 billion key value pairs. Game configuration gets complicated.

At this scale, individual files can get over 100 megabytes, a key file size in Git. At this point, GitHub will no longer accept the files and you must switch to Git LFS. Git itself recommends files stay much smaller than that to maintain acceptable performance. Git LFS does not support granular diffs. To get useful diffs even at smaller files sizes, the configuration must be ordered and structured the same way. JSON, YAML, and XML are generally unordered file formats so some tools will not write files back in the same order they were read. So, at some scale you lose the ability to have human readable diffs on your configuration changes. As configuration gets large, version controlling it in standard version control systems like Git becomes very challenging.

What a mess

With that much configuration, you also start to want features that files aren't very good at. How do you find the configuration value you are looking for? How do you prevent duplicate configuration from showing up in another part of the file tree? How do you make bulk updates to similar configuration values? Game developers go to their tools engineer or team and start to ask for solutions. The tools folks do their best to create custom tools that help remediate or fix point issues.

But the root of the problem is, structured data is hard to manage using files when it's this big.

Use a database

There's a purpose built tool for managing large amounts of structured data. Databases have been up to this task for over 50 years. Some game developers we talk to moved their configuration to some sort of database but then they lose the software development lifecycle powered by their version control system that works so well. So most plod along with many large configuration files in Git.

Before Dolt, the tools team had a terrible choice. Ditch version control for configuration and get a better system to store and query configuration data. Or, keep version control for configuration and fight a losing battle keeping hundreds of gigabytes of files clean and organized.

Enter Dolt

Dolt allows game developers to have the best of both worlds for game configuration. You get the data integrity and querying capabilities at scale of a modern SQL database. You get the software development lifecycle you know and love powered by version control including branches, merges, diffs, and pull requests (ie. integrated human review of changes). Dolt is an obvious choice to manage your game configuration. Dolt manages game configuration at any scale.

Last year for GDC, we built a Dolt powered port of Endless Sky, a popular open source game, to show off Dolt's capabilities. We wrote about how Scorewarrior uses Dolt to manage configuration for their popular mobile game, Total Battle. We have a number of other game companies using Dolt. Scorewarrior is the only one kind enough to let us talk about them, yet.

Sound interesting? Come by our booth at GDC in the next couple days or year round at our Discord. We're happy to discuss your game development use case.