Dolt is a version controlled database. In order to offer full, robust, Git-style version control for both schema and data, Dolt makes some performance trade offs. Each write needs to be stored to disk and hashed for easy comparison against what is there. This extra work makes Dolt slower on write by design. It also adds overhead on read but not much. So, Dolt will be a slower than its sibling databases, MySQL and Postgres.

But how much slower? Starting late 2020, we implemented sysbench benchmarks against MySQL and we've been optimizing since. On the sysbench tests, we're down to 7X MySQL on writes and 9X MySQL on reads. We started around 20X on these tests so Dolt is about twice as fast. Moreover, we just published a blog on the TPC-C benchmark that shows us 70X slower than MySQL for a high write contention use case involving transactions.

That sounds bad. Why would anyone want a database that is 8X slower on average and 70X slower in some cases than it's competitors?

First, we're fixing it. We think we can get Dolt to 2-4X slower on these benchmarks by the end of the year. Second, the sysbench benchmarks test really basic query performance like select a range or insert a single value. Traditional RDBMSs are really fast at these operations, so fast that being 8X slower may not even be noticeable for your application. Lastly, and maybe most importantly, Dolt performs better when compared to other database technologies that are not RDBMSs in real world use cases.

In this blog entry, we'll explore two of our customers' use cases to exhibit how Dolt performs in real world applications.

Nautobot

Nautobot is an open source network configuration manager. We worked with the maintainer, Network To Code, to abstract away the database layer to allow Nautobot to work with Postgres, MySQL, or Dolt. With the Dolt version, you can enable branches, diffs, and pull requests to manage your network configuration like code. Nautobot shows off the power of adding Git-like versioning features to your application using Dolt.

It also allows for a side-by-side performance comparison of a MySQL versus Dolt backed web application.

MySQL is on the left and Dolt is on the right.

MySQL is a hair faster. But, for most pages, Dolt and MySQL backed Nautobot instances have similar performance at the application layer. Feel free to try out both a Nautobot demo instance backed by Dolt and backed by MySQL for yourself.

For the Dolt one, the credentials are username: nautobot, password: nautobot. This server will eventually go away. We spun it up for this blog. If the link no longer works, feel free to follow the README to set up your own. There will be a Dolt backed Nautobot demo instance soon.

For the MySQL one, credentials are in the banner. That is Network To Code's demo instance. It will remain operational.

The performance difference is minimal and the extra version control features are awesome. Seems like a good trade.

Turbine

Turbine.ai uses Dolt to master their genomic cancer data. Previously, they used MongoDB to master this data. As part of migrating to Dolt, they run a parallel MySQL instance. Turbine gave us the mission to be no slower than 20X MySQL on any query. We have hit this goal so far.

Recently, Turbine migrated the read path and shared the following performance results with us, comparing Dolt, MySQL, and MongoDB for their use case. These are large scale analytical queries with the numbers in the headers representing the number of dimensions in their queries.

        10Clx100drug    100Clx10drug    100Clx200drug
mysql        0:00:59         0:01:12          0:30:20
dolt         0:02:55         0:04:31          0:40:29
mongodb      0:11:41         0:36:42          0:56:40

As you can see, for these large scale analytical queries, Dolt is between 50% and 400% slower than MySQL, converging on parity as the queries get larger. Not bad, certainly outperforming the 8X on average we report for the sysbench test suite.

Somewhat surprisingly, Dolt is up to 8X faster than MongoDB for this use case. Benchmarking Dolt against MongoDB is not easy because Dolt and MongoDB have entirely different storage and query syntax. Any attempt at an apples-to-apples benchmark would be unfair. However, based on this example and other Turbine anecdotes, Dolt and MongoDB have at least comparable query performance. Performance has certainly not stopped MongoDB from becoming the 5th most popular database as ranked by DB engines. Maybe query performance shouldn't stop you from trying Dolt?

Conclusion

Dolt is slower than MySQL. We're making it faster expecting to land at 2-4X slower than MySQL on sysbench tests.

For real world applications, results may vary. Often, at the application layer, database performance differences are negligible. Moreover, compared against NoSQL databases like MongoDB, Dolt can be surprisingly fast for real world use cases.

Performance should not be a barrier for you to try Dolt for your use case. Want to try it out? Dolt is free and open source. Need help getting started? Come chat with us on our Discord.