Standard Dolt Deployments
Here at DoltHub, we ship a number of products. Dolt is an Online Transaction Processing (OLTP) database with Git-style version control, like Git and MySQL had a baby. DoltHub is Dolt's GitHub, an online collaboration platform for Dolt databases, complete with Pull Requests on your data. DoltLab is a self-hosted version of DoltHub for collaboration in your internal network. Hosted Dolt is AWS for Dolt. We'll deploy and manage a Dolt database for you.
We also have Doltgres, a Postgres flavored version of Dolt and the Dolt Workbench. These two products won't make much of an appearance in this article but they exist.
All these products can be used individually or together in interesting ways. This article explains the common ways Dolt is deployed, including some very helpful diagrams.
MySQL Inspired
Dolt is, first and foremost, a production-grade SQL database. Dolt is MySQL-compatible and can be deployed anywhere you would deploy MySQL. In fact, if you never use any of the Dolt features like commits or branches, Dolt will work exactly like MySQL. Thus, our first category of deployments is MySQL-inspired.
On Premises
Dolt is free and open source. You can download it and self-host.
In the above deployment, Dolt replicas are pictured. You can have zero to many Dolt replicas using Dolt replication. Dolt replicas add read scalability and redundancy just like replicas for MySQL or Postgres.
Use Cases
In this deployment, Dolt is used as a MySQL replacement. Customers build applications powered by Dolt. Customers use Dolt version control features for improved disaster recovery and developer productivity. Or, Dolt features like branches and merges can be exposed in your application, making adding version control features to your application easy.
This deployment is commonly used to build an internal data or configuration development application. Data is migrated from complex file-based (JSON, YAML) configuration in Git to SQL while retaining Git features. Or, the data exists in Mongo, MySQL, or Postgres and the customer wants to add version control features at the database layer using Dolt.
Threekit, Turbine, Scorewarrior, FJA, and Nautobot all use flavors of this deployment in production.
Pros
- Version control features on your OLTP database
- Full control
- Data never leaves your network
- Free with paid support options
Cons
- Need to maintain replication, failover, backups yourself.
- ~1.5X slower than MySQL
Hosted Dolt
Hosted Dolt gives you the on-premises deployment experience but we do it for you in AWS or Google Cloud. You can configure zero to many replicas. Every Hosted Dolt database gets a public IP address and cname
for the primary and replicas.
Use Cases
Hosted Dolt can be used to build applications in the same way as an on premises deployment. Teams that prefer cloud databases like AWS RDS over on premises deployments use Hosted Dolt.
We've found customers that use Dolt for Machine Learning data management tend to prefer Hosted Dolt over on on premises deployment. Flock Safety uses Hosted Dolt to store and version the data used in their vision models.
Pros
- Version control features on your OLTP database
- Fully Hosted. Dolt engineers handle database uptime and maintenance.
- Built-in SQL Workbench complete with Pull Requests
Cons
- Database is in the cloud accessible via the Internet
- ~1.5X slower than MySQL
- Paid service
MySQL with a Dolt Replica
Dolt can act as a MySQL replica. Dolt can consume a standard MySQL binlog
. In this deployment, you run a standard MySQL instance and add Dolt as a replica.
Use Cases
In this deployment, Dolt gives you improved disaster recovery on a MySQL deployment. Your MySQL gets a full, queryable audit log with diffs of every change. You can use this in a disaster scenario to find and undo any bad SQL query.
We have a handful of customers running this deployment but none have allowed us to discuss it publicly yet.
Pros
- Queryable audit log of every change to your MySQL primary
- Run MySQL in production. No change to your MySQL architecture.
- Purely additive. No downside risk.
Cons
- Incomplete Dolt feature set. No branches and merges on your primary.
Dolt with a MySQL Replica
Just recently, we released Dolt to MySQL replication. A single Dolt branch will produce a binlog
and this can be consumed by MySQL to make a replica.
Use Cases
This deployment gives you MySQL read performance on a single branch and allows you to integrate Dolt with tools that use a binlog
, like Debezium.
This is a customer requested feature for integration with Debezium to do change data capture. That customer is actively testing the solution as I write this.
Pros
- Version control features on your OLTP database
- Full control
- Data never leaves your network
- MySQL read performance if you need it
Cons
- Need to maintain replication, failover, backups yourself.
- Complexity. You run an additional MySQL and must route to it.
Git Inspired
Dolt is not only a MySQL-compatible database. Dolt also implements the Git model of version control. This is on full display in the Dolt command line, where all the commands you know from Git work the same way in Dolt.
This means you can interact with a Dolt remote like DoltHub or DoltLab via clone, push, pull, and fetch, inspiring some Git-style deployments.
Decentralized Development
This deployment is where we started. In this deployment, you and collaborators from around the world clone databases from DoltHub to your local machine, make changes, push those changes to DoltHub, and review using Pull Requests. This gives you a decentralized database collaboration workflow.
Use Cases
This deployment was designed for data sharing and collaboration, specifically open data. The best exhibition of DoltHub's capabilities in this regard are post-no-preference
US stock market databases, used by hundreds of people around the world.
DoltHub also features private repositories for data collaboration within companies. A number of companies use DoltHub to replace a very complicated, highly collaborative spreadsheet.
Pros
- Decentralized database development
- Pull Requests on your database
- Fully Hosted. No maintenance.
- Free for public databases
Cons
- Can't connect with a MySQL client. Programmatically accessible via low-scale API or webhooks.
- Paid service for private databases
Internal Data Collaboration
If you want a DoltHub-style collaboration experience but don't want data to leave your network, there is DoltLab. DoltLab has a very similar user interface and feature set to DoltHub.
Use Cases
DoltLab is used for internal data sharing and collaboration much like DoltHub private databases. KAPSARC uses Dolt and DoltLab in this way.
Pros
- Decentralized database development
- Pull Requests on your database
- Data never leaves your network
- Paid enterprise features like HTTPS, email support, Single Sign On
Cons
- Self-deployed. Must maintain yourself.
- Can't connect with a MySQL client. Programmatically accessible via low-scale API or webhooks.
Hybrid
Of course, you can take advantage of both Dolt's MySQL and Git features in the same deployment. Here are some examples of hybrid Dolt deployments.
On Premises with DoltLab Dev Environment
You can add a DoltLab remote to your on premises deployment. Your DoltLab deployment can act as a development environment for your databases, using push or pull to deploy to your running Dolt primary instance.
Use Cases
This deployment gives you change isolation via DoltLab as your development environment. You also can have very strict permissions on your primary Dolt instance while allowing less restrictive access to your DoltLab instance.
Many Dolt deployments start as on premises deployments and realize later that a DoltLab instance would be useful. We recommend starting with a Dolt on premises deployment and evaluating whether DoltLab is a good fit after you get that deployment working.
Pros
- Version control features on your OLTP database
- Full control
- Data never leaves your network
- Decentralized database development
- Pull Requests on your database
Cons
- Need to maintain DoltLab, replication, failover, backups yourself.
- ~1.5X slower than MySQL
Hosted as Production
This is the exact same deployment as above but Hosted by DoltHub. Hosted Dolt provides the same features as an on premises deployment and private DoltHub databases provide the same functionality as DoltLab.
Use Cases
The use case is very similar to the above deployment but you don't have to maintain it yourself. Hosted Dolt customers opt for this deployment when they realize DoltHub would be useful.
Pros
- Version control features on your OLTP database
- Fully Hosted. Dolt engineers handle database uptime and maintenance.
- Built-in SQL Workbench complete with Pull Requests
- Decentralized database development
Cons
- Database is in the cloud accessible via the Internet
- ~1.5X slower than MySQL
- Paid service
Hosted with Development Clones
Hosted Dolt allows you to open a remote endpoint and use that to clone, push, pull and fetch. This removes DoltHub from the above topology. However, all users need super user permissions to push.
Use Cases
The use case is very similar to the two previous deployments. DoltLab and DoltHub are removed making the deployment simpler. But this deployment requires all users who want to push be granted super user permissions.
Pros
- Version control features on your OLTP database
- Fully Hosted. Dolt engineers handle database uptime and maintenance.
- Built-in SQL Workbench complete with Pull Requests
- Decentralized database development
- Simpler than adding DoltHub or DoltLab
Cons
- Database is in the cloud accessible via the Internet
- ~1.5X slower than MySQL
- Paid service
- Super User permissions required for all people who push to the server.
Conclusion
As you can see, there are many ways to deploy Dolt, Hosted Dolt, DoltHub, DoltLab and MySQL in concert. Hopefully, this article helps you to find the Dolt deployment that fits your needs. Need help choosing? Stop by our Discord and we'll be happy to help.