Earlier this week we released DoltLab v2.3.0. This release signifies the end of our months-long endeavor to make the first-time DoltLab user experience as simple and straightforward as possible. In this release, we've removed TLS configuration in DoltLab's free edition and have made this an exclusive DoltLab Enterprise feature. Additionally, DoltLab's installer tool now supports automatic upgrades for DoltLab Enterprise instances as well. This makes upgrading to the latest DoltLab Enterprise version trivial.

These two changes follow a number of related changes we've been steadily releasing since the start of 2024.

If you've been following along, you'll recall that we revamped DoltLab's configuration interface by creating the installer for simple, consistent instance configuration, we removed the requirement to connect DoltLab to an existing SMTP server which made getting a fully operational DoltLab instance slow to setup, and we have finalized the feature set of DoltLab free edition versus that of DoltLab Enterprise edition.

Here's a summary of the current feature set of each edition of DoltLab.

As you can see, each product offers a range of features geared toward different users.

The free edition on the left is designed to support customers looking to set up small-scale, trial instances for three to four individual users. These users are looking for quick and easy deployments with minimal configuration. Typically these users are prospective DoltLab enterprise clients looking to use DoltLab for their company's particular use cases, but want to start evaluating the product's core features as soon as possible.

Enterprise edition users are different. They are organizations with medium to large teams deploying production DoltLab instances. These users require much more configuration and customization options, have specific security and performance requirements, and regularly require custom features, shipped quickly.

Oftentimes enterprise edition users started as free edition users, but have since upgraded their edition to start running DoltLab at scale.

As I mentioned earlier, the feature sets shown above have only recently been solidified, culminating in the release of DoltLab v2.3.0.

For this reason, we thought it would be a cool idea to explore the history of DoltLab's different editions, and trace their evolution from DoltLab's humble origins as a one-to-one, self-hosted DoltHub.com clone, to the two-headed database beast that it is today.

A history of features

The diagram above shows the feature set evolution of DoltLab free edition on the left and that of DoltLab Enterprise on the right. The light green features in each list show the currently available features of each product. The dark green features in the DoltLab Enterprise list show exclusive features not available in DoltLab's free edition. And, the gray features in the DoltLab free edition list show features that we've migrated to be exclusive DoltLab Enterprise features.

Since DoltLab's inception, we've intended to have two distinct DoltLab editions—one free and one paid.

However, as you can see from the diagram, we've had some feature overlap between these editions over the years. It's taken some time (until this year, really) to better separate each product's feature set, and in large part this is because DoltLab shares the same source code as DoltHub.com.

This reality means that feature changes and separation of concerns is always a bit more challenging navigate with DoltLab, more so than it is with products that have their own dedicated code base.

In fact, what you may not realize is that when you're running a DoltLab instance, you're actually running a DoltHub instance, in DoltLab mode.

But sharing the same code does not make these products identical.

Though they consist of the same suite of services, each product runs in very different contexts, is operated by very different parties, and each has very different technical and business requirements to consider.

Making a DoltHub clone

When we set out to create DoltLab in 2022, our goal was to make a version of DoltHub that user's could host themselves. In the early stages of development, although we had a vision for what the future product(s) of DoltLab might look like, our first step was just getting a one-to-one DoltHub clone released to our users.

To do this as quickly as possible we decided against hard-forking our DoltHub source code, and instead decided to update DoltHub's service code to operate in multiple runtime modes: one mode for DoltHub, and one for DoltLab.

DoltLab v0.1.0

+ Public databases

+ Diff viewer

+ Pull requests

+ Database collaborators

+ Spreadsheet editor

+ SQL editor

+ Issues

+ Releases

+ Organizations and teams

DoltLab launched with version v0.1.0 in 2022, as the self-hosted version of DoltHub.com. At launch, DoltLab included many of the core features of DoltHub.com. These included the ability to host public Dolt databases, easily view data diffs, modify databases using a pull-request style workflow, collaborate with other users on changes, edit database data using a spreadsheet, edit database data using SQL, create GitHub-style issues, create database tags and releases, and create and manage organizations and teams.

One side-effect of DoltHub and DoltLab sharing the same source code was that early on in DoltLab's development, most DoltHub features, by default, would automatically become DoltLab features.

If at some point we did not want this to be the case, each DoltHub feature would need to be updated to account for how it should work (or not work) in DoltLab too.

But, to keep things simple, we decided to put off updating every DoltHub feature and just included it in DoltLab, if it worked as intended. At the time, another top priority was ensuring we were in fact making DoltLab as close to a one-to-one clone of DoltHub as possible, since we knew prospective DoltLab customers were looking for a true self-hosted DoltHub experience.

While the features above worked in DoltLab v0.1.0 out-of-the-box, a number of other/standard DoltHub features unfortunately did not. So we decided to triage these and focus on adding support for them first.

DoltLab v0.2.0

+ Automated Emails

+ SQL API

In DoltLab's next minor release, we added support for automated emails and made DoltLab's SQL API functional to match the behavior of DoltHub.

Automated emails enabled a DoltLab v0.2.0 instance to send emails using a third-party SMTP server. DoltLab sends emails to users for various reasons including account verification, password resetting, and inviting others to collaborate on databases.

DoltHub does this as well.

In fact, at the time of this release, DoltHub required a connection to an SMTP server for some of its most basic features to work. Without it, DoltHub would be unusable. It would have been a large lift to remove this dependency on a third-party SMTP server, so we opted to just make connecting to an SMTP server a requirement of DoltLab also.

This enabled functional automated emails on DoltLab v0.2.0, which was good, but it greatly complicated DoltLab's configuration, which was not ideal.

Version v0.2.0 also included a working SQL API. Like on DoltHub, this feature allows users to perform actions on their databases by making API calls. We had a number of customers using this feature on DoltHub , so it was important that our new product also included this functionality.

DoltLab v0.5.0

+ Private databases

+ CSV downloads

A short time later, we shipped v0.5.0 with support for private databases and CSV downloads.

New users interested in DoltLab often requested that private databases be supported, since earlier DoltLab versions only supported public databases. A private database, in contrast to a public one, can only be read and written to by users with explicit permission to do so. These permissions are managed through the UI of DoltHub and DoltLab. You can think of private databases as equivalent to private repositories on GitHub.com.

On DoltHub at this time, private databases could only be created by users or organizations who provided their billing information. DoltHub charges monthly for data stored across a user's or organization's various private databases.

In the DoltHub code base, there was a lot of business logic surrounding how private databases should work. Their amount of data stored in private databases is constantly measured and users are billed for storage.

To support private databases in DoltLab, which was going to be free, we needed to change this logic so that private databases work correctly in both contexts, something we've had to do with many of the features both DoltHub and DoltLab support.

Version v0.5.0 also fixed CSV downloads for DoltLab which until this point did not work. This feature allows users to execute a SELECT query in the DoltLab SQL editor and download the resulting rows as a CSV file.

DoltLab v0.5.8

+ Database webhooks

Continuing on our initial trajectory of making DoltLab support the same features as DoltHub, we released v0.5.8 which enabled database webhooks on DoltLab.

This feature allows users to register an endpoint for receiving webhook requests sent by their DoltLab instance. A webhook request is sent whenever an event occurs against a particular database. These events might be pushes to a branch, or a new pull-request being opened or merged. DoltHub supports webhooks as well, so it was crucial we supported this early on in DoltLab too.

DoltLab v0.7.0

+ File import Job

The release of v0.7.0 came shortly after we released a brand new feature on DoltHub called Jobs.

DoltHub Jobs were out-of-band, asynchronous processes that could perform large, long-running operations on DoltHub databases. These operations that we named "Jobs" were things like large file imports into a database, or merging large pull-requests into a database branch.

Of course, prior to the Jobs implementation, DoltHub supported file imports and pull-request merges, but limited this work to the resources available in DoltHub's main API.

Jobs removed this limitation altogether by moving this work out of the main API and onto dedicated hosts that would do the work and report back asynchronously. Once we had this working for DoltHub, we did the same for DoltLab.

The first Job supported by DoltLab was the file import Job. This allows DoltLab users to upload a CSV or SQL file of any size directly into their DoltLab database.

DoltLab v0.7.1

+ Pull request merge Job

Similarly, in v0.7.1, we released a DoltLab version that supported pull-request merge Jobs to match DoltHub. This allows large merges to run asynchronously and reports the status of the running merge on the database Jobs page. Again, this was a crucial feature to make DoltLab a true clone of DoltHub.

Enterprise is born

At this stage in DoltLab's development the product had great parity with DoltHub's feature set, and we no longer needed to prioritize adding legacy features or fixing bugs that made DoltLab work like DoltHub. Instead, around this time, we had new enterprise customers that required specific features for their DoltLab instances, so we pivoted our focus to start building those out.

One thing to note, though, is that we did not release an official, separate enterprise edition of DoltLab, even though the features we were building would eventually be exclusive enterprise features.

Creating an official DoltLab Enterprise edition required a non-trivial amount of engineering work that would further splinter the DoltHub/DoltLab code base, so we opted to delay this work and began releasing enterprise features in the free versions of DoltLab.

And if you know us, you know we can never keep customers waiting!

DoltLab v0.7.6

+ Custom logo

+ Custom email templates

DoltLab v0.7.6 supported the first two requests by our new enterprise customers that focused on customization options: allowing the DoltLab logo displayed in the UI to be customized, and allowing the content of the automated emails to be customized.

Not only do these customization options provide a consistent branding for a DoltLab instance within an organization, it also ensures that users of the instance know they aren't using a spoofed DoltLab site.

DoltLab v0.8.1

+ Custom colors

The option to customize the colors used across a DoltLab instance followed soon after, as enterprise branding wouldn't be complete without this option. This feature required a revamp of how styles were used across DoltHub and DoltLab altogether.

Prior to this customizable implementation, many colors and styles for DoltHub and DoltLab were hard-coded. With this DoltLab release, these styles became dynamically generated and this feature actually improved many styling options for DoltHub, which was a pretty cool side-effect.

DoltLab v1.0.1

+ Super admins

Soon after the release of DoltLab v1.0.0 which discontinued the use of PostgreSQL as its application database in favor of using Dolt v1.0.0, one enterprise customer requested the option to designate "super admins" on their DoltLab instance.

A super admin is a user who has unlimited access and the highest level of permissions on every database, organization, and team across a DoltLab instance. This is useful for DoltLab administrators to audit how users are using their instance.

DoltHub had no concept of a super admin, or any user with these types of permissions, and this is still true today. This feature is exclusive to DoltLab, and really only makes sense in that context.

DoltLab v1.0.2

+ Database forks

Though DoltLab shared almost every legacy feature with DoltHub up to this point, one final feature of DoltHub that had not yet been supported on DoltLab was database forks. There were some technical challenges to overcome that prevented us from making forks work correctly in the DoltLab in the beginning, so we'd punted on doing any of that work.

However, we had no choice but to overcome those challenges in preparation for v1.0.2 since an enterprise customer said they actually required database forks for their use case.

A database fork is a copy of a database hosted on DoltLab that you own.

Most often, you will only have read permissions on a database and only the database owner can grant you permission to write to it. By forking the database instead, you get your own copy of the database and become the owner of that copy.

After forking, you can write to your copy freely and even submit pull requests back to the original database if you want changes in your copy to be merged upstream.

DoltLab v1.0.6

+ Native TLS support (HTTPS)

As DoltLab users increased we began receiving more requests from free edition and enterprise edition users alike to support TLS. Prior to v1.0.6 DoltLab could only be served over http, which is not secure. In this release, we made it possible to easily serve DoltLab over https by supplying TLS certificates to DoltLab's start-up scripts.

This change had the biggest practical impact on free edition users who typically deployed their instances on the public internet, but enterprise users welcomed this feature as well. Although DoltLab Enterprise most often runs on internal networks which are not discoverable externally, oftentimes organizations impose a security requirement that all internal services be served over https regardless.

DoltLab v1.1.0

+ Two-factor authentication

The next DoltLab release saw another boost to security features as well. We had just added two-factor authentication to DoltHub, so it followed naturally that this should be added to DoltLab too.

Two-factor authentication is enabled per user account, and if configured on a DoltLab instance, will send a one-time code to the email address associated with a user account. This code is then used to sign-in to the DoltLab instance, ensuring the real user is the one signing in.

DoltLab v2.0.0

− Custom logo

− Custom email templates

− Custom colors

− Super admins

+ SAML single-sign-on

Fast forward to DoltLab v2.0.0. Prior to this release DoltLab Enterprise edition was not an officially distinct edition of DoltLab. We had been building and releasing specific features for our enterprise customers, but we made these features part of the DoltLab free edition. Well, v2.0.0 changed this. We had finally reached the point where making a distinct, official enterprise product became our top priority.

To build this official product we redesigned DoltLab so that it supported two distinct run-time modes: a default, free edition mode and an enterprise edition mode. We also integrated DoltLab Enterprise with a third-party licensing service so that license keys would be required by users running DoltLab in enterprise mode.

Additionally, in version v2.0.0 we began the process of migrating enterprise exclusive features from DoltLab's free edition to its enterprise edition. We started this migration process with the customization features added in DoltLab in v0.7.6 and ended with the super admin feature added in DoltLab v1.0.1.

But we didn't just remove features in this release.

We also gave our enterprise customers a brand new feature a number of them were excited about.

We'd just completed supporting SAML single-sign-on in DoltHub and on Hosted Dolt, so in v2.0.0 we made it available in DoltLab Enterprise, as well. This was a great security feature to add to DoltLab, but as the configuration options for DoltLab increased, so did the deployment complexity.

DoltLab v2.0.3

+ Automated remote backups

Adding to DoltLab's growing deployment and configuration complexity was another enterprise exclusive feature added in v2.0.3, automated remote backups. An automatic DoltLab backup is a scheduled backup of the Dolt application database running as part of a DoltLab stack. When configured, a DoltLab service will regularly create backups of the running database and push the backup data to a supported cloud storage bucket. If needed, the database can be restored from these backups at any time.

Our motivation for adding this feature to DoltLab Enterprise came after talking with a customer about the challenges they experienced building scheduled DoltLab backups themselves. Their preference was to have this supported as a first-class feature of DoltLab Enterprise, which would greatly simplify things for them.

DoltLab v2.0.5

+ Large query Job

In v2.0.5 we got back to adding features in both DoltLab free edition and DoltLab Enterprise edition by supporting large query Jobs. Previously in both DoltLab editions and on DoltHub, users could execute SELECT queries against their databases using the SQL editor. However, these queries limited results to, at most, 1000 rows. These limits prevent browser degradation, which can result from loading too much data into memory, ruining the end-user experience.

But, it is very useful for users to be able to execute SELECT queries that return unlimited results by simply using DoltHub and DoltLab's sleek UI. So to support this, we created large query Jobs for DoltHub and both DoltLab editions.

Like the file import Job and pull-request merge Job before it, this Job executes a user's SELECT query against a database and stores the results of the query as a CSV file. This file can then be downloaded at any time by the user, giving them more query flexibility whenever they need it.

Streamlining for the future

With an official DoltLab enterprise edition fully released, it was finally time to shift our focus to streamlining DoltLab and DoltLab Enterprise. Up to this point, each DoltLab release shipped with a number of configuration files, one-off binaries, and various templates—some of which needed to be manually edited and maintained by DoltLab administrators.

Oftentimes administrators would need to keep notes on how they've configured their DoltLab instance and what manual changes they'd made, so that when they were ready to upgrade to a newer DoltLab version, they could repeat the steps they'd taken before.

We knew this needed to change and be greatly simplified to get DoltLab ready for the next level. Even future features we planned to implement would require a simpler, less manual DoltLab deployment process, so we shifted our focus to improve DoltLab's configuration system and first-time user experience.

DoltLab v2.1.0

+ Installer interface

The release of DoltLab v2.1.0 introduced the new interface that would greatly improve DoltLab for all users. In this release, the installer tool debuted, replacing all of the hard-coded configuration files and templates previously shipped in earlier DoltLab versions.

Instead of static files, this tool would deterministically generate all of the assets DoltLab required. This meant that DoltLab administrators didn't need to track any changes they'd made to a DoltLab deployment. Instead, they only needed to run the installer with the same arguments they'd used before and they'd generate an identical DoltLab deployment.

This was a huge improvement to the overall DoltLab experience and made it easier to write the guide we published around this time about how to get started with DoltLab quickly.

DoltLab v2.1.1

+ Multi-host deployments

The installer interface for DoltLab configuration also allowed us to ship our next DoltLab Enterprise feature, multi-host deployments. Until this release, DoltLab Enterprise could only be run on a single host. Like DoltLab free edition, enterprise edition runs all of its services on one host via Docker Compose, and uses the host's disk for persisting data.

Starting with v2.1.1, enterprise deployments could optionally run across a multi-host architecture by deploying each of DoltLab's services to a dedicated host, which is similar to the deployment architecture of DoltHub.com. To do this, DoltLab administrators can download DoltLab on separate hosts and use the installer to generate the necessary assets.

This feature enables horizontal scaling of DoltLab's services which can use smaller/cheaper hosts per service, instead of a single, larger, more expensive host required by the traditional DoltLab deployment.

DoltLab v2.1.4

+ Installer config file

After using the installer in practice, it soon became clear to us that while it was indeed a huge improvement for DoltLab, generally, it needed some quality-of-life improvements itself to make it simpler to use.

In v2.1.4 we updated the installer to use a single configuration file, installer_config.yaml. This file could be shared across DoltLab upgrades, and truly made deploying a DoltLab instance easier than it has ever been.

With the new installer_config.yaml file, DoltLab admins could deploy a new instance from start to finish in under 30 minutes!

DoltLab v2.1.6

+ Cloud storage

Another ask from our enterprise customers had been to support cloud storage options. Previously, all data on a DoltLab instance was persisted to the host's local disk. While convenient, this limited the amount of data a DoltLab instance could store to the size of the host's disk, and could result in data loss if the disk wasn't properly and regularly backed up.

For this reason v2.1.6 introduced the option to use cloud storage to persist DoltLab data. To enable this feature, administrators only need to provide the location of a supported cloud resource where they want their DoltLab data persisted. Then, assuming their DoltLab instance has the correct permissions to access the given cloud resource(s), all writes will be persisted there, instead of the host disk.

DoltLab v2.2.0

− Organizations and teams

− Automated emails

− Two-factor authentication

+ User badges

We then returned to the task of migrating exclusive enterprise features out of the free edition of DoltLab. In v2.2.0, organizations and teams became exclusive to DoltLab Enterprise, as the feature was designed to be used primarily by organizations. In this release, automated emails and two-factor authentication were made exclusive to DoltLab Enterprise as well.

Automated emails were migrated, partly so that an SMTP server connection would no longer be a requirement to get the full functionality of DoltLab's free edition. In prior versions of DoltLab, user's were unable to create accounts if an SMTP server had not been properly connected. Now, starting with v2.2.0 all features of DoltLab free edition worked out-of-the-box, without need for additional, more cumbersome configuration.

This change also meant two-factor authentication was migrated to DoltLab Enterprise exclusively as well. Since the free edition no longer sent emails to users, it could not send two-factor sign-in codes to users.

Following in DoltHub's footsteps, user badges were added to both DoltLab free edition and DoltLab Enterprise edition.

This feature rewards users of your instance with cool, unique achievement icons for performing various tasks across the site. This is a fun way to encourage engagement from your users and drive a little friendly competition.

DoltLab v2.3.0

− Native TLS support (HTTPS)

+ Automatic upgrades

And finally, we arrive at DoltLab's latest release, the one that puts a bow on our migration of exclusive DoltLab Enterprise features. In this new release we've made native TLS support exclusive to DoltLab Enterprise as well.

Our vision for DoltLab free edition is to support smaller-scale, evaluation based use-cases of DoltLab. On the other hand, we want DoltLab Enterprise to provide secure, production ready deployments fit for teams of any size. So considering this, we further pruned our free edition's configurations options.

One of the benefits of having so many products though, is that teams looking for a secure, production-ready Dolt remote, but don't want to deploy a DoltLab Enterprise instance, can simply use DoltHub.com. It provides all of the same functionality and guarantees as DoltLab Enterprise, without the need to host anything yourself.

Version v2.3.0 also introduces another cool quality of life feature for our DoltLab Enterprise customers: automatic upgrades. As I mentioned before, upgrading a DoltLab instance was not always easy. Though the upgrade process did improve greatly with the introduction of the installer and installer_config.yaml, we still wanted to make upgrades easier for people.

Now with the v2.3.0 installer, the --upgrade flag can be used to download the latest DoltLab version. The installer will upgrade your installation and automatically migrate your existing configuration file so it can be used with the newer version. All you need to do is rerun the installer once the upgrade completes, and you'll have the most up-to-date DoltLab version available.

Conclusion

We hope you found this history of DoltLab's features interesting and we encourage you to give DoltLab a shot. As you've seen from the details of all the feature changes, DoltLab's free edition is super easy to stand up and use, now more so than ever before! Simply follow the steps in our Getting started documentation and you'll be running in under 30 minutes.

Alternatively, if you want the full enterprise experience, we'd be happy to provide you with a free trial. Just pop by our Discord and ask! We'd love to chat with you and learn more about how you want to use DoltLab.

Thanks for reading and don't forget to check out each of our cool products below:

• Dolt—it's Git for data.
• DoltHub—it's GitHub for data.
• DoltLab—it's GitLab for data.
• Hosted Dolt—it's RDS for Dolt databases.