QLDB Deprecated. Looking for an Alternative Immutable Database?
Recently, AWS announced they were no longer accepting new Quantum Ledger Database (QLDB) users and the service would be deprecated in July 2025.
What are you supposed to do if you are a QLDB user? AWS recommends migrating to Postgres and maintaining a history table, slowly changing dimension-style. Postgres is not an immutable database. This is not a great alternative.
What if you liked the immutable qualities of QLDB? You want your database, not your application, to maintain history. What are your options? In 2022, we published a popular blog called So you want and Immutable Database? highlighting three open source alternatives to QLDB, ImmuDB, TerminusDB, and Dolt.
As maintainers of Dolt we're biased, but we think Dolt is the best alternative to QLDB in the immutable database category. Dolt uses a Git-style model to provide immutability and version control. Honestly, Dolt was always more ergonomic than QLDB because the database and the audit log were accessible via SQL. In QLDB, the audit log was offline and would be dumped to an S3 bucket upon request. Plus, Dolt is free and open source. QLDB was...not...and now it's deprecated.
This blog will rehash and update the popular So you want and Immutable Database? blog. We think QLDB users and the QLDB-curious will find a welcome home with Dolt.
What is an Immutable database?
QLDB is in the broad category of immutable databases. In my original blog post, I make the distinction between ledgers, blockchains, and version controlled databases.
All immutable databases are ledgers. The database maintains history to achieve immutability, usually by some sort of copy-on-write mechanism. Ledgers are centralized. There is one authoritative copy. You must trust the writers to your database because there is only one copy. QLDB was a ledger.
Blockchains and version controlled databases are decentralized ledgers. There can be many copies of the ledger and deciding which copy is authoritative is up to the users of the system. In blockchain, deciding which copy is authoritative is defined by the consensus function of the system. In version controlled databases, users of the system use merges and human review to build the authoritative copy.
What was QLDB used for?
Immutable database are useful for data that you want to secure and audit. QLDB did a good job laying out the high level use cases for ledgers. Note the focus on security and auditability.
These use cases are still valid and can be served by any of the alternatives presented later in this blog.
Why not just use Postgres?
AWS suggests migrating to Postgres with a history table. This gives you the same functionality as an immutable database but now your application must enforce immutability, not the database itself.
Anyone who has run databases knows this is not a solution that provides immutability. One bad administrator query and your history is ruined. The reason to choose immutable databases is because you want the database to ensure data cannot be changed or altered. Immutable databases are more resilient to bad operator queries because the database itself is responsible for maintaining history, not the application or operator. Moreover, most immutable databases simplify the application logic needed to maintain immutability, providing a better interface for engineers to integrate with. Moving to Postgres with a history table means abandoning immutable databases as a concept.
Alternatives
If you are using QLDB or were hoping to use it, you are not out of luck. There are a few open source immutable database alternatives. We focus on open source because relying on closed source got you in trouble with QLDB in the first place! It's gone and you don't have the option to maintain it yourself.
We break these alternatives down into the two categories described above: ledgers and version controlled databases. Ledgers are the closest functionality-wise to QLDB. Version controlled databases can be used as ledgers but offer additional features, like branch and merge.
We ignore blockchains because QLDB has little in common with Blockchain technology use cases. The overhead of running your own blockchain cluster or using a public blockchain like Ethereum seems like overkill for most QLDB use cases.
Ledgers
Immudb
- Tagline
- Open Source Immutable Database
- Initial Release
- May 2020
- GitHub
- https://github.com/codenotary/immudb/
ImmuDB is an open source immutable database written in Golang. It can be run client-server or embedded. ImmuDB uses a subset of the PostgreSQL SQL dialect and also supports key/value, like QLDB.
Accessing history is done through a set of history functions. In SQL, there is some custom syntax to access history.
Additionally, ImmuDB has a separate auditor to ensure your database has not been tampered with. This added layer of security would be useful in some use cases.
ImmuDB is a great, open source alternative to Amazon QLDB. ImmuDB has a similar feature set and the migration from QLDB to the ImmuDB key/value variant looks straightforward on the surface.
Version Controlled Databases
Terminus DB
- Tagline
- Making Data Collaboration Easy
- Initial Release
- October 2019
- GitHub
- https://github.com/terminusdb/terminusdb
TerminusDB is an immutable graph or document database with full schema and data versioning capability. The graph database interface is Web Object Query Language (WOQL) or GraphQL as of TerminusDB 11. TerminusDB also has the option to query JSON directly, similar to MongoDB, giving users a more document database style interface.
The versioning syntax is exposed via TerminusDB Console or a command line interface. The versioning metaphors are similar to Git. You branch, push, and pull. See their how to documentation for more information.
Like ImmuDb, TerminusDB looks like a fairly straightforward migration from QLDB. The document based database syntax seems like a natural fit.
Dolt
- Tagline
- Git for Data
- Initial Release
- August 2019
- GitHub
- https://github.com/dolthub/dolt
Dolt is an immutable SQL database that leans heavily on Git version control metaphors and syntax. Dolt implements Git-style version control on table rows instead of files. Data and schema are modified in the working set using SQL. When you want to permanently store a version of the working set, you make a commit.
Dolt implements Git read operations (ie. diff, log) as system tables and write operations (ie. commit, merge) as procedures. Dolt supports AS OF syntax using commits, making history queries user-friendly. Dolt produces cell-wise diffs and merges, making data debugging between versions tractable. That makes Dolt the only SQL database on the market that has branches and merges.
To migrate from QLDB to Dolt, you would dump the audit log, convert it to SQL, and then replay it from the beginning making a Dolt commit after each entry. This would be much the same process as AWS advertises migrating to Postgres but Dolt is immutable and Postgres is not. It seems like that is why you opted for QLDB in the first place.
We want to write a how-to guide about migrating from QLDB to Dolt but we need a QLDB audit log. If you have one you don't mind making public, please stop by our Discord. We can't make our own because QLDB is not accepting new customers but we are working with AWS.
Conclusion
Dolt is a great alternative to AWS QLDB, now that QLDB is deprecated. There are other alternatives like ImmuDB or TerminusDB. Want to discuss immutable databases? Come by our Discord and we'd be happy to chat.