We’re one week away from Doltgres 1.0, launching August 6th. The Doltgres momentum has been strong! Over the past three weeks, we’ve merged 50 pull requests, and landed 136 commits from 9 different contributors against the doltgresql repo. Three weeks ago we announced that Doltgres hit 99% compliance on our SQL Logic Test suite, checking off a major correctness target for 1.0. In this post we look at what’s been happening in Doltgres, and how it maps back to the launch goals we laid out in the 1.0 pre-announcement: correctness, storage stability, performance, and compatibility, plus a few other key features we called out as needing to land before launch.
1.0 Progress: Remotes, Replication, and Garbage Collection#
In the 1.0 pre-announcement, we called out three features we still needed to finish before 1.0: pushing and pulling with remotes, the Dolt replication protocol, and automatic garbage collection. We’ve made progress on all three since our last update and are happy to report that these three features are all available in Doltgres now.
Remotes. Remotes let you push and pull data between Doltgres databases, the same distributed workflow Git users are familiar with when they clone, push, and pull. PR #2906 added over 1,000 lines of new test coverage for dolt_remote() operations, the stored procedure interface for configuring and managing remotes. Doltgres directly uses the support from Dolt for working with remotes, but additional testing and a few small code fixes were needed to ensure Doltgres specific entities, like sequences and user-defined types, could be pushed and pulled correctly via remotes.
The following example shows how you can create a file system remote and push to it, then clone that remote in a separate Doltgres server. From there, the two servers can continue synchronizing via the file system remote, by explicitly pushing and pulling changes.
-- On server A: create a custom type and a sequence, use them, then push
CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
CREATE SEQUENCE counter START 100 INCREMENT 50;
CREATE TABLE items (id INT PRIMARY KEY, label TEXT, feeling mood);
INSERT INTO items VALUES (nextval('counter'), 'apple', 'happy'); -- id 100
SELECT dolt_commit('-Am', 'seed items and counter');
SELECT dolt_remote('add', 'origin', 'file:///remotes/items');
SELECT dolt_push('origin', 'main');
-- On server B: an entirely separate process with its own data directory
SELECT dolt_clone('file:///remotes/items', 'cloned');
\c cloned
SELECT id, feeling FROM items;
-- id | feeling
-- -----+---------
-- 100 | happy
-- The sequence's current value came along too, it doesn't reset to its start value
SELECT nextval('counter');
-- nextval
-- ---------
-- 150There are more features for remotes that we want to bring to Doltgres, most notably support for the remotes API that lets you authenticate against a running Doltgres server and push and pull changes directly from it, without having to go through a file system as a middleman.
Replication. Doltgres supports a variety of replication protocols: remote-based replication, cluster replication, and PostgreSQL replication. Remote-based replication and cluster replication are custom Dolt replication protocols. They work off of the database’s commit graph, which makes it very efficient to determine the data that needs to be synchronized from a source to a replica. We’re happy to announce that after a lot of testing, we’re confident that replication is now ready for customers to use in production with Doltgres. We generally recommend customers start with remote-based replication, because it is the simplest option and works well for most customers. If a customer is building a high-availability system, then cluster replication is the right choice, since it gives you a hot standby that is ready to flip over to primary if the primary fails. If you need to replicate from a PostgreSQL server to a Doltgres replica, then using Doltgres’ support for PostgreSQL replication is the right choice.
Let’s take a closer look at an example using remote-based replication. When data is written to the source, the source pushes that update to the remote. On the replica, when data is read, it will first check with the remote to make sure it’s synchronized, then pull any data it’s missing before running your query. The following example shows setting up replication between two Doltgres servers, using a file-system remote as the intermediary for the replicated data.
You’ll need one Doltgres server running as the primary where you configure the remote, set dolt_replicate_to_remote, and create some sample data:
-- On the primary: every commit pushes to a remote automatically
SELECT dolt_remote('add', 'origin', 'file:///var/share/myremote/');
ALTER SYSTEM SET dolt_replicate_to_remote = 'origin';
CREATE TABLE test (pk INT PRIMARY KEY, c1 INT);
INSERT INTO test VALUES (0, 0);
SELECT dolt_commit('-Am', 'trigger replication'); -- pushes to origin as part of the commitOn a second server, clone from the remote to get the initial database contents, then configure a few settings for replication, and restart:
-- On a read replica: clone once, then pull the latest before every read
SELECT dolt_clone('file:///var/share/myremote/', 'read_replica');
ALTER SYSTEM SET dolt_replicate_heads = 'main';
ALTER SYSTEM SET dolt_read_replica_remote = 'origin';
-- (restart the server for these settings to take effect)Once you’ve restarted the replica Doltgres server, it will query the remote to check for additional data to synchronize before it runs the queries sent to it. This ensures that the replica has the latest data from the primary, so it can execute your queries correctly.
Automatic Garbage Collection. Doltgres has supported manually invoking garbage collection via the dolt_gc() stored procedure for a while now, but we didn’t want to launch 1.0 without automatic garbage collection turned on. Automatic Garbage Collection launched for Dolt in 2025 and runs garbage collection automatically, in the background. Before automatic garbage collection, customers needed to manually invoke dolt_gc() to prevent their database files from getting too large on disk. Many customers set up a cron job to run this cleanup command once a week or so. Automatic garbage collection has been enabled by default in Dolt for about a year, and now customers never need to worry about manually running garbage collection.
Automatic garbage collection runs opportunistically in the background rather than on a fixed schedule. There’s no query or system table that tells you garbage collection just ran, so the most direct way to see the results of garbage collection is to run queries that generate garbage, then watch the data directory’s size shrink over time. If you want to see this in action, you can follow the example below.
Start a Doltgres server, then open up a terminal where you can watch the size of Doltgres’ data directory (postgres is the default database name Doltgres creates, but you can change this if you’re using a database with a different name):
watch -n 1 'du -sh ~/doltgres/databases/postgres/.dolt/noms'In a separate psql session connected to your Doltgres server, generate some garbage: insert a big enough batch of rows to push the store past auto GC’s internal 128MB size threshold, then delete half of them, so the deleted rows’ old versions stick around in Dolt’s history as reclaimable garbage.
CREATE TABLE vals (id INT PRIMARY KEY, v FLOAT, payload TEXT);
INSERT INTO vals
SELECT g1, random(), md5(random()::text) || md5(random()::text) || md5(random()::text) || md5(random()::text)
FROM generate_series(1, 1000000) g(g1);
SELECT dolt_commit('-Am', 'insert batch');
DELETE FROM vals WHERE id % 2 = 0;
SELECT dolt_commit('-Am', 'delete half the batch');One million rows with a chunk of random-looking text in each one is comfortably past that 128MB threshold, so this should reliably trigger garbage collection to run. In my testing, I saw the directory size get up to 240MB, then quickly drop back down to 142MB. You may need to watch the terminal closely, since garbage collection can run very quickly after all the inserts, giving you a short window to see the peak in disk usage before it drops down again.
Correctness#
Our ultimate goal for correctness is that any query on Doltgres returns the exact same results as running the same query on Postgres. We hit our 99% goal on the SQL logic test suite, and we’ve been bashing as many bugs and gaps as we can find. We’re very grateful for the customers who have submited issues on GitHub to let us know about correctness issues they’ve found. Just like with Dolt, we push ourselves to get customer-reported bugs knocked out in 24 hours.
One of the more interesting bugs we fixed recently is how Doltgres infers the type of a “bind variable,” the placeholder values a client sends separately from the SQL text itself when using the Postgres extended query protocol (i.e. how most drivers and ORMs send parameterized queries). Clients can tell the server “I don’t know the type of this parameter yet, you figure it out,” and Doltgres is supposed to infer a type for that one bind variable from context. PR #2975 found that if even a single bind variable in a query was left unspecified, Doltgres was discarding the type information the client did provide for every other bind variable in that same query, and re-inferring all of them from scratch. When that inference landed on a different type than the client was actually using, Doltgres could misread how many bytes it needed to consume off the wire for that value, which could lead to writing the wrong data into a column. That kind of bug is easy to miss in testing (most queries either fully specify their types or fully omit them) but shows up in the wild once real client libraries and ORMs start mixing the two. The fix scopes the inference down to just the bind variables that actually need it, and the PR also made the wire-reading code fail loudly instead of silently if it ever encounters a similar mismatch in the future.
Performance#
We’ve been making steady gains on the performance track, too. So much so, that we upgraded our performance goal to be even more aggressive. Our original goal was to get Doltgres performance under a 3x multiplier of Postgres’ performance, as measured through a series of sysbench tests. We hit that goal and are now pushing towards a 2.6x multiplier, which will put Doltgres on par with MySQL’s perf multiplier of Postgres’ performance. These performance increases have come from a variety of small PRs focused on reducing unnecessary computation, removing excess memory usage, and enabling the query engine to make smarter decisions about join order and index selection.
On Track for August 6th#
We’re on track for launching Doltgres-1.0 next week, on August 6th. We’ve been building Doltgres since September, 2023. It’s taken us just shy of three years to get Doltgres up to a production-ready, 1.0 milestone and we’re proud of where we are. Our 1.0 release signals to customers that Doltgres is ready to handle their production loads, and that our team is ready to support those customers in production. There are of course still gaps in features and SQL syntax that we’ll continue to fill in, based on what customers tell us is most important for them. We’re also anticipating an even bigger push on query engine performance to continue making Dolt and Doltgres faster.
If you want to help us improve Doltgres, the best thing you can do is install it ( through brew install doltgres on a Mac, or by downloading from our GitHub releases), take it for a test drive, and report any issues or gaps you hit. If there is SQL syntax missing that your app needs to use, let us know so we can scope out the work and deliver it for you. If you run into anything where Doltgres behavior doesn’t match Postgres behavrior, let us know and we’ll be happy to dig in and understand what’s going on.
Finally, if you just want to chat about databases, Postgres, or version control, come by our Discord and say hello.
See you back here next Thursday for the Doltgres 1.0 launch announcement!