Doltgres is a Postgres-compatible, version-controlled SQL database. To ensure Doltgres behaves like Postgres, we've started running it through Postgres Regression Test suite. This process performs correctness testing for Postgres-specific features. In this post, we’ll explain Doltgres' current performance, identify what's still broken, and describe how we're using these results to prevent regressions.

What's Postgres regression tests?

Postgres regression tests are a set of automated tests included with the PostgreSQL source code. They ensure the correctness and stability of the database system by checking that various features and behaviors of PostgreSQL work as expected and continue to do so after changes are made to the codebase. Because these tests are used to Postgres itself, the types of queries and functionality they exercise is much more complete than any tests we have for Dolt. Getting to 100% on these tests would be a huge accomplishment for Doltgres.

How Did Doltgres Do?

When we initially ran these tests back in November 2024, Doltgres passed 33.8% of them. Today, that number has increased to 39.38%. We've been making steady progress since then, primarily by focusing on implementing new features rather than solely optimizing for these specific tests. The initial increase to 39.07% was mainly due to adding more Postgres features to Doltgres, which allowed many tests to run successfully instead of failing with syntax errors. A month ago, we began actively working on these tests to boost the pass rate as much as possible. As of today, we stand at 39.38%. This represents a very small increment (+0.31% to be exact) over a month-long period.

That's... not great. But it’s a starting point. Every failing test highlights an edge case where Doltgres doesn’t yet match Postgres, presenting another opportunity for improvement.

What’s Broken?

The majority of failed tests fall into the unsupported category. Many of these involve features that are not widely used, such as cursors and publications. Additionally, many Postgres-specific built-in types, along with functions and operators that use those types, are not yet supported in Doltgres.

• We first analyzed test results by grouping and sorting them by the most common error messages returned from queries.

The top 10 most frequent error messages using the GROUP BY result clause are:

+--------------------------------------------------------------------------------+-----+
| result                                                                         | ct  |
+--------------------------------------------------------------------------------+-----+
| PARTITION OF is not yet supported                                              | 885 |
| PSQL command not yet supported                                                 | 386 |
| table function: 'check_estimated_rows' not found (errno 1105) (sqlstate HY000) | 380 |
| SET SESSION AUTHORIZATION is not yet supported                                 | 370 |
| CREATE FUNCTION only supports PL/pgSQL for now                                 | 322 |
| @@ is not yet supported                                                        | 300 |
| function: 'pg_relation_is_publishable' not found (errno 1105) (sqlstate HY000) | 294 |
| at or near "(": syntax error                                                   | 284 |
| at or near "fetch": syntax error                                               | 260 |
| function: 'jsonb_path_query' not found (errno 1105) (sqlstate HY000)           | 239 |
+--------------------------------------------------------------------------------+-----+

For example, if we were to add PARTITION OF syntax support, it would enable 885 tests to pass, representing a 2.1% gain. However, since we don't currently support table partitioning, focusing solely on PARTITION OF syntax is not a high priority. Instead, we prioritize these issues based on factors such as customer requests and feature commonality.

From the sorted list, we began by implementing Set-Returning Function support.

The next issue we addressed was missing timestamp parsing format.

Note that these fixes were not merged into the main branch of Doltgres in this specific order.

The current issue we're working on is supporting the LANGUAGE sql syntax for functions.

• Another approach we used to evaluate failed tests is to look for common error message patterns. There are 3,848 unique

error messages returned from a total of 25,080 tests. Of these unique errors, 3,142 are associated with 5 or fewer failing queries. This is because these error messages are often generated with unique input arguments to provide more specific details about the issue. Therefore, we can use a LIKE clause to identify specific issues, even if the exact error message varies.

For instance, only one query fails with the exact error message: could not parse "2025-06-03 12:00 PDT" as TimeTZ (errno 1105) (sqlstate HY000). However, multiple queries might fail due to the same underlying issue, each returning a slightly different error message based on the input used to generate it. The issue was to support parsing timezone in time values.

Preventing Regressions

To make sure Doltgres doesn't regress as we develop it, we now automatically run this test suite on every pull request (PR). If a PR causes more tests to fail, it gets flagged, meaning it can only be merged once we fix the regressed tests. One exception is when a previously passing test was actually incorrect — in that case, fixing a related issue may reveal a hidden failure.

For example, this happened when we improved type handling in Doltgres: a previously passing query began to fail due to its use of an unsupported function.

This acts as a crucial safety net as we add new features or refactor internal components. It also gives contributors early visibility into whether they've accidentally broken anything. We started by only showing regressions but soon added a progressions section to the test runner as well. As we make changes to Doltgres we can see what tests now pass.

Conclusion

Running Postgres regression tests against Doltgres has helped us pinpoint gaps in compatibility and prioritize meaningful improvements. While a 39.38% pass rate shows we still have a long way to go, each fix gets us closer to full compatibility. By integrating these tests into every PR, we’re ensuring Doltgres moves forward without slipping backward—and making steady, deliberate progress toward a more complete Postgres experience.

As always, if there is any issue you'd like us to fix, report them to us or let us know on Discord. We'll get it resolved as soon as possible.