Last month, I announced that we were finally starting development on DoltgreSQL. In a little over a month, we've made quite a lot of progress and released a new pre-release version. We've also expanded the development team that is working on DoltgreSQL, meaning that updates will continue at an even faster pace!

With that said, DoltgreSQL is an open-source project, and we'd love to welcome new developers as fellow contributors. If you'd like to dive into development, then this blog will help you get set up!

What is DoltgreSQL?

Before we begin, it's worth mentioning exactly what DoltgreSQL is. DoltgreSQL is a PostgreSQL-flavored version of Dolt, except it is being built with the database server experience being prioritized. Well then, what is Dolt? I wrote a bit about it in my last blog, but in short it's a version-controlled database. Imagine that Git and MySQL had a baby. I've seen other companies mention that they're also version controlled, but they'll often refer to the schema only, or they implement some subset of versioning features. Dolt implements all of the features, at scale, on 100GB+ databases that are currently in production. Branching, merging, diffs, etc. on your schema and your data.

And, well, not everyone likes MySQL's syntax, rules, etc., so DoltgreSQL is set to accomplish the same objective as Dolt, except for PostgreSQL's syntax, rules, etc.

Now that you're familiar with DoltgreSQL, let's get your environment set up to become a contributor!

Setting up your environment

1. Install the latest version of Go

DoltgreSQL is built primarily using the Go language. As of the writing of this blog post, we are using Go 1.21, however I highly suggest checking the current go.mod file to see our minimum Go version. It is generally okay to use a higher version as Go is quite good with backwards compatibility, but you must use at least the minimum version. We try to stay up-to-date with the latest major stable Go version, so always check that you're using the minimum.

2. Install Git

DoltgreSQL is hosted on GitHub, so you need a Git client to clone it.

3. Install Bats

We make use of Bats for some of our tests, so it is important to have it installed. Assuming you have NPM installed, installing Bats is as simple as running npm install -g bats.

4. Install PSQL 15

Although we do not use PostgreSQL, we do use the PSQL client (for PostgreSQL 15) for testing. In addition, it is worthwhile to have a local PostgreSQL instance installed to cross-check behavior, especially when writing tests (which should always be verified by a PostgreSQL 15 instance). Installing PostgreSQL 15 is the recommended way to get the PSQL client. If you do not want to download the full PostgreSQL server, then make sure that you download the PSQL client for version 15, as some distributions ship the client for version 14 by default. Eventually we'll also support the version 14 client, but for now we require the version 15 client.

5. Ensure access to a Unix-like environment

Linux and MacOS users may skip this step, as you already have a Unix-like environment by default. For Windows users, we require the use of a Unix-like environment, which means that additional tools are needed. Several full-time DoltgreSQL developers use Windows (including me), and we all prefer a bash terminal to the Command Prompt or Powershell (for development specifically). We have a few recommendations, so feel free to choose one from this non-exhaustive list if you don't already have a preference:

• Git Bash: The Mintty client that may be installed alongside Git.
• WSL 1: The Linux subsystem for Windows, specifically version 1. We've attempted to use version 2 with Dolt, however we ran into a few issues. It's safe to assume that DoltgreSQL would experience the same problems that Dolt did. It's possible that it works, but since we do not test or support it, we heavily suggest sticking to version 1 instead.
• Cygwin: Cygwin is a full Unix-like environment for Windows. Unlike WSL, which runs a full Linux kernel within Windows, Cygwin emulates Unix through native Windows processes. This has not been extensively tested, however some have had success using it for development.

Setting up DoltgreSQL

Now that we have our environment set up, we can download DoltgreSQL to our local machine.

1. Fork the repository

You can fork DoltgreSQL by clicking "Fork" on the DoltgreSQL GitHub page, or by following this link. This will create a copy of the repository within your GitHub account that you are free to modify.

2. Clone the repository

From your terminal, you may run the following command:

git clone https://github.com/USERNAME/doltgresql.git

Replace USERNAME with your own username. This will download a local copy of your forked repository to your computer, and also set your fork as the remote origin. We'll assume that the root of the cloned repository lives in the following directory:

~/doltgresql

3. Build the parser

We cannot build the project just yet, as the parser uses a YACC grammar to parse the SQL statements. We must first build this parser, which you can do so with the following:

$ cd ~/doltgresql/postgres/parser
$ ./build.sh

You must remember to run this build script every time you make a change to the YACC grammar, or whenever you merge the latest changes into your fork. We do not include the generated files as they are guaranteed to cause merge conflicts if two people touch the parser at the same time.

4. Run the Go tests

Before we do our first real build, we should verify that all of the tests pass locally. Our main branch will always have passing tests, so if they fail for you, then there is an issue with your local environment. You can run all of the tests in the repository with the following:

$ cd ~/doltgresql
$ go run test ./... --count=1

The ./... argument states that the tester should look in all subdirectories, and --count=1 ensures that the tester does not use any cached results. Assuming that the tests pass, we're ready to build the project!

5. Build DoltgreSQL

With everything else done, building DoltgreSQL is easy!

$ cd ~/doltgresql
$ go build -o doltgres .

For Windows users, executables generally end with the .exe file extension, so the command would look like:

$ go build -o doltgres.exe .

Feel free to move this to a folder in your PATH, so that you may call doltgres from any directory. If you use go install, and Go's binary directory is in your PATH (typically ~/go/bin), then it will create an executable in the binary directory named doltgresql rather than doltgres. This is due to the project name being DoltgreSQL, which Go automatically uses.

6. Run the Bats tests

With DoltgreSQL built, we now have one final set of tests to ensure that it's all working correctly.

Our Bats tests assume that there's a DoltgreSQL binary in your PATH named doltgresql. For Windows users, depending on the terminal being used, you may need to create a copy of your doltgresql.exe binary that excludes the file extension (same may go for your psql.exe client as well).

To run these tests, you should do the following:

$ cd ~/doltgresql/testing/bats
$ bats .

If all of the tests pass, then congratulations, you've successfully built DoltgreSQL!

Making a Contribution

Now that you have a fully-functioning development environment for DoltgreSQL, let's look at how to contribute! We have a Contribution Guide that goes into way more depth on what we expect from pull requests, so I highly recommend reading through that document to ensure that your pull request is approved. For now, let's look at the high-level workflow.

1. Format your repository

Our CI workflow checks the format of all pull requests, so to ensure that your changes conform to our requirements, you may run the following script:

$ cd ~/doltgresql
$ ./scripts/format_repo.sh

2. Push your changes to your fork

Bundle up your changes, and push them to your fork. We'll assume that you're using the default main branch within your forked repository. You can commit from any directory within the repository.

$ git add -A
$ git commit -m "Short description of changes"
$ git push

3. Create a Pull Request

Navigate to your forked repository on GitHub, and click on the Pull requests tab. On the next page, click on the New pull request button. This should then open a page showing all of the changes that you've made against our main branch (base repository dolthub/doltgresql). You'll then click the Create pull request button, which will give you the ability to add a title and description. Once you're done, click the Create pull request button, and await feedback!

4. Address any feedback

Someone from DoltHub will review your pull request, assuming all of the tests pass. They may directly approve the pull request, or they may assign feedback that you must address. Once all feedback has been properly addressed and a DoltHub employee has signed off on the pull request, we'll merge your PR! Congratulations, you're officially a contributor.

Conclusion

This is a fairly high-level overview of how to become a contributor to the DoltgreSQL project. It has quite a lot of steps, and the Contribution Guide is quite in-depth, but it's all to maintain the highest level of quality possible. Databases are critical pieces of software, so we're very rigorous in our processes!

Even if you're not looking to contribute as a developer, we welcome all issue reports! We'd love to hear your ideas, thoughts, and opinions, so you can find us on Twitter/X and chat with us on Discord. We are very excited for the future of DoltgreSQL, and we hope you'll join us for the ride!