Migrating from Jenkins to Github Actions
Dolt is a SQL database with Git-style versioning. DoltHub is the place on the internet to share Dolt databases. For both Dolt and DoltHub, we've always used Jenkins for our continuous integration pipeline but have recently migrated our Dolt repository from Jenkins to Github Actions.
"GitHub Actions help you automate your software development workflows in the same place you store code and collaborate on pull requests and issues. You can write individual tasks, called actions, and combine them to create a custom workflow. Workflows are custom automated processes that you can set up in your repository to build, test, package, release, or deploy any code project on GitHub.
With GitHub Actions you can build end-to-end continuous integration (CI) and continuous deployment (CD) capabilities directly in your repository. GitHub Actions powers GitHub's built-in continuous integration service."
As part of Dolt's continuous integration (CI) process, we run checks against pull requests for the repository's format and history, we run Dolt's Golang and BATS test suites, and we test Dolt's forward and backward compatibility by creating and reading from repositories using older versions of Dolt.
We've been running this setup with Jenkins successfully for well over a year now, but our team's motivation for transitioning to Github Actions really centered around the out-of-the-box, first class features provided by Github Actions that we feel are very attractive for open source projects.
For example, one challenge we faced with our previous CI integration was enabling community contributors who were not a part of our company to investigate why a pull request's CI step might have failed. For our team, a failing CI step can be inspected directly from the Jenkins logs in our private cloud. Configuring Jenkins to allow community contributors read-only access to certain parts of Jenkins is certainly possible, but there are non-trivial costs in doing so.
Enter Github Actions. Github Actions enables us to avoid this problem and others like it, all together. Since the workflows live in the repository itself, and all Github Action logs are now featured on Github's UI under the Actions tab, our community contributors get instant access to Dolt's CI for free. There's no credentialing or permissions configuration, it just works. We consider this a huge win!
So what was our migration experience actually like? In a word, pleasant. Getting up and running with actions is really easy. If you're working from the shell, it takes only a minimal yaml file located in a Github repository's .github/workflows directory, followed by a push to see your workflows in action. Github also enables workflow creation right on the web.
For us, running Dolt's Golang test suite on Linux, Mac, and Windows required the following lines of yaml checked in at .github/workflows/:
name: Test Go
on:
pull_request:
branches: [ master ]
jobs:
test:
name: Go tests
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
steps:
- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: ^1.13
id: go
- uses: actions/checkout@v2
- name: Test All
working-directory: ./go
run: go test ./...That's it. Like, seriously, that's it. A small configuration file; instant payoff. That's one of the features that make Github Actions so compelling. The simplicity involved in setup and use is great, not to mention how much of the task definitions or actions are abstracted away in favor of a clean yaml interface.
From the snippet above, both actions actions/setup-go@v2 and actions/checkout@v2 are publically available and can be added as steps to run in a workflows file. In the same way that Dockerhub hosts a plethora of docker images that abstract away the complexities of setting up a particular container environment, Github has an Actions community where users share useful, public actions. If you can't find an action from the community that meets your use case, you can also define custom actions and release them yourself should you choose to.
It's also very convenient to run Dolt's Golang test suite on Linux, Mac, and Windows only required including those virtual environments, or what Github calls Runners, in the jobs.runs_on.matrix list. There was no need for us to define an image or custom runner the way we had to with Jenkins, which provides full CI control with a lot of configuration. Github Actions provides the option to have less CI control with less config, which is actually perfect for our use case. Just look at this snippet from our old Jenkinsfile that defined how Dolt's BATS tests should run on Linux and Windows:
stage ("BATS-LINUX") {
agent {
kubernetes {
label "dolthub-ld-build"
}
}
environment {
PATH = "${pwd()}/.ci_bin/pyenv/bin:${pwd()}/.ci_bin:${pwd()}/.ci_bin/node_modules/.bin:${env.PATH}"
AWS_SDK_LOAD_CONFIG = "1"
AWS_REGION = "us-west-2"
DOLT_BATS_AWS_TABLE = "dolt-ci-bats-manifests-us-west-2"
DOLT_BATS_AWS_BUCKET = "dolt-ci-bats-chunks-us-west-2"
DOLT_BATS_AWS_EXISTING_REPO = "aws_remote_bats_tests"
}
steps {
dir (".ci_bin") {
sh "npm i bats"
}
dir ("go") {
sh "go get -mod=readonly ./..."
sh "go build -mod=readonly -o ../.ci_bin/dolt ./cmd/dolt/."
sh "go build -mod=readonly -o ../.ci_bin/git-dolt ./cmd/git-dolt/."
sh "go build -mod=readonly -o ../.ci_bin/git-dolt-smudge ./cmd/git-dolt-smudge/."
sh "go build -mod=readonly -o ../.ci_bin/remotesrv ./utils/remotesrv/."
}
sh "python3 -m venv .ci_bin/pyenv"
sh "./.ci_bin/pyenv/bin/pip install doltpy"
sh "dolt config --global --add user.name 'DoltHub Jenkins'"
sh "dolt config --global --add user.email 'jenkins@dolthub.com'"
dir ("bats") {
sh "bats ."
}
}
}
stage ("BATS-WINDOWS") {
agent {
label "windows"
}
environment {
PATH = "C:\\tools\\msys64\\mingw64\\bin;${pwd()}\\.ci_bin;${env.PATH}"
}
steps {
dir ("go/") {
bat "go test ./..."
bat "go build -mod=readonly -o ..\\.ci_bin\\dolt.exe ./cmd/dolt/."
bat "copy /Y ..\\.ci_bin\\dolt.exe ..\\.ci_bin\\dolt"
bat "go build -mod=readonly -o ..\\.ci_bin\\git-dolt.exe ./cmd/git-dolt/."
bat "copy /Y ..\\.ci_bin\\git-dolt.exe ..\\.ci_bin\\git-dolt"
bat "go build -mod=readonly -o ..\\.ci_bin\\git-dolt-smudge.exe ./cmd/git-dolt-smudge/."
bat "copy /Y ..\\.ci_bin\\git-dolt-smudge.exe ..\\.ci_bin\\git-dolt-smudge"
bat "go build -mod=readonly -o ..\\.ci_bin\\remotesrv.exe ./utils/remotesrv/."
bat "copy /Y ..\\.ci_bin\\remotesrv.exe ..\\.ci_bin\\remotesrv"
}
dir ("bats/") {
bat "dolt config --global --add user.name \"DoltHub Jenkins\""
bat "dolt config --global --add user.email \"jenkins@dolthub.com\""
bat "C:\\wsl.exe bats `pwd`"
}
dir ("./") {
bat(returnStatus: true, script: "setLocal EnableDelayedExpansion && pushd %LOCALAPPDATA%\\Temp && del /q/f/s .\\* >nul 2>&1 && rmdir /s/q . >nul 2>&1 && popd")
bat(returnStatus: true, script: "setLocal EnableDelayedExpansion && pushd C:\\batstmp && del /q/f/s .\\* >nul 2>&1 && rmdir /s/q . >nul 2>&1 && popd")
}
}
}In the above snippet you can see that both stages do the same thing, run Dolt's BATS tests in different environments, but there is a lot of custom configuration required in order to get this running successfully. Not only do we need to duplicate the steps required to run the tests on each agent, we also have to define and maintain all of the additional resources required to just get these stages running, including custom Docker images for the Jenkins server and Linux agent, a custom Windows ec2 instance to serve as our Windows agent, and all of the accompanying configuration required to deploy these within our private cloud.
Needless to say it's a lot of configuration just to simply run some tests. And while some use cases require full control of all aspects of CI, our current use case does not. As a result, we've really enjoyed that Github Actions provides us the choice over how much or how little control we have over our pipeline.
In contrast to the complicated setup with Jenkins, here's how we run Dolt's BATS tests with Github Actions:
jobs:
test:
name: Bats tests
runs-on: ${{ matrix.os }}
strategy:
fail-fast: true
matrix:
os: [ ubuntu-latest, macos-latest ]
steps:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }}
role-duration-seconds: 3600
- name: Setup Go 1.x
uses: actions/setup-go@v2
with:
go-version: ^1.13
id: go
- name: Setup Python 3.x
uses: actions/setup-python@v2
with:
python-version: ^3.6
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
with:
node-version: ^12
- name: Create CI Bin
run: |
mkdir -p ./.ci_bin
echo "::add-path::$(pwd)/.ci_bin"
- name: Install Bats
run: |
npm i bats
echo "::add-path::$(pwd)/node_modules/.bin"
working-directory: ./.ci_bin
- name: Install Doltpy
run: |
python3 -m pip install --upgrade pip
pip install doltpy
- name: Install Dolt
working-directory: ./go
run: |
go build -mod=readonly -o ../.ci_bin/dolt ./cmd/dolt/.
go build -mod=readonly -o ../.ci_bin/git-dolt ./cmd/git-dolt/.
go build -mod=readonly -o ../.ci_bin/git-dolt-smudge ./cmd/git-dolt-smudge/.
go build -mod=readonly -o ../.ci_bin/remotesrv ./utils/remotesrv/.
- name: Setup Dolt Config
run: |
dolt config --global --add user.name 'DoltHub Actions'
dolt config --global --add user.email 'actions@dolthub.com'
- name: Install expect
if: matrix.os == 'ubuntu-latest'
run: sudo apt-get install -y expect
- name: Check expect
run: expect -v
- name: Test all
run: |
bats --tap .
working-directory: ./bats
env:
AWS_SDK_LOAD_CONFIG: "1"
AWS_REGION: "us-west-2"
DOLT_BATS_AWS_TABLE: "dolt-ci-bats-manifests-us-west-2"
DOLT_BATS_AWS_BUCKET: "dolt-ci-bats-chunks-us-west-2"
DOLT_BATS_AWS_EXISTING_REPO: "aws_remote_bats_tests"To be fair, this snippet does not yet include a Windows runner, which will add some additional complexity to our workflow file. But with that in mind, this above snippet demonstrates how powerful and convenient Github Actions really is. The above snippet is all that's required to run Dolt's BATS tests on the Unix runners. There's no duplication of CI steps, no Docker images or cloud resources to define and maintain; just one yaml file!
It is this simplicity and well thought-out design pattern that really drew us to Github Actions, and we are excited to explore more features in the months to come.
We also need to give Github Actions a special shoutout for their documentation. We found it to be very well organized and thorough, even providing useful guides for the migration process. Here's the guide we referenced when migrating from Jenkins. This particular guide even shows how to install BATS on a runner in an example yaml snippet!
Conclusion
If you're unsure about whether Github Actions is right for your team, we hope we've inspired you to try it out. For open source project it provides an easy way to get started with CI and makes it easy for devs to ignore some of the complexities involved in orchestrating a robust CI pipeline. That's why we chose to make the switch, and have been very pleased with the results. If you'd like to see all of our CI workflows, you can check them out here.
Currently, we continue to run Dolt's BATS test suite on Windows through Jenkins. One challenge we've encountered with Github Actions is the hosted Windows runner's lack of wsl support, which we need for running BATS. We hope to overcome this by running our own in-house Windows runner, but will set that up at a later date.
Curious about Dolt, DoltHub and the versioned data format of the future? There's no better place to get started than Dolthub.com where you can download Dolt, host your own public and private repositories, or just clone some amazing public repositories you won't find anywhere else, including some interesting and up-to-date COVID-19 data.
Questions, comments, or looking to start publishing your data in Dolt? Get in touch with our team here!