Dolt is Git for data and DoltHub is our web application that houses Dolt repositories. We use Cypress.io as our end-to-end testing solution for DoltHub. To learn more about our journey with Cypress, check out our previous blogs:

• Why we chose Cypress, setting up and writing tests, some examples and downsides
• Making our Cypress tests open source and some gotchas we found converting to Typescript

We made our Cypress tests open source in August and you can check out the tests we've written so far in our dolthub-cypress repository on GitHub. As of a few days ago, we only had tests for our public-facing pages.

It’s our goal to get full Cypress test coverage of all our DoltHub pages. We were inspired by how successful our Bats tests were for testing the Dolt command line and we chose Cypress in hopes of having an equivalent for DoltHub. Our Cypress tests now support DoltHub login, which not only allows us to test our login workflow, but also to test our pages that require authentication.

Cypress has an awesome Chrome UI where you can watch your tests run against your web page. You can see a user successfully being logged in as our tests are running:

A comparison of two login options

The Cypress docs have a few recipes for various login options. Having the ability to log in a user with Cypress accomplishes two goals:

1. Test the login workflow on our signin page
2. Test other pages that require a logged in user like our profile page

I decided to compare using an API request with using our HTML form to log in a user. Using the HTML form directly to test the login workflow on the signin page (goal #1) was obviously going to be the way to go. The point of using Cypress is to test how a user would interact with our website.

However, was navigating to the /signin page and logging in using the HTML form necessary for testing every single page that requires a logged in user (goal #2)? This is where logging in using our API could come in handy.

Logging in using our API

Using our API to log in seemed like faster and more efficient way to test private paths. We have a few endpoints exposed for our DoltHub SQL API, so I decided to temporarily create and test out a /login endpoint.

While this required a little more time to get working, it was very fast to run. This is what that code ended up looking like:

function loginUsingAPI(username: string, password: string) {
  // Send request to the DoltHub API login endpoint
  cy.request({
    url: "/api/v1alpha1/login",
    body: { username, password },
  }).then((res) => {
    // If successful, check to make sure usernames match
    expect(res.body.username).to.eq(username);
    // Set the cookie value for dolthubToken
    cy.setCookie("dolthubToken", res.body.cookie_value);
  });

  // Assert login successful by checking for existence of cookie
  cy.getCookie("dolthubToken").should("exist");
}

In the tests for private paths, instead of visiting the /signin page and then routing to whatever next page we want to test, we can log in using a request to the API and only route once to the page we want.

Logging in using a HTML form

When we initially wrote our Cypress tests, we only supported login using Google authentication. We had to manually set the appropriate cookies, which was slightly tricky and less representative of how a real user would log in. A few months ago we released login with username and password, allowing users to enter their information in a form.

This was pretty straightforward and quick to get working. Here’s what that code looks like:

function loginUsingForm(username: string, password: string) {
  // Visit signin page on macbook 15
  cy.visit("/signin");
  cy.visitViewport("macbook-15"); // `visitViewport` is a custom command

  // Our login form is in a modal, so we need to click on the button to open the modal first
  cy.get("[data-cy=signin-button]").click();
  cy.get("[data-cy=signin-email-form]").should("be.visible");

  // Enter username and password in form inputs
  cy.get("input[name=username]").type(username);
  cy.get("input[name=password]").type(password).type("{enter}"); // '{enter}' submits the form

  // Ensure login is successful:
  // Successfully route to `/profile` path
  cy.location("pathname").should("include", "/profile");
  // Ensure user avatar is visible in navbar
  cy.get("[data-cy=navbar-menu-avatar]").should("be.visible");
}

When you run the tests you'll see the username and password input in the form, submitted using the enter key, and routed to the /profile page with the user avatar in the navbar.

And the winner is...

After some discussion with teammates, we decided to do all logging in for both goals using the HTML form. Although logging in using the API was faster, we wanted to recreate the user workflow as much as possible, and using the API was less representative of how a user would interact with our website.

Once a user is logged in, we can't forget to log out after running the tests for the page. This ensures the correct state for the tests for next pages, so we added an option to log out a user as well:

function logout() {
  // Click on user avatar in navbar
  cy.get("[data-cy=navbar-menu-avatar]").click();
  // Click on signout button in opened navbar menu
  cy.get("[data-cy=sign-out-button]").click();
}

We ended up making custom Cypress commands for both the log in and out functions. You can check them out here.

Using secrets with GitHub Actions

Before I could even start testing login using Cypress, I had to create a test account on DoltHub. It's important that we don't change the state of this account in order to maintain consistency in our tests. We run our Cypress tests on a schedule using GitHub Actions, so we could easily add the username and password of our newly created test account as secrets on GitHub.

I added the newly created secrets as env variables to our GitHub workflow file like this:

env:
  CYPRESS_TEST_USERNAME: ${{ secrets.CYPRESS_TEST_USERNAME }}
  CYPRESS_TEST_PASSWORD: ${{ secrets.CYPRESS_TEST_PASSWORD }}

GitHub does not allow you to log secrets and secrets are not used in a pull request workflows if opened by a non-collaborator. However, Cypress logs some information when a test fails so we turned off the logs for the tests involving the username and password to be safe:

// Enter username and password in inputs
cy.get("input[name=username]").type(username, { log: false });
// '{enter}' submits the form
cy.get("input[name=password]").type(password, { log: false }).type("{enter}");

If a dev on our team wants to test the login form or run our tests for private paths locally, they can add the username and password to a Cypress env file, which we include in our .gitignore file to avoid checking in our secrets.

Conclusion

Going back to the video at the beginning, you can see the final product for what testing a private path now looks like. Before the tests are run for the /profile/discover page we're routed first to the /signin page. The username and password are entered in the form and submitted and we're successfully taken the user's profile. After the tests for the discover profile page are run, we click on the avatar in the upper right corner to log the user out.

We’re continually adding more Cypress tests for DoltHub. If you ever come across an bug on DoltHub, feel free to file a dolthub-cypress issue so we can write a test to make sure it never happens again. You can also chat with us about any issues, questions, or ideas you may have on our Discord server.