cypress-integration-tests.md

Cypress integration testing

Background

Cypress is a browser-automation testing suite that we use for end-to-end tests. See ADR019 - Integration Tests for some additional background on testing goals and decisions.

CI/CD Pipeline

All tests added into the tdrs-frontend/cypress/e2e/ folder will be run against the newly deployed develop environment as part of our pipeline to help ensure site reliability and that no bugs have been introduced. These are run against the actual deployed environment and so might catch issues connecting with real backend services instead of having everything local.

Running tests

1. Have both the backend and frontend running in separate terminal processes, the app needs to be reachable and usable at localhost:3000 when testing locally
2. In a new terminal, set up test users by running

   cd tdrs-backend
   docker-compose exec web python manage.py generate_cypress_users

3. Be sure your tdrs-backend/.env file contains the following

   # testing
   CYPRESS_TOKEN=local-cypress-token
   
   DJANGO_DEBUG=yes

4. In a new terminal, run the following commands to launch the Cypress runner

   cd tdrs-frontend
   npm run test:e2e

5. Select "E2E Testing" from the testing type menu
6. Select a browser and click "Start E2E Testing"
7. Now you can select a spec file from the menu and watch the tests run

Using the Cypress runner

It is highly recommended that you check out the Cypress Test Runner feature overview. Here are the main highlights

• Command log - Cypress will show you each cy command that executes in the command log, alongside the rendered page.
  • You can hover over past commands to "time travel" and replay individual steps, and clicking on a command in the log will "pin" the point in time in the runner, and log output information about that command in the console.
  • The command log also shows network requests and other dispatched events.
• Dev tools - you can open up chrome's devtools (right click and click "Inspect") and see all the normal DOM information about the page, including for pinned commands which is useful when debugging. The console will also show output from the page as well as the tests.
• Automatic rerun - whenever you save your test code or frontend app code, Cypress automatically reruns the open test.

Writing tests

On top of Cypress, we've layered cypress-cucumber-preprocessor to provide Gherkin syntax for tests. This gives the tests more structure and allows us to easily separate and organize step implementations.

Test files are defined as .feature files within the tdrs-frontend/cypress/e2e directory. Feature "areas" can be grouped into a folder.

Step implementations are defined as .js. The cucumber preprocessor uses the glob pattern cypress/e2e/*/[filepath].js to discover the step files at the same level as and within the feature directory so long as the step file has the same name as the .feature file. The cucumber preprocessor uses the glob pattern cypress/e2e/common-steps/*.js to identify it's common step file(s) which are available to all .feature files.

Here's an example feature file

tdrs-frontend/cypress/e2e/accounts/accounts.feature

Feature: Users can create and manage their accounts
    Scenario: A user can log in and request access
        When '[email protected]' visits the home page
        And '[email protected]' logs in
        Then {string} sees a Request Access form

At its top level, it defines a Feature with multiple Scenarios (this can be likened to a describe and it in jest, respectively). Scenarios belonging to a feature area should be grouped within a Feature. Scenarios should describe a specific task/goal the user is required to perform in the system.

Each line within a Scenario is a "step", which should describe the user interaction with the system in the business context. There are three types of steps, Given, When, and Then

• Given steps define prerequisite state of the system in order for the tests to run. Mostly commonly, we'll use a Given I am logged in as xyz step to perform user login and setup prior to the test steps actually executing.
• When steps define user actions needed to perform the task.
• Then steps describe validation or confirmationt that the user receives, indicating success of the task.

In general, all Given, When, and Then steps should reflect things the user of the system knows about the system. This is intended to hide both technical and administrator implemenation details and focus on the end-user experience.

Each step defined in a Scenario must have a corresponding "step implementation" (loaded from a .js file). Here is an example step implementation file

tdrs-frontend/cypress/e2e/accounts/accounts.js

/* eslint-disable no-undef */
import { When, Then } from '@badeball/cypress-cucumber-preprocessor'

When('{string} visits the home page', (username) => {
  cy.visit('/')
  cy.contains('Sign into TANF Data Portal', { timeout: 30000 })
})

When('{string} logs in', (username) => {
  cy.login(username)
})

Then('{string} sees a Request Access form', (username) => {
  cy.contains('Welcome').should('exist')
  cy.get('button').contains('Request Access').should('exist')
})

For each step implementation, a good rule of thumb is to perform both an action and an assertion. An action should be something the user can do in the system (click, type, etc.). Assertions help "slow down" the test and limit unexpected behavior when applications run a lot of asynchronous processes. By asserting on something verifiable in each step, we can ensure the test is in a proper state to move forward. This applies to Given, When, and Then steps (though Then steps can often omit an action).

Shared step implementations, which apply to all feature files, can be added as common step definitions in tdrs-frontend/cypress/e2e/common-steps/common-steps.js