Chore/2141 cypress (#2305)

* add and configure cypress

* move tests

* add cypress authentication override/support

* generate cypress users management cmd

* update test w/ shared steps

* add cypress docs

* commit cypress support

* fix editor noise

* sp

* add cypress circle ci job

* formatting

* format

* format again

* add npm install for cypress

* linter

* override cypress token

* fix var name

* add cypress users

* change local token

* store screenshots/videos as artifacts

* adr title

* update email domain

* broken link

* Update docs/Technical-Documentation/cypress-integration-tests.md

Co-authored-by: Andrew <[email protected]>

* update test name

* update owasp circle token

* try set browser

* oops

* Update docs/Technical-Documentation/Architecture-Decision-Record/019-integration-tests.md

Co-authored-by: Alex P.  <[email protected]>

* Update docs/Technical-Documentation/Architecture-Decision-Record/019-integration-tests.md

Co-authored-by: Alex P.  <[email protected]>

Co-authored-by: Andrew <[email protected]>
Co-authored-by: Alex P <[email protected]>

.circleci/build-and-test/jobs.yml

@@ -41,6 +41,29 @@
       - store_artifacts:
           path: tdrs-frontend/pa11y-screenshots/
 
+  test-e2e:
+    executor: machine-executor
+    working_directory: ~/tdp-apps
+    steps:
+      - checkout
+      - docker-compose-check
+      - docker-compose-up-backend
+      - docker-compose-up-frontend
+      - install-nodejs-machine
+      - disable-npm-audit
+      - install-nodejs-packages:
+          app-dir: tdrs-frontend
+      - run:
+          name: Set up cypress test users
+          command: cd tdrs-backend; docker-compose exec web python manage.py generate_cypress_users
+      - run:
+          name: Run Cypress e2e tests
+          command: cd tdrs-frontend; npm run test:e2e-ci
+      - store_artifacts:
+          path: tdrs-frontend/cypress/screenshots/
+      - store_artifacts:
+          path: tdrs-frontend/cypress/videos/
+
   secrets-check:
     executor: docker-executor
     steps:

.circleci/build-and-test/workflows.yml

@@ -12,3 +12,6 @@
       - test-backend:
           requires:
             - secrets-check
+      - test-e2e:
+          requires:
+            - secrets-check

.gitignore

@@ -3,7 +3,6 @@ tdrs-frontend/node_modules/
 tdrs-frontend/build/
 tdrs-frontend/coverage/
 tdrs-frontend/cypress/videos/
-tdrs-frontend/cypress/support/
 tdrs-frontend/cypress/screenshots/
 tdrs-frontend/cypress/fixtures/example.json
 tdrs-frontend/npm-debug.log*
@@ -105,3 +104,5 @@ tfapply
 *.tfstate
 
 *.~undo-tree~
+
+cypress.env.json

docs/Technical-Documentation/Architecture-Decision-Record/019-integration-tests.md

@@ -0,0 +1,111 @@
+# 19. Integration tests
+
+Date: 2022-12-05
+
+## Status
+
+Accepted
+
+## Context
+
+Deployments are slow and require a lot of manual testing; this can cause quite a backup during PR reviews and releases. Unit tests are valuable for fast feedback close to the code, but we are lacking broader-scale automated testing of features.
+
+### Agile testing quadrants
+
+Resources
+* [Testing Quadrants](http://www.exampler.com/old-blog/2003/08/22/#agile-testing-project-2)
+* [Using the Agile Testing Quadrants](https://lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/)
+
+The Agile Testing Quadrants is a useful way to explore how different types of tests can be used to help achieve different testing goals. In Brian Marick's original post, tests are described as being either **business facing** or **technology facing**.
+
+* _Business-facing_ tests describe functionality in user's own terms, as a set of actions taken in the system to achieve a broader goal for the person or business.
+   * Example: Account holders have to verify their PIN before they're able to submit a transaction.
+* _Technology-facing_ tests describe technical aspects of the system that are relevant to the team that owns it.
+   * Example: Transactions submitted without PIN verification should fail with "Error Code 4".
+
+Both tests can be for the same functionality, but the buiness-facing example implicitly covers more rules (frontend-validation, integration between different system parts) and approaches validation from the perspective of helping the user achieve their end-goal. The technology-facing example, however, is useful in its exact-ness: a test like this can be isolated to a single part of the codebase and provide fast-feedback to developers about potential breaking changes. There's some fuzziness here, of course, as many technical decisions are partly business decisions and vice-versa.
+
+
+Both business-facing and technology-facing tests can be described as having one of two goals: **support programming** or **critique the product**.
+
+* Tests that _support programming_ are used by developers to safeguard their development of the system; they are written and run often during development and help developers identify any unintended side-effects of changes they've made. We can also use these tests to prepare ourselves and test assumptions prior to starting development.
+* Tests that _critique the product_ are used by developers and stakeholders alike to uncover issues that already exist; whether they are issues in planning, technology, design, or development. These tests find bugs and help identify other ways the system can be made better for the end-user.
+
+
+We can consider these ideas as a matrix. The following example from Lisa Crispin shows the quadrants and examples of types of tests applying to each
+
+![Quadrants and test types](http://www.exampler.com/testing-com/blog/agile/test-matrix.jpg)
+
+Tests in the left half of the quadrant are generally valuable to automate - they consider "known" paths for the system and expected behaviors/results. We can codify these and run them every time a change is made to the system in order to give us confidence in releases.
+
+Tests in the right half of the quadrants generally try to seek answers to unknowns about the behavior of the system or expectations of the people using it. We're trying to uncover issues and test our assumptions rather than our implementations, therefore these tests need the power of human subjectivity to be wholly effective. They can be supported by tools, but ultimately are manual tests.
+
+### Types of automated tests
+
+To simplify the above quadrants, we can categorize the "Automated" tests (Q1 and Q2) as
+
+* Unit tests (Q1) - tests that are technology-facing and support programming. For these tests, fast-feedback is key. They should cover isolated cases in individual parts of the system, and mock as much as is reasonable to do so.
+* Integrations Tests (Q2) - tests that are business-facing and support programming. These tests should be automated to provide the fastest feedback possible, but more emphasis should be placed on accurately modeling the user's experience and workflow. Mock as little of the system as possible and perform steps in the system as the user would actually do them, also using indicators within the system to provide results.
+
+As stated before, there's a grey area in this type of testing and tests can be both business- _and_ technology-facing, so consider the quadrants to be starting points and extremes in a spectrum. Ultimately, the most valuable test is the test that verifies as much of the system as possible for the functionality being tested. Also be sure to consider the tradeoff with reliability and speed.
+
+### Tools
+
+* [Selenium](https://www.selenium.dev/) is a browser-automation tool that has been a cornerstone of the automated testing industry for quite some time.
+   * Pro: extensive language support, documentation, and community support.
+   * Con: requires a lot of setup for common tasks, especially for modern applications. `wait`-ing on elements and organizing page objects, selectors, etc
+   * Con: flaky, especially with modern, async frontend applications. very inflexible runtime which is difficult to run in ci
+* [Cypress](https://www.cypress.io/) is another browser-automation tool that seeks to be simpler for developers and address gaps in the capabilities of older tools to test more modern, asynchronous application arhitectures.
+   * Pro: growing community adoption, support for major frameworks and most architectures.
+   * Pro: substantial runtime improvement on Selenium with native support for async frontends, works very well locally and in CI, Cypress dashboard provides test replay analysis
+   * Con: Cypress doesn't support testing more than one "superdomain", which means redirects to Login.gov to authenticate are not supported. Authentication workarounds are generally required.
+
+Pro: Both tools would allow us to use [Cucumber/Gherkin](https://cucumber.io/docs/gherkin/) to write frontend test scenarios in plain-language steps, which document the system behavior in business-language (useful for Q2 tests).
+
+
+
+## Decision
+
+
+### Which type of test?
+
+The team has expressed interest in expanding the integration testing suite. We have a well-rounded unit test suite that addresses most of our quadrant-1 needs, but we're lacking q2 "business scenario" tests that can be run against `develop` or another deployed environment to test our system end-to-end. These types of tests would provide continuous regression testing and improve our release confidence.
+
+We recommend creating a new "end-to-end" integration test suite, which mocks the least amount of the system possible. These tests will perform actions by launching a browser and going through the actual steps a user would use to perform a task in the system.
+
+### Which tool?
+
+Cypress improves the end-to-end testing game a lot by providing a simple interface that requires minimal configuration. Selenium suites require a lot of command-line configuration that Cypress simply removes, allowing us to focus on writing quality tests. We still need to understand some of the basics of testing with a browser automation tool in order to make the most of it, but overall Cypress lowers the barrier to entry for automated testing, makes tests easier to write and debug, and provides a well-documented and well-supported environment requiring minimal configuration. For those reasons, we recommend Cypress.
+
+### When will tests be run?
+
+Tests should be run locally as much as possible during development to ensure changes don't unexpectedly break something else in the system. Tests should at least be run before and during the PR process, so we should add a CircleCI task to run Cypress tests along with the unit tests.
+
+We should also regularly run tests in a deployed environment. In order to best simulate a production deployment, we should automatically deploy to a specific environment after merges to `develop`, after which the end-to-end tests should automaticfally be run. This would require a few more changes to our CircleCI configuration and processes.
+
+### How will failures be responded to?
+
+Failures during development should be addressed by the developer that broke the tests. Tests should be passing in the PR pipeline before a PR is merged, so if a PR fails after a merge, the developer of the PR should look into the failure. With Cypress, failures due to test flakiness should be minimized, but are still possible. Depending on risk and priority, tests can be rerun or work prioritized to fix/rearchitect the test or feature to improve reliability.
+
+### How will we plan tests into the scope of upcoming work?
+
+Tests for new work should be planned into the scope of that work before it is assigned, so may impact any existing estimates.
+
+Tests for existing features require new work items to be prioritized among the backlog.
+
+## Consequences
+
+The benefits and any known/potential risks of the decision should be described herein. See Michael Nygard's article, linked above, for more details.
+
+benefits
+- more consistent deployments, testing more often in deployed environments
+- opens the door to more continuous delivery
+
+risks
+- make changes to the ci/deployment workflow, cannot enable in prod
+- authentication workaround, potential security risk
+- additional tooling; learning curve and additional support
+- impacts scope of future work and requires new work be prioriized
+
+## Notes
+

docs/Technical-Documentation/cypress-integration-tests.md

@@ -0,0 +1,97 @@
+# Cypress integration testing
+
+## Background
+
+[Cypress](https://cypress.io) is a browser-automation testing suite that we use for end-to-end tests. See [ADR019 - Integration Tests](./Architecture-Decision-Record/019-integration-tests.md) for some additional background on testing goals and decisions.
+
+## Running tests
+
+1. Have both the backend and frontend running in separate terminal processes, the app needs to be reachable and usable at `localhost:3000` when testing locally
+1. In a new terminal, set up test users by running
+   ```bash
+   cd tdrs-backend
+   docker-compose exec web python manage.py generate_cypress_users
+   ```
+1. Be sure your `tdrs-backend/.env` file contains the following
+   ```bash
+   # testing
+   CYPRESS_TOKEN=local-cypress-token
+
+   DJANGO_DEBUG=yes
+   ```
+1. In a new terminal, run the following commands to launch the Cypress runner
+   ```bash
+   cd tdrs-frontend
+   npm run test:e2e
+   ```
+1. Select "E2E Testing" from the testing type menu
+   ![Select e2e testing](./images/testing/01-e2e-selection.png)
+1. Select a browser and click "Start E2E Testing"
+   ![Select a browser](./images/testing/02-browser-selection.png)
+1. Now you can select a spec file from the menu and watch the tests run
+   ![Select a spec](./images/testing/03-spec-selection.png)
+   ![Run tests](./images/testing/04-run-test.png)
+
+## Using the Cypress runner
+
+It is highly recommended that you check out the [Cypress Test Runner](https://docs.cypress.io/guides/core-concepts/cypress-app#The-Test-Runner) feature overview. Here are the main highlights
+
+* Command log - Cypress will show you each `cy` command that executes in the command log, alongside the rendered page. 
+   * You can hover over past commands to "time travel" and replay individual steps, and clicking on a command in the log will "pin" the point in time in the runner, and log output information about that command in the console.
+   * The command log also shows network requests and other dispatched events.
+* Dev tools - you can open up chrome's devtools (right click and click "Inspect") and see all the normal DOM information about the page, including for pinned commands which is useful when debugging. The console will also show output from the page as well as the tests.
+* Automatic rerun - whenever you save your test code or frontend app code, Cypress automatically reruns the open test.
+
+## Writing tests
+
+On top of Cypress, we've layered `cypress-cucumber-preprocessor` to provide [Gherkin](https://cucumber.io/docs/gherkin/reference/) syntax for tests. This gives the tests more structure and allows us to easily separate and organize step implementations.
+
+Test files are defined as `.feature` files within the `tdrs-frontend/cypress/e2e` directory. Feature "areas" can be grouped into a folder.
+
+Step implementations are defined as `.js` files within the `tdrs-frontend/cypress/e2e` directory. A `.feature` file will load any `.js` files in its same directory.
+
+Here's an example feature file
+
+`tdrs-frontend/cypress/e2e/accounts/accounts.feature`
+```gherkin
+Feature: Users can create and manage their accounts
+    Scenario: A user can log in and request access
+        When I visit the home page
+        And I log in as a new user
+        Then I see a Request Access form
+```
+
+At its top level, it defines a `Feature` with multiple `Scenarios` (this can be likened to a `describe` and `it` in jest, respectively). Scenarios belonging to a feature area should be grouped within a Feature. Scenarios should describe a specific task/goal the user is required to perform in the system.
+
+Each line within a `Scenario` is a "step", which should describe the user interaction with the system in the business context. There are three types of steps, `Given`, `When`, and `Then`
+* `Given` steps define prerequisite state of the system in order for the tests to run. Mostly commonly, we'll use a `Given I am logged in as xyz` step to perform user login and setup prior to the test steps actually executing.
+* `When` steps define user actions needed to perform the task.
+* `Then` steps describe validation or confirmationt that the user receives, indicating success of the task.
+
+In general, all `Given`, `When`, and `Then` steps should reflect things the _user_ of the system knows about the system. This is intended to hide both technical and administrator implemenation details and focus on the end-user experience.
+
+Each step defined in a `Scenario` must have a corresponding "step implementation" (loaded from a `.js` file). Here is an example step implementation file
+
+`tdrs-frontend/cypress/e2e/accounts/steps.js`
+```js
+/* eslint-disable no-undef */
+import { When, Then } from '@badeball/cypress-cucumber-preprocessor'
+
+When('I visit the home page', () => {
+  cy.visit('/')
+  cy.contains('Sign into TANF Data Portal', { timeout: 30000 })
+})
+
+When('I log in as a new user', () => {
+  cy.login('[email protected]')
+})
+
+Then('I see a Request Access form', () => {
+  cy.contains('Welcome').should('exist')
+  cy.get('button').contains('Request Access').should('exist')
+})
+```
+
+For each step implementation, a good rule of thumb is to perform both an action and an [assertion](https://docs.cypress.io/guides/references/assertions#Chai). An action should be something the user can do in the system (click, type, etc.). Assertions help "slow down" the test and limit unexpected behavior when applications run a lot of asynchronous processes. By asserting on something verifiable in each step, we can ensure the test is in a proper state to move forward. This applies to `Given`, `When`, and `Then` steps (though `Then` steps can often omit an action).
+
+Shared step implementations, which apply to all feature files, can be added as [common step definitions](https://github.com/badeball/cypress-cucumber-preprocessor/blob/master/docs/step-definitions.md#example-2-directory-with-common-step-definitions) (which may still need to be configured).

tdrs-backend/.env.example

@@ -1,12 +1,12 @@
-### Local Development Environment
+# ## Local Development Environment
 # Copy this file to `.env` and replace variables as needed
-#
+# 
 
-###
+# ##
 # Required environment variables
 # These must be defined or the application will encounter fatal errors
 # Ask the project Tech Lead or Product Manager for these values
-#
+# 
 
 # Private JWT Key used to generate the client assertion
 JWT_KEY=a_secret_key
@@ -18,16 +18,16 @@ JWT_CERT_TEST=a_public_cert
 DJANGO_SU_NAME=[email protected]
 
 
-### AMS OpenID vars ###
+# ## AMS OpenID vars ###
 
 AMS_CONFIGURATION_ENDPOINT=
 AMS_CLIENT_ID=
 AMS_CLIENT_SECRET=
 
-###
+# ##
 # Optional environment variables
 # These need not be defined, but can be overwritten as needed
-#
+# 
 LOGGING_LEVEL=DEBUG
 
 # Local django settings to define the execution environment
@@ -82,4 +82,7 @@ POSTGRES_CHECK_TIMEOUT=30
 POSTGRES_CHECK_INTERVAL=1
 
 # Elastic stack
-ELASTIC_HOST=elastic:9200
+ELASTIC_HOST=elastic:9200
+
+# testing
+CYPRESS_TOKEN=local-cypress-token

tdrs-backend/docker-compose.yml

@@ -85,6 +85,8 @@ services:
       - REDIS_URI=redis://redis-server:6379
       - REDIS_SERVER_LOCAL=TRUE
       - ACFTITAN_SFTP_PYTEST
+      - CYPRESS_TOKEN
+      - DJANGO_DEBUG
     volumes:
       - .:/tdpapp
     image: tdp

tdrs-backend/tdpservice/settings/common.py

@@ -439,19 +439,22 @@ class Common(Configuration):
             'options': {
                 'expires': 15.0,
             },
-        },     
+        },
         'name': {
             'task': 'tdpservice.scheduling.tasks.check_for_accounts_needing_deactivation_warning',
             'schedule': crontab(day_of_week='*', hour='13', minute='*'), # Every day at 1pm UTC (9am EST)
 
             'options': {
                 'expires': 15.0,
             },
-        },     
+        },
     }
+
     # Elastic
     ELASTICSEARCH_DSL = {
         'default': {
             'hosts': os.getenv('ELASTIC_HOST', 'elastic:9200')
         },
     }
+
+    CYPRESS_TOKEN = os.getenv('CYPRESS_TOKEN', None)

tdrs-backend/tdpservice/urls.py

@@ -10,8 +10,10 @@
 from drf_yasg.views import get_schema_view
 from rest_framework.permissions import AllowAny
 
+
 from .users.api.authorization_check import AuthorizationCheck
 from .users.api.login import TokenAuthorizationLoginDotGov, TokenAuthorizationAMS
+from .users.api.login import CypressLoginDotGovAuthenticationOverride
 from .users.api.login_redirect_oidc import LoginRedirectAMS, LoginRedirectLoginDotGov
 from .users.api.logout import LogoutUser
 from .users.api.logout_redirect_oidc import LogoutRedirectOIDC
@@ -38,6 +40,12 @@
     path("logs/", write_logs),
 ]
 
+if settings.DEBUG:
+    urlpatterns.append(
+        path("login/cypress", CypressLoginDotGovAuthenticationOverride.as_view(), name="login-cypress")
+    )
+
+
 # Add 'prefix' to all urlpatterns to make it easier to version/group endpoints
 BASE_DIR = os.path.dirname(os.path.abspath(__file__))
 urlpatterns = [