Skip to content

Commit

Permalink
Changes from the guild meeting
Browse files Browse the repository at this point in the history
  • Loading branch information
aaronrussellHO committed Dec 21, 2023
1 parent a5dbb59 commit f7d5678
Showing 1 changed file with 15 additions and 45 deletions.
60 changes: 15 additions & 45 deletions docs/standards/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,68 +34,38 @@ Testing can come in different forms unit/integration/mutation/property based, bu
## Requirement(s)

- [You MUST test early and often](#you-must-test-early-and-often)
- [You MUST automate tests](#you-must-automate-tests)
- [You MUST make tests repeatable](#you-must-make-tests-repeatable)
- [You MUST have a way of measuring the effectiveness of testing](#you-must-have-a-way-of-measuring-the-effectiveness-of-testing)
- [It MUST be clear what is being tested and what the results are](#it-must-be-clear-what-is-being-tested-and-what-the-results-are)
- [Tests MUST be repeatable](#tests-must-be-repeatable)
- [Tests MUST be included for both the happy path and unhappy path](#tests-must-be-included-for-both-the-happy-path-and-unhappy-path)
- [Tests that block deployments MUST complete in a reasonable time](#tests-that-block-deployments-must-complete-in-a-reasonable-time)
- [You MUST use automated security testing](#you-must-use-automated-security-testing)
- [You MUST make sure tests fail](#you-must-make-sure-tests-fail)
- [You MUST make it clear what is being tested and what the results are](#you-must-make-it-clear-what-is-being-tested-and-what-the-results-are)
- [You MUST think about the edge cases](#you-must-think-about-the-edge-cases)

### You MUST test early and often

The earlier testing happens in the development process, the less time is spent debugging, the better quality of the code through more tests, and the different paths are considered more. Using TDD can help here, as you can test the logic before writing it, which can help develop logic and sets the foundations of what you expect your code to do.

### You MUST have a way of measuring the effectiveness of testing

You need to know ehether your test effictiveness has changed over the course of your developments and that you haven't had any regressions on your changes. Tracking this effectiveness over times helps to understand the landscape of your testing moving forward.

One way of doing this is through monitoring test coverage, Test coverage helps a team understand the amount of code that is tested within a codebase. Setting a minimum threshold (such as 80%) ensures that any new code is tested. Another example is mutation testing.

### It MUST be clear what is being tested and what the results are
### You MUST automate tests

Understanding why tests are failing and been transparent about why can help the whole team see the value, and understand where failures are occurring. You shouldn't need to check manually that tests have failed.
Automating tests enabled the tests to be ran often, and block pipelines that fail testing. It also means code is tested often, making sure that everything is working as expected.

An example is through making sure piplelines fail when tests fail, and you're able to see more information about why.

### Tests MUST be repeatable
### You MUST make tests repeatable

The ability to repeat the same tests, gives confidence that if there is a failure, something has changed to affect that failure. The values shouldn't change based on enviroments and unit tests should not depend on other external factors to run.

### Tests MUST be included for both the happy path and unhappy path

Testing for all eventualities helps to write code that can handle errors in a better way. This should include things like testing for illegal arguements, or no arguements to be passed.

### Tests that block deployments MUST complete in a reasonable time

Tests such as unit/integration tests should complete in a reasonable time to help keep a fast path to live and help developers run tests quickly to understand the impact of their changes.

Other tests such as soak that don't block deployments can be completed in a greater length of time.

If tests completion time significantly increases when a code change is made, then a test should be treated as a failing test.

### You MUST use automated security testing

Automated security testing can help understand where your code may have any security issues. These tests should be be ran as part of your pipelines to help identify issues and block the deployment of insecure code.

Automated security testing, good peer reviews and good developer knowledge around keeping applications secure help to make sure an application stays secure.

A good example of this is using Sonarqube as part of your pipelines.

### You MUST make sure tests fail
### You MUST have a way of measuring the effectiveness of testing

Making sure tests fail helps to verify that your tests are doing the correct thing and that they would fail if there is an issue.
You need to know whether your test effectiveness has changed over the course of your development and that you haven't had any regressions on your changes. Tracking this effectiveness over time helps to understand the landscape of your testing moving forward.

One way of achieving this is using red/green testing, when using TDD you get this for free, as you write the test first, have it fail (Red), then write the code to make the test pass (Green).
One way of doing this is through monitoring test coverage, Test coverage helps a team understand the amount of code that is tested within a codebase. Setting a minimum threshold (such as 80%) ensures that any new code is tested. Another example is mutation testing.

Another way of making sure tests fail is using mutation testing. Mutation testing creates small varitions of source code and subjects them to the same tests and dataset to detect any errors in the code.
### You MUST make it clear what is being tested and what the results are

### You MUST use automated accessibility testing
Understanding why tests are failing and been transparent about why can help the whole team see the value, and understand where failures are occurring. You shouldn't need to chesck manually that tests have failed.

Accessibility testing can help understand where any accessibility issuses may be in your code. If there is a failure, it shouldblock your code from deploying so you have opportunity to fix any issues first.
An example is through making sure piplelines fail when tests fail, and you're able to see more information about why.

Automated acessibility testing alongside QAT testing should (hopefully) pick up most issues related to accessibility.
### You MUST think about the edge cases

A good example is using aXe in your pipelines.
Testing for all eventualities helps to write code that can handle errors in a better way. This should include things like testing for illegal arguements, or no arguements to be passed.

---

0 comments on commit f7d5678

Please sign in to comment.