Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create testing standard #320

Merged
merged 33 commits into from
Jan 19, 2024
Merged
Changes from 16 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
357af17
Create testing.md
ThomasWilliamsHO Oct 26, 2023
0d4fb28
Update docs/standards/testing.md
aaronrussellHO Nov 23, 2023
6273ce5
Update docs/standards/testing.md
aaronrussellHO Nov 23, 2023
1ee55d0
Update docs/standards/testing.md
aaronrussellHO Nov 23, 2023
beadde8
Update docs/standards/testing.md
aaronrussellHO Nov 23, 2023
e3f280d
Updated testing standard following feedback in last SEGAS session
aaronrussellHO Nov 23, 2023
7dd13c5
Updated testing standard following feedback in last SEGAS session and…
aaronrussellHO Dec 7, 2023
a4cf5e1
Update docs/standards/testing.md
aaronrussellHO Dec 7, 2023
bec6f1a
Added section on making sure tests fail
aaronrussellHO Dec 7, 2023
51086c4
Added more on tests completing in a reasonable time
aaronrussellHO Dec 7, 2023
5bb9e15
Updated contents
aaronrussellHO Dec 7, 2023
a5dbb59
Added changes from PR
aaronrussellHO Dec 7, 2023
f7d5678
Changes from the guild meeting
aaronrussellHO Dec 21, 2023
75f9b37
Apply suggestions from code review
aaronrussellHO Dec 21, 2023
3583b5b
Apply suggestions from code review
aaronrussellHO Jan 5, 2024
cd9573c
Made changes from code review
aaronrussellHO Jan 5, 2024
4f0b8ba
Update docs/standards/testing.md
aaronrussellHO Jan 5, 2024
95226cc
Made changes from code review
aaronrussellHO Jan 5, 2024
0fd9a64
Renamed the file to match the title
aaronrussellHO Jan 5, 2024
c33c72e
Updated page info
aaronrussellHO Jan 5, 2024
a3bd994
Fixed tags
aaronrussellHO Jan 5, 2024
ea8c410
Made changes from code review
aaronrussellHO Jan 5, 2024
90fc67a
Made changes from code review
aaronrussellHO Jan 5, 2024
ed7021b
Made changes from code review
aaronrussellHO Jan 5, 2024
bcbce60
Update docs/standards/developer-testing.md
aaronrussellHO Jan 19, 2024
bc74ce8
Update docs/standards/developer-testing.md
aaronrussellHO Jan 19, 2024
56d9e8e
Update docs/standards/developer-testing.md
aaronrussellHO Jan 19, 2024
50573fe
Update docs/standards/developer-testing.md
aaronrussellHO Jan 19, 2024
6b61dfe
Added related links
aaronrussellHO Jan 19, 2024
c7f9809
Update docs/standards/developer-testing.md
aaronrussellHO Jan 19, 2024
8029ec0
Removed weak language
aaronrussellHO Jan 19, 2024
8191218
Reworded into to align more with standard
aaronrussellHO Jan 19, 2024
8f923f3
Added about Red, Green, Refactor approach
aaronrussellHO Jan 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions docs/standards/testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
layout: standard
order: 1
title: Developer Testing
aaronrussellHO marked this conversation as resolved.
Show resolved Hide resolved
date: 2023-10-26
id: SEGAS-TOBEFILLED
tags:
- Source management
- Secure development
- Ways of working
- Testing
related:
sections:
- title: Related links
items:
---

Testing code forms part of the core of developing code. Testing your changes is necessary to gain assurance that they can be deployed safely to production and that the service will still function as expected.

The intent of any test should be clear as they can provide a form of documentation to the overall solution.
The tests should be relatively easy to understand with a track of the overall testing coverage kept. It is good to keep in mind that test coverage of a code base is not a definitive marker of quality.
aaronrussellHO marked this conversation as resolved.
Show resolved Hide resolved

A good test should be clear on the intent of the test, have one test case, be readable and should pass every time without changes to the underlying code.

Testing can come in different forms, unit/integration/mutation/property based, but all must adhere to the requirements as the following benefits can be achieved:

- Easy to correct failures in code from failed tests.
- Security & confidence when continously integrating code
- Easy to understand intent and functionality of code
- Developers should be able to write and run tests and not silo'd to QAT
aaronrussellHO marked this conversation as resolved.
Show resolved Hide resolved

---

## Requirement(s)

- [You MUST test early and often](#you-must-test-early-and-often)
- [You MUST automate tests](#you-must-automate-tests)
- [You MUST make tests repeatable](#you-must-make-tests-repeatable)
- [You MUST have a way of measuring the effectiveness of testing](#you-must-have-a-way-of-measuring-the-effectiveness-of-testing)
- [Tests MUST have a purpose and explicit result](#tests-must-have-a-purpose-and-explicit-result)
- [You MUST think about the edge cases](#you-must-think-about-the-edge-cases)

### You MUST test early and often

The earlier testing happens in the development process, the less time is spent debugging, the better quality of the code through more tests, and the different paths are considered more. Using TDD can help here, as you can test the logic before writing it, which can help develop logic and sets the foundations of what you expect your code to do.
aaronrussellHO marked this conversation as resolved.
Show resolved Hide resolved

### You MUST automate tests

Automating tests enables the tests to be run consistently and often. Automated tests can be run as part of a build pipeline, and block the pipeline if they fail. It also means code is tested often, making sure that everything is working as expected.

### You MUST make tests repeatable

The ability to repeat the same tests, gives confidence that if there is a failure, something has changed to affect that failure. The values shouldn't change based on enviroments and unit tests should not depend on other external factors to run.
aaronrussellHO marked this conversation as resolved.
Show resolved Hide resolved

### You MUST have a way of measuring the effectiveness of testing

You need to know whether your test effectiveness has changed over the course of your development and that you haven't had any regressions on your changes. Tracking this effectiveness over time helps to understand the landscape of your testing moving forward.

One way of doing this is through monitoring test coverage, Test coverage helps a team understand the amount of code that is tested within a codebase. Setting a minimum threshold (such as 80%) ensures that any new code is tested. Another example is mutation testing.

### Tests MUST have a purpose and explicit result

Tests must indicate why they are failing, showing expected and actual responses and the name of the test.

An example is through making sure pipelines fail when tests fail, and you're able to see more information about why.

aaronrussellHO marked this conversation as resolved.
Show resolved Hide resolved
A test failing due to a regression often exposes an engineer to functionality they are not familiar with. The information logged along with the test implementation should provide enough context that they can understand what is being tested and what has gone wrong, and be able to reason about how they can resolve it.
aaronrussellHO marked this conversation as resolved.
Show resolved Hide resolved
### You MUST think about the edge cases

Testing for all eventualities helps to write code that can handle errors in a better way. This should include things like testing for illegal arguements, or no arguements to be passed.
aaronrussellHO marked this conversation as resolved.
Show resolved Hide resolved

aaronrussellHO marked this conversation as resolved.
Show resolved Hide resolved
---
aaronrussellHO marked this conversation as resolved.
Show resolved Hide resolved