From 357af17b2b42ae0d84e605eb7fe34a102358263c Mon Sep 17 00:00:00 2001 From: Thomas Williams <125026090+ThomasWilliamsHO@users.noreply.github.com> Date: Thu, 26 Oct 2023 12:10:11 +0100 Subject: [PATCH 01/33] Create testing.md --- docs/standards/testing.md | 53 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 docs/standards/testing.md diff --git a/docs/standards/testing.md b/docs/standards/testing.md new file mode 100644 index 00000000..8813f591 --- /dev/null +++ b/docs/standards/testing.md @@ -0,0 +1,53 @@ +--- +layout: standard +order: 1 +title: Testing +date: 2023-10-26 +id: SEGAS-TOBEFILLED +tags: + - Source management + - Secure development + - Ways of working + - Testing +related: + sections: + - title: Related links + items: +--- + +Testing code forms part of the core of developing code. Testing should provide confidence to stakeholders and the intent of any test should be clear as they can provide as a form of documentation to the overall solution, they should be relatively easy to understand with coverage tracked, though this must be stressed coverage is not the be all end all. + +Testing can come in different forms unit/integration/mutation/property based, but all must adhere to the requirements laid out as the following benefits can be achieved from it: + +- Easy to connect failures in code from failed tests +- Security & confidence when continously integrating code +- Easy to understand intent and functionality of code + +--- + +## Requirement(s) + +- [Track test coverage](#track-test-coverage) +- [Tests should be clear and easy to connect failures](#tests-should-be-clear-and-easy-to-connect-failures) +- [Testing should be done early and often](#testing-should-be-done-early-and-often) + + +### Track test coverage + +Guidance on tracking test coverage is available as below. + +- [Test Link](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) + +### Tests should be clear and easy to connect failures + +Guidance on making tests clear is available as below. + +- [Test Link](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) + +### Testing should be done early and often + +Guidance on testing often and early is available as below. + +- [Test Link](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) + +--- From 0d4fb283abc6eec451e3babff5633be1d7e9fbbd Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Thu, 23 Nov 2023 13:25:51 +0000 Subject: [PATCH 02/33] Update docs/standards/testing.md --- docs/standards/testing.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 8813f591..08c5295a 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -15,7 +15,9 @@ related: items: --- -Testing code forms part of the core of developing code. Testing should provide confidence to stakeholders and the intent of any test should be clear as they can provide as a form of documentation to the overall solution, they should be relatively easy to understand with coverage tracked, though this must be stressed coverage is not the be all end all. +Testing code forms part of the core of developing code. Testing should provide confidence to stakeholders and be proportionate to risk. + +The intent of any test should be clear as they can provide as a form of documentation to the overall solution, they should be relatively easy to understand with coverage tracked, though this must be stressed coverage is not the be all end all. Testing can come in different forms unit/integration/mutation/property based, but all must adhere to the requirements laid out as the following benefits can be achieved from it: From 6273ce5c0aaa81e33368ccdc18ae7392cab5a733 Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Thu, 23 Nov 2023 13:26:10 +0000 Subject: [PATCH 03/33] Update docs/standards/testing.md --- docs/standards/testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 08c5295a..7bfd3613 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -1,7 +1,7 @@ --- layout: standard order: 1 -title: Testing +title: Developer Testing date: 2023-10-26 id: SEGAS-TOBEFILLED tags: From 1ee55d0a998b0abaed18a1b35e4faa3897d6c072 Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Thu, 23 Nov 2023 13:26:24 +0000 Subject: [PATCH 04/33] Update docs/standards/testing.md --- docs/standards/testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 7bfd3613..edc9bc68 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -40,7 +40,7 @@ Guidance on tracking test coverage is available as below. - [Test Link](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) -### Tests should be clear and easy to connect failures +### It MUST be clear when and why tests fail. Guidance on making tests clear is available as below. From beadde8477c4a9d5469727d61a8c2b552c5f1f3b Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Thu, 23 Nov 2023 13:26:32 +0000 Subject: [PATCH 05/33] Update docs/standards/testing.md --- docs/standards/testing.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index edc9bc68..87066941 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -52,4 +52,9 @@ Guidance on testing often and early is available as below. - [Test Link](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) + +### Tests MUST be repeatable + + + --- From e3f280d75607a2e71b7cb15f237325b6a1afe776 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Thu, 23 Nov 2023 14:24:41 +0000 Subject: [PATCH 06/33] Updated testing standard following feedback in last SEGAS session --- docs/standards/testing.md | 42 ++++++++++++++++++++++++--------------- 1 file changed, 26 insertions(+), 16 deletions(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 87066941..9b413c15 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -15,46 +15,56 @@ related: items: --- -Testing code forms part of the core of developing code. Testing should provide confidence to stakeholders and be proportionate to risk. +Testing code forms part of the core of developing code. Testing should provide confidence to stakeholders and be proportionate to risk. The intent of any test should be clear as they can provide as a form of documentation to the overall solution, they should be relatively easy to understand with coverage tracked, though this must be stressed coverage is not the be all end all. +A good test should be clear on the intent of the test, should only have one test case, and be readable and the test should pass everytime without changes to the underlying code. + Testing can come in different forms unit/integration/mutation/property based, but all must adhere to the requirements laid out as the following benefits can be achieved from it: - Easy to connect failures in code from failed tests - Security & confidence when continously integrating code -- Easy to understand intent and functionality of code +- Easy to understand intent and functionality of code --- ## Requirement(s) -- [Track test coverage](#track-test-coverage) -- [Tests should be clear and easy to connect failures](#tests-should-be-clear-and-easy-to-connect-failures) -- [Testing should be done early and often](#testing-should-be-done-early-and-often) - - -### Track test coverage +- [You MUST track test coverage](#you-must-track-test-coverage) +- [It MUST be clear when and why tests fail](#it-must-be-clear-when-and-why-tests-fail) +- [You MUST test early and often](#you-must-test-early-and-often) +- [Tests MUST be repeatable](#tests-must-be-repeatable) +- [Tests MUST be included for both the happy path and unhappy path](#tests-must-be-included-for-both-the-happy-path-and-unhappy-path) +- [Tests MUST be fast](#tests-must-be-fast) +- [You MUST test for security issues as part of your pipelines](#you-must-test-for-security-issues-as-part-of-your-pipelines) -Guidance on tracking test coverage is available as below. +### You MUST track test coverage -- [Test Link](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) +Test coverage helps a team understand the amount of code that is tested within a codebase. Setting a minimum threshold (such as 80%) ensures that any new code is tested. Coverage is not always accurate, as if a test has an always truthy assertion then coverage can over estimate. However, it is a good indication overall. ### It MUST be clear when and why tests fail. -Guidance on making tests clear is available as below. +Understanding why tests are failing and been transparent about why (such as a report, or CI pipeline) can help the whole team see the value, and understand where failures are occuring. It also builds confidence within the team and business when tests wither catch something, or that they are running regularily. You shouldn't need to check manually whether a test has passed or failed. + +### You MUST test early and often -- [Test Link](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) +The earlier testing happens in the development process, the less time is spent debugging, the better quality of the code through more tests, and the different paths are considered more. Using TDD can help here, as you can test the logic before writing it, which can help develop logic and sets the foundations of what you expect your code to do. -### Testing should be done early and often +### Tests MUST be repeatable -Guidance on testing often and early is available as below. +The ability to repeat the same tests, gives confidence that if there is a failure, something has changed to affect that failure. The values shouldn't change based on enviroments and unit tests should not depend on other external factors to run. -- [Test Link](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) +### Tests MUST be included for both the happy path and unhappy path +Testing for all eventualities helps to write code that can handle errors in a better way. This should include things like testing for illegal arguements, or no arguements to be passed. -### Tests MUST be repeatable +### Tests MUST be fast + +A developer should be able to run tests quickly to help understand the inpact of change. The longer it takes for a suite of tests to run, the less likely a developer is going to run the entire suite, which might cause issues later on in the development cycle. +### You MUST test for security issues as part of your pipelines +Testing security issues in your pipeline is vital to help make sure your application is secure and doesn't have any vulnerabilities. --- From 7dd13c5c7389b5cfb8545d0e288c011e6704763e Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Thu, 7 Dec 2023 09:05:33 +0000 Subject: [PATCH 07/33] Updated testing standard following feedback in last SEGAS session and on this PR --- docs/standards/testing.md | 40 ++++++++++++++++++++++++--------------- 1 file changed, 25 insertions(+), 15 deletions(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 9b413c15..6a1fd9db 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -15,7 +15,7 @@ related: items: --- -Testing code forms part of the core of developing code. Testing should provide confidence to stakeholders and be proportionate to risk. +Testing code forms part of the core of developing code. Testing your changes is necessary to gain assurance that they can be deployed safely to production and that the service will still function as expected. The intent of any test should be clear as they can provide as a form of documentation to the overall solution, they should be relatively easy to understand with coverage tracked, though this must be stressed coverage is not the be all end all. @@ -31,25 +31,29 @@ Testing can come in different forms unit/integration/mutation/property based, bu ## Requirement(s) -- [You MUST track test coverage](#you-must-track-test-coverage) -- [It MUST be clear when and why tests fail](#it-must-be-clear-when-and-why-tests-fail) - [You MUST test early and often](#you-must-test-early-and-often) +- [You MUST have a way of measuring the effectiveness of testing](#you-must-have-a-way-of-measuring-the-effectiveness-of-testing) +- [It MUST be clear what is being tested and what the results are](#it-must-be-clear-what-is-being-tested-and-what-the-results-are) - [Tests MUST be repeatable](#tests-must-be-repeatable) - [Tests MUST be included for both the happy path and unhappy path](#tests-must-be-included-for-both-the-happy-path-and-unhappy-path) -- [Tests MUST be fast](#tests-must-be-fast) -- [You MUST test for security issues as part of your pipelines](#you-must-test-for-security-issues-as-part-of-your-pipelines) +- [Tests that block deployments MUST complete in a reasonable time](#tests-that-block-deployments-must-complete-in-a-reasonable-time) +- [You MUST use automated security testing](#you-must-use-automated-security-testing) -### You MUST track test coverage +### You MUST test early and often -Test coverage helps a team understand the amount of code that is tested within a codebase. Setting a minimum threshold (such as 80%) ensures that any new code is tested. Coverage is not always accurate, as if a test has an always truthy assertion then coverage can over estimate. However, it is a good indication overall. +The earlier testing happens in the development process, the less time is spent debugging, the better quality of the code through more tests, and the different paths are considered more. Using TDD can help here, as you can test the logic before writing it, which can help develop logic and sets the foundations of what you expect your code to do. -### It MUST be clear when and why tests fail. +### You MUST have a way of measuring the effectiveness of testing -Understanding why tests are failing and been transparent about why (such as a report, or CI pipeline) can help the whole team see the value, and understand where failures are occuring. It also builds confidence within the team and business when tests wither catch something, or that they are running regularily. You shouldn't need to check manually whether a test has passed or failed. +You need to know ehether your test effictiveness has changed over the course of your developments and that you haven't had any regressions on your changes. Tracking this effectiveness over times helps to understand the landscape of your testing moving forward. -### You MUST test early and often +One way of doing this is through monitoring test coverage, Test coverage helps a team understand the amount of code that is tested within a codebase. Setting a minimum threshold (such as 80%) ensures that any new code is tested. Another example is mutation testing. -The earlier testing happens in the development process, the less time is spent debugging, the better quality of the code through more tests, and the different paths are considered more. Using TDD can help here, as you can test the logic before writing it, which can help develop logic and sets the foundations of what you expect your code to do. +### It MUST be clear what is being tested and what the results are + +Understanding why tests are failing and been transparent about why can help the whole team see the value, and understand where failures are occurring. You shouldn't need to check manually that tests have failed. + +An example is through making sure piplelines fail when tests fail, and you're able to see more information about why. ### Tests MUST be repeatable @@ -59,12 +63,18 @@ The ability to repeat the same tests, gives confidence that if there is a failur Testing for all eventualities helps to write code that can handle errors in a better way. This should include things like testing for illegal arguements, or no arguements to be passed. -### Tests MUST be fast +### Tests that block deployments MUST complete in a reasonable time + +Tests such as unit/integration tests should complete in a reasonable time to help keep a fast path to live and help developers run tests quickly to understand the impact of their changes. + +Other tests such as soak that don't block deployments can be completed in a greater length of time. + +### You MUST use automated security testing -A developer should be able to run tests quickly to help understand the inpact of change. The longer it takes for a suite of tests to run, the less likely a developer is going to run the entire suite, which might cause issues later on in the development cycle. +Automated security testing can help understand where your code may have any security issues. These tests should be be ran as part of your pipelines to help identify issues and block the deployment of insecure code. -### You MUST test for security issues as part of your pipelines +Automated security testing, good peer reviews and good developer knowledge around keeping applications secure help to make sure an application stays secure. -Testing security issues in your pipeline is vital to help make sure your application is secure and doesn't have any vulnerabilities. +A good example of this is using Sonarqube as part of your pipelines. --- From a4cf5e11dfdea56599c4d1732d8d883eb7ff0ce5 Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Thu, 7 Dec 2023 09:06:43 +0000 Subject: [PATCH 08/33] Update docs/standards/testing.md Co-authored-by: Daniel A.C. Martin --- docs/standards/testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 6a1fd9db..926a9838 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -17,7 +17,7 @@ related: Testing code forms part of the core of developing code. Testing your changes is necessary to gain assurance that they can be deployed safely to production and that the service will still function as expected. -The intent of any test should be clear as they can provide as a form of documentation to the overall solution, they should be relatively easy to understand with coverage tracked, though this must be stressed coverage is not the be all end all. +The intent of any test should be clear as they can provide a form of documentation to the overall solution, they should be relatively easy to understand with coverage tracked, although it must be stressed that coverage is not the be all end all. A good test should be clear on the intent of the test, should only have one test case, and be readable and the test should pass everytime without changes to the underlying code. From bec6f1a6d3c111a73050d7bb671e3bc4a60e769b Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Thu, 7 Dec 2023 09:21:14 +0000 Subject: [PATCH 09/33] Added section on making sure tests fail --- docs/standards/testing.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 926a9838..2848e585 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -77,4 +77,12 @@ Automated security testing, good peer reviews and good developer knowledge aroun A good example of this is using Sonarqube as part of your pipelines. +### You MUST make sure tests fail + +Making sure tests fail helps to verify that your tests are doing the correct thing and that they would fail if there is an issue. + +One way of achieving this is using red/green testing, when using TDD you get this for free, as you write the test first, have it fail (Red), then write the code to make the test pass (Green). + +Another way of making sure tests fail is using mutation testing. Mutation testing creates small varitions of source code and subjects them to the same tests and dataset to detect any errors in the code. + --- From 51086c4995ef8e4f75a2279e5afd3162cdf03b13 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Thu, 7 Dec 2023 09:26:43 +0000 Subject: [PATCH 10/33] Added more on tests completing in a reasonable time --- docs/standards/testing.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 2848e585..fe5c6342 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -69,6 +69,8 @@ Tests such as unit/integration tests should complete in a reasonable time to hel Other tests such as soak that don't block deployments can be completed in a greater length of time. +If tests completion time significantly increases when a code change is made, then a test should be treated as a failing test. + ### You MUST use automated security testing Automated security testing can help understand where your code may have any security issues. These tests should be be ran as part of your pipelines to help identify issues and block the deployment of insecure code. From 5bb9e152c222cfb26e74a51b8c8f9706df150512 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Thu, 7 Dec 2023 09:31:22 +0000 Subject: [PATCH 11/33] Updated contents --- docs/standards/testing.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index fe5c6342..72ead40e 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -38,6 +38,7 @@ Testing can come in different forms unit/integration/mutation/property based, bu - [Tests MUST be included for both the happy path and unhappy path](#tests-must-be-included-for-both-the-happy-path-and-unhappy-path) - [Tests that block deployments MUST complete in a reasonable time](#tests-that-block-deployments-must-complete-in-a-reasonable-time) - [You MUST use automated security testing](#you-must-use-automated-security-testing) +- [You MUST make sure tests fail](#you-must-make-sure-tests-fail) ### You MUST test early and often From a5dbb59cdf60006f4ad65b195926a09d45f7752c Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Thu, 7 Dec 2023 12:59:49 +0000 Subject: [PATCH 12/33] Added changes from PR --- docs/standards/testing.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 72ead40e..7e69b6a3 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -26,6 +26,8 @@ Testing can come in different forms unit/integration/mutation/property based, bu - Easy to connect failures in code from failed tests - Security & confidence when continously integrating code - Easy to understand intent and functionality of code +- Tests should be automated when possible +- Developers should be able to write and run tests and not silo'd to QAT --- @@ -88,4 +90,12 @@ One way of achieving this is using red/green testing, when using TDD you get thi Another way of making sure tests fail is using mutation testing. Mutation testing creates small varitions of source code and subjects them to the same tests and dataset to detect any errors in the code. +### You MUST use automated accessibility testing + +Accessibility testing can help understand where any accessibility issuses may be in your code. If there is a failure, it shouldblock your code from deploying so you have opportunity to fix any issues first. + +Automated acessibility testing alongside QAT testing should (hopefully) pick up most issues related to accessibility. + +A good example is using aXe in your pipelines. + --- From f7d5678ed74363b5ec8616bcdd8f98b418ebcae4 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Thu, 21 Dec 2023 13:13:17 +0000 Subject: [PATCH 13/33] Changes from the guild meeting --- docs/standards/testing.md | 60 ++++++++++----------------------------- 1 file changed, 15 insertions(+), 45 deletions(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 7e69b6a3..31236596 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -34,68 +34,38 @@ Testing can come in different forms unit/integration/mutation/property based, bu ## Requirement(s) - [You MUST test early and often](#you-must-test-early-and-often) +- [You MUST automate tests](#you-must-automate-tests) +- [You MUST make tests repeatable](#you-must-make-tests-repeatable) - [You MUST have a way of measuring the effectiveness of testing](#you-must-have-a-way-of-measuring-the-effectiveness-of-testing) -- [It MUST be clear what is being tested and what the results are](#it-must-be-clear-what-is-being-tested-and-what-the-results-are) -- [Tests MUST be repeatable](#tests-must-be-repeatable) -- [Tests MUST be included for both the happy path and unhappy path](#tests-must-be-included-for-both-the-happy-path-and-unhappy-path) -- [Tests that block deployments MUST complete in a reasonable time](#tests-that-block-deployments-must-complete-in-a-reasonable-time) -- [You MUST use automated security testing](#you-must-use-automated-security-testing) -- [You MUST make sure tests fail](#you-must-make-sure-tests-fail) +- [You MUST make it clear what is being tested and what the results are](#you-must-make-it-clear-what-is-being-tested-and-what-the-results-are) +- [You MUST think about the edge cases](#you-must-think-about-the-edge-cases) ### You MUST test early and often The earlier testing happens in the development process, the less time is spent debugging, the better quality of the code through more tests, and the different paths are considered more. Using TDD can help here, as you can test the logic before writing it, which can help develop logic and sets the foundations of what you expect your code to do. -### You MUST have a way of measuring the effectiveness of testing - -You need to know ehether your test effictiveness has changed over the course of your developments and that you haven't had any regressions on your changes. Tracking this effectiveness over times helps to understand the landscape of your testing moving forward. - -One way of doing this is through monitoring test coverage, Test coverage helps a team understand the amount of code that is tested within a codebase. Setting a minimum threshold (such as 80%) ensures that any new code is tested. Another example is mutation testing. - -### It MUST be clear what is being tested and what the results are +### You MUST automate tests -Understanding why tests are failing and been transparent about why can help the whole team see the value, and understand where failures are occurring. You shouldn't need to check manually that tests have failed. +Automating tests enabled the tests to be ran often, and block pipelines that fail testing. It also means code is tested often, making sure that everything is working as expected. -An example is through making sure piplelines fail when tests fail, and you're able to see more information about why. - -### Tests MUST be repeatable +### You MUST make tests repeatable The ability to repeat the same tests, gives confidence that if there is a failure, something has changed to affect that failure. The values shouldn't change based on enviroments and unit tests should not depend on other external factors to run. -### Tests MUST be included for both the happy path and unhappy path - -Testing for all eventualities helps to write code that can handle errors in a better way. This should include things like testing for illegal arguements, or no arguements to be passed. - -### Tests that block deployments MUST complete in a reasonable time - -Tests such as unit/integration tests should complete in a reasonable time to help keep a fast path to live and help developers run tests quickly to understand the impact of their changes. - -Other tests such as soak that don't block deployments can be completed in a greater length of time. - -If tests completion time significantly increases when a code change is made, then a test should be treated as a failing test. - -### You MUST use automated security testing - -Automated security testing can help understand where your code may have any security issues. These tests should be be ran as part of your pipelines to help identify issues and block the deployment of insecure code. - -Automated security testing, good peer reviews and good developer knowledge around keeping applications secure help to make sure an application stays secure. - -A good example of this is using Sonarqube as part of your pipelines. - -### You MUST make sure tests fail +### You MUST have a way of measuring the effectiveness of testing -Making sure tests fail helps to verify that your tests are doing the correct thing and that they would fail if there is an issue. +You need to know whether your test effectiveness has changed over the course of your development and that you haven't had any regressions on your changes. Tracking this effectiveness over time helps to understand the landscape of your testing moving forward. -One way of achieving this is using red/green testing, when using TDD you get this for free, as you write the test first, have it fail (Red), then write the code to make the test pass (Green). +One way of doing this is through monitoring test coverage, Test coverage helps a team understand the amount of code that is tested within a codebase. Setting a minimum threshold (such as 80%) ensures that any new code is tested. Another example is mutation testing. -Another way of making sure tests fail is using mutation testing. Mutation testing creates small varitions of source code and subjects them to the same tests and dataset to detect any errors in the code. +### You MUST make it clear what is being tested and what the results are -### You MUST use automated accessibility testing +Understanding why tests are failing and been transparent about why can help the whole team see the value, and understand where failures are occurring. You shouldn't need to chesck manually that tests have failed. -Accessibility testing can help understand where any accessibility issuses may be in your code. If there is a failure, it shouldblock your code from deploying so you have opportunity to fix any issues first. +An example is through making sure piplelines fail when tests fail, and you're able to see more information about why. -Automated acessibility testing alongside QAT testing should (hopefully) pick up most issues related to accessibility. +### You MUST think about the edge cases -A good example is using aXe in your pipelines. +Testing for all eventualities helps to write code that can handle errors in a better way. This should include things like testing for illegal arguements, or no arguements to be passed. --- From 75f9b37b4a8d62ac377a9a1064242fd5d2ac534b Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Thu, 21 Dec 2023 15:46:01 +0000 Subject: [PATCH 14/33] Apply suggestions from code review --- docs/standards/testing.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 31236596..cfbdcf7e 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -37,7 +37,7 @@ Testing can come in different forms unit/integration/mutation/property based, bu - [You MUST automate tests](#you-must-automate-tests) - [You MUST make tests repeatable](#you-must-make-tests-repeatable) - [You MUST have a way of measuring the effectiveness of testing](#you-must-have-a-way-of-measuring-the-effectiveness-of-testing) -- [You MUST make it clear what is being tested and what the results are](#you-must-make-it-clear-what-is-being-tested-and-what-the-results-are) +- [Tests MUST have a purpose and explicit result](#tests-must-have-a-purpose-and-explicit-result) - [You MUST think about the edge cases](#you-must-think-about-the-edge-cases) ### You MUST test early and often @@ -58,11 +58,11 @@ You need to know whether your test effectiveness has changed over the course of One way of doing this is through monitoring test coverage, Test coverage helps a team understand the amount of code that is tested within a codebase. Setting a minimum threshold (such as 80%) ensures that any new code is tested. Another example is mutation testing. -### You MUST make it clear what is being tested and what the results are +### Tests MUST have a purpose and explicit result -Understanding why tests are failing and been transparent about why can help the whole team see the value, and understand where failures are occurring. You shouldn't need to chesck manually that tests have failed. +Tests must indicate why they are failing, showing expected and actual responses and the name of the test. -An example is through making sure piplelines fail when tests fail, and you're able to see more information about why. +An example is through making sure pipelines fail when tests fail, and you're able to see more information about why. ### You MUST think about the edge cases From 3583b5b860e0df53f2caf8aa7028c6d8a47d82ed Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Fri, 5 Jan 2024 08:56:13 +0000 Subject: [PATCH 15/33] Apply suggestions from code review Co-authored-by: Jeff Horton <87995501+jeff-horton-ho-sas@users.noreply.github.com> --- docs/standards/testing.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index cfbdcf7e..297db61c 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -46,7 +46,7 @@ The earlier testing happens in the development process, the less time is spent d ### You MUST automate tests -Automating tests enabled the tests to be ran often, and block pipelines that fail testing. It also means code is tested often, making sure that everything is working as expected. +Automating tests enables the tests to be run consistently and often. Automated tests can be run as part of a build pipeline, and block the pipeline if they fail. It also means code is tested often, making sure that everything is working as expected. ### You MUST make tests repeatable @@ -64,6 +64,7 @@ Tests must indicate why they are failing, showing expected and actual responses An example is through making sure pipelines fail when tests fail, and you're able to see more information about why. +A test failing due to a regression often exposes an engineer to functionality they are not familiar with. The information logged along with the test implementation should provide enough context that they can understand what is being tested and what has gone wrong, and be able to reason about how they can resolve it. ### You MUST think about the edge cases Testing for all eventualities helps to write code that can handle errors in a better way. This should include things like testing for illegal arguements, or no arguements to be passed. From cd9573c4e898f95dea37767de5652ae440f6b0c5 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 5 Jan 2024 09:04:27 +0000 Subject: [PATCH 16/33] Made changes from code review --- docs/standards/testing.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 297db61c..f13fac12 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -17,16 +17,16 @@ related: Testing code forms part of the core of developing code. Testing your changes is necessary to gain assurance that they can be deployed safely to production and that the service will still function as expected. -The intent of any test should be clear as they can provide a form of documentation to the overall solution, they should be relatively easy to understand with coverage tracked, although it must be stressed that coverage is not the be all end all. +The intent of any test should be clear as they can provide a form of documentation to the overall solution. +The tests should be relatively easy to understand with a track of the overall testing coverage kept. It is good to keep in mind that test coverage of a code base is not a definitive marker of quality. -A good test should be clear on the intent of the test, should only have one test case, and be readable and the test should pass everytime without changes to the underlying code. +A good test should be clear on the intent of the test, have one test case, be readable and should pass every time without changes to the underlying code. -Testing can come in different forms unit/integration/mutation/property based, but all must adhere to the requirements laid out as the following benefits can be achieved from it: +Testing can come in different forms, unit/integration/mutation/property based, but all must adhere to the requirements as the following benefits can be achieved: -- Easy to connect failures in code from failed tests +- Easy to correct failures in code from failed tests. - Security & confidence when continously integrating code - Easy to understand intent and functionality of code -- Tests should be automated when possible - Developers should be able to write and run tests and not silo'd to QAT --- From 4f0b8ba252e7138793b9ff5aacf4ef6462104c13 Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Fri, 5 Jan 2024 09:21:19 +0000 Subject: [PATCH 17/33] Update docs/standards/testing.md --- docs/standards/testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index f13fac12..016110a2 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -27,7 +27,7 @@ Testing can come in different forms, unit/integration/mutation/property based, b - Easy to correct failures in code from failed tests. - Security & confidence when continously integrating code - Easy to understand intent and functionality of code -- Developers should be able to write and run tests and not silo'd to QAT +- Tests are owned by the same team who owns the code and not within a silo external to the team. --- From 95226cccc4a4a1dc67d09e2c9703eedce2710444 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 5 Jan 2024 09:23:42 +0000 Subject: [PATCH 18/33] Made changes from code review --- docs/standards/testing.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/standards/testing.md b/docs/standards/testing.md index 016110a2..5cc84ce3 100644 --- a/docs/standards/testing.md +++ b/docs/standards/testing.md @@ -18,7 +18,8 @@ related: Testing code forms part of the core of developing code. Testing your changes is necessary to gain assurance that they can be deployed safely to production and that the service will still function as expected. The intent of any test should be clear as they can provide a form of documentation to the overall solution. -The tests should be relatively easy to understand with a track of the overall testing coverage kept. It is good to keep in mind that test coverage of a code base is not a definitive marker of quality. + +The tests should be relatively easy to understand with a track of the overall testing coverage kept. Strong test coverage of a code base contributes towards overall quality of application code, although depending on your application context, it may not necessarily be the only definitive marker of quality code. A good test should be clear on the intent of the test, have one test case, be readable and should pass every time without changes to the underlying code. @@ -65,6 +66,7 @@ Tests must indicate why they are failing, showing expected and actual responses An example is through making sure pipelines fail when tests fail, and you're able to see more information about why. A test failing due to a regression often exposes an engineer to functionality they are not familiar with. The information logged along with the test implementation should provide enough context that they can understand what is being tested and what has gone wrong, and be able to reason about how they can resolve it. + ### You MUST think about the edge cases Testing for all eventualities helps to write code that can handle errors in a better way. This should include things like testing for illegal arguements, or no arguements to be passed. From 0fd9a64d5f89be8b610080d822cd6ecc67a44438 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 5 Jan 2024 09:24:52 +0000 Subject: [PATCH 19/33] Renamed the file to match the title --- docs/standards/{testing.md => developer-testing.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename docs/standards/{testing.md => developer-testing.md} (100%) diff --git a/docs/standards/testing.md b/docs/standards/developer-testing.md similarity index 100% rename from docs/standards/testing.md rename to docs/standards/developer-testing.md From c33c72e72ba5f5082146bd67007650cf25f9fd77 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 5 Jan 2024 09:26:24 +0000 Subject: [PATCH 20/33] Updated page info --- docs/standards/developer-testing.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index 5cc84ce3..5020d844 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -2,11 +2,10 @@ layout: standard order: 1 title: Developer Testing -date: 2023-10-26 -id: SEGAS-TOBEFILLED +date: 2024-01-05 +id: SEGAS-00013 tags: - - Source management - - Secure development + - Software Design - Ways of working - Testing related: From a3bd994984bf15a2b9b3b2826596fc8184a83c42 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 5 Jan 2024 09:29:53 +0000 Subject: [PATCH 21/33] Fixed tags --- docs/standards/developer-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index 5020d844..097edb4f 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -5,7 +5,7 @@ title: Developer Testing date: 2024-01-05 id: SEGAS-00013 tags: - - Software Design + - Software design - Ways of working - Testing related: From ea8c4101fb4b9161d709559cdc5b1d475ac2e664 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 5 Jan 2024 09:56:54 +0000 Subject: [PATCH 22/33] Made changes from code review --- docs/standards/developer-testing.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index 097edb4f..de6ce740 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -42,7 +42,9 @@ Testing can come in different forms, unit/integration/mutation/property based, b ### You MUST test early and often -The earlier testing happens in the development process, the less time is spent debugging, the better quality of the code through more tests, and the different paths are considered more. Using TDD can help here, as you can test the logic before writing it, which can help develop logic and sets the foundations of what you expect your code to do. +Writing tests early and often in a software development process tends to reduce the amount of time spent debugging. Good tests written early generally helps improve the overall assurance and design quality of your application. + +Test Driven Development (TDD) is a valuable development approach that can help towards achieving this. This development approach encourages you to think ahead on how your requirements can be met by thinking on how your solution can be tested first at unit level. Once a test is written first albeit failing due to non-existing solution, you can implement your solution to pass the test, and then refactor. "Testing early and often" in this case leads to "refactoring early and often", which continuously improves the quality of your application code and design. ### You MUST automate tests From 90fc67a5e68fdac398790e3445095cf952ddd81f Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 5 Jan 2024 10:02:47 +0000 Subject: [PATCH 23/33] Made changes from code review --- docs/standards/developer-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index de6ce740..909e389a 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -52,7 +52,7 @@ Automating tests enables the tests to be run consistently and often. Automated t ### You MUST make tests repeatable -The ability to repeat the same tests, gives confidence that if there is a failure, something has changed to affect that failure. The values shouldn't change based on enviroments and unit tests should not depend on other external factors to run. +The ability to repeat the same tests gives confidence that if there is a failure, something has changed to affect that failure. The values that a test rely on shouldn't change based on environments, and tests such as unit tests should not depend on other external factors and dependencies (such as third party API, etc) to run. ### You MUST have a way of measuring the effectiveness of testing From ed7021b77e0b213388dd2dbe3239cbe1421aca11 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 5 Jan 2024 10:26:04 +0000 Subject: [PATCH 24/33] Made changes from code review --- docs/standards/developer-testing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index 909e389a..735a4c6f 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -66,10 +66,10 @@ Tests must indicate why they are failing, showing expected and actual responses An example is through making sure pipelines fail when tests fail, and you're able to see more information about why. -A test failing due to a regression often exposes an engineer to functionality they are not familiar with. The information logged along with the test implementation should provide enough context that they can understand what is being tested and what has gone wrong, and be able to reason about how they can resolve it. +A test failing due to a regression often implies that a functionality has been added that contradict existing (and previous expectations), this should be reviewed to understand why this happened and what needs to be done to fix the regression issue. Although the area of concern may be unfamiliar, the information logged along with the test implementation should provide enough context to help with this review - in understanding what is being tested, what has gone wrong, and hence be able to reason about how the issue can be resolved. ### You MUST think about the edge cases -Testing for all eventualities helps to write code that can handle errors in a better way. This should include things like testing for illegal arguements, or no arguements to be passed. +Testing for all eventualities helps to write code that can handle errors and exceptions. This should include things like testing for illegal or no arguments, dependencies' edge cases (which can often be simulated through mocking dependencies as appropriate). --- From bcbce6081ea58b78dbd19bb6b9a48aa7c5fff35a Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Fri, 19 Jan 2024 09:57:03 +0000 Subject: [PATCH 25/33] Update docs/standards/developer-testing.md Co-authored-by: Robert Deniszczyc <72561986+robertdeniszczyc2@users.noreply.github.com> --- docs/standards/developer-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index 735a4c6f..a2b783e7 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -22,7 +22,7 @@ The tests should be relatively easy to understand with a track of the overall te A good test should be clear on the intent of the test, have one test case, be readable and should pass every time without changes to the underlying code. -Testing can come in different forms, unit/integration/mutation/property based, but all must adhere to the requirements as the following benefits can be achieved: +Testing can come in different forms, for example unit/integration/mutation/property based, but all must adhere to the requirements as the following benefits can be achieved: - Easy to correct failures in code from failed tests. - Security & confidence when continously integrating code From bc74ce8f1cf17bd4dcbdbb6ea303565cb108c819 Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Fri, 19 Jan 2024 09:57:53 +0000 Subject: [PATCH 26/33] Update docs/standards/developer-testing.md Co-authored-by: Robert Deniszczyc <72561986+robertdeniszczyc2@users.noreply.github.com> --- docs/standards/developer-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index a2b783e7..315cc315 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -52,7 +52,7 @@ Automating tests enables the tests to be run consistently and often. Automated t ### You MUST make tests repeatable -The ability to repeat the same tests gives confidence that if there is a failure, something has changed to affect that failure. The values that a test rely on shouldn't change based on environments, and tests such as unit tests should not depend on other external factors and dependencies (such as third party API, etc) to run. +The ability to repeat the same tests gives confidence that if there is a failure, something has changed to affect that failure. The values that a test rely on shouldn't change based on environments, and tests such as unit tests should not depend on other external factors and dependencies (for example third party APIs) to run. ### You MUST have a way of measuring the effectiveness of testing From 56d9e8edab7329ecc76d30b7f3f1be63c074ad76 Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Fri, 19 Jan 2024 09:58:05 +0000 Subject: [PATCH 27/33] Update docs/standards/developer-testing.md Co-authored-by: Robert Deniszczyc <72561986+robertdeniszczyc2@users.noreply.github.com> --- docs/standards/developer-testing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index 315cc315..7a94e107 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -24,10 +24,10 @@ A good test should be clear on the intent of the test, have one test case, be re Testing can come in different forms, for example unit/integration/mutation/property based, but all must adhere to the requirements as the following benefits can be achieved: -- Easy to correct failures in code from failed tests. +- Easy to correct failures in code from failed tests - Security & confidence when continously integrating code - Easy to understand intent and functionality of code -- Tests are owned by the same team who owns the code and not within a silo external to the team. +- Tests are owned by the same team who owns the code and not within a silo external to the team --- From 50573fe6403d319f4c408e714bcee834434a016e Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Fri, 19 Jan 2024 09:58:16 +0000 Subject: [PATCH 28/33] Update docs/standards/developer-testing.md Co-authored-by: Robert Deniszczyc <72561986+robertdeniszczyc2@users.noreply.github.com> --- docs/standards/developer-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index 7a94e107..3d1efd39 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -58,7 +58,7 @@ The ability to repeat the same tests gives confidence that if there is a failure You need to know whether your test effectiveness has changed over the course of your development and that you haven't had any regressions on your changes. Tracking this effectiveness over time helps to understand the landscape of your testing moving forward. -One way of doing this is through monitoring test coverage, Test coverage helps a team understand the amount of code that is tested within a codebase. Setting a minimum threshold (such as 80%) ensures that any new code is tested. Another example is mutation testing. +One way of doing this is through monitoring test coverage. Test coverage helps a team understand the amount of code that is tested within a codebase. Setting a minimum threshold (such as 80%) ensures that any new code is tested. Another example is mutation testing. ### Tests MUST have a purpose and explicit result From 6b61dfe03e07ddc4d0a2d7b8ff53cfae145a4073 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 19 Jan 2024 11:02:05 +0000 Subject: [PATCH 29/33] Added related links --- docs/standards/developer-testing.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index 3d1efd39..879874b3 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -12,6 +12,8 @@ related: sections: - title: Related links items: + - text: Write maintainable, reusable and evolutionary code + href: /principles/write-maintainable-reusable-and-evolutionary-code/ --- Testing code forms part of the core of developing code. Testing your changes is necessary to gain assurance that they can be deployed safely to production and that the service will still function as expected. From c7f980916f8dfe9150129f840f8643125076892d Mon Sep 17 00:00:00 2001 From: Aaron Russell <128606235+aaronrussellHO@users.noreply.github.com> Date: Fri, 19 Jan 2024 11:51:23 +0000 Subject: [PATCH 30/33] Update docs/standards/developer-testing.md Co-authored-by: Robert Deniszczyc <72561986+robertdeniszczyc2@users.noreply.github.com> --- docs/standards/developer-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index 879874b3..5ffc16d6 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -20,7 +20,7 @@ Testing code forms part of the core of developing code. Testing your changes is The intent of any test should be clear as they can provide a form of documentation to the overall solution. -The tests should be relatively easy to understand with a track of the overall testing coverage kept. Strong test coverage of a code base contributes towards overall quality of application code, although depending on your application context, it may not necessarily be the only definitive marker of quality code. +The tests should be easy to understand with a track of the overall testing coverage kept. Strong test coverage of a code base contributes towards overall quality of application code, although depending on your application context, it may not necessarily be the only definitive marker of quality code. A good test should be clear on the intent of the test, have one test case, be readable and should pass every time without changes to the underlying code. From 8029ec00a70bcb917d1c3bac1dfa555e5cfd0736 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 19 Jan 2024 11:55:37 +0000 Subject: [PATCH 31/33] Removed weak language --- docs/standards/developer-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index 5ffc16d6..eacedd1e 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -44,7 +44,7 @@ Testing can come in different forms, for example unit/integration/mutation/prope ### You MUST test early and often -Writing tests early and often in a software development process tends to reduce the amount of time spent debugging. Good tests written early generally helps improve the overall assurance and design quality of your application. +Writing tests early and often in a software development process reduces the amount of time spent debugging. Good tests written early helps improve the overall assurance and design quality of your application. Test Driven Development (TDD) is a valuable development approach that can help towards achieving this. This development approach encourages you to think ahead on how your requirements can be met by thinking on how your solution can be tested first at unit level. Once a test is written first albeit failing due to non-existing solution, you can implement your solution to pass the test, and then refactor. "Testing early and often" in this case leads to "refactoring early and often", which continuously improves the quality of your application code and design. From 8191218a357d3defd8e1bdb68bc82f5aad837bb7 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 19 Jan 2024 11:58:30 +0000 Subject: [PATCH 32/33] Reworded into to align more with standard --- docs/standards/developer-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index eacedd1e..1b7219d9 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -20,7 +20,7 @@ Testing code forms part of the core of developing code. Testing your changes is The intent of any test should be clear as they can provide a form of documentation to the overall solution. -The tests should be easy to understand with a track of the overall testing coverage kept. Strong test coverage of a code base contributes towards overall quality of application code, although depending on your application context, it may not necessarily be the only definitive marker of quality code. +Tests must be easy to understand with a way of measuring test effectiveness (for example coverage). Strong test coverage of a code base contributes towards overall quality of application code, although depending on your application context, it may not necessarily be the only definitive marker of quality code. A good test should be clear on the intent of the test, have one test case, be readable and should pass every time without changes to the underlying code. From 8f923f3f4a2878b1a919e0b8986a5c1f10207570 Mon Sep 17 00:00:00 2001 From: Aaron Russell Date: Fri, 19 Jan 2024 12:01:26 +0000 Subject: [PATCH 33/33] Added about Red, Green, Refactor approach --- docs/standards/developer-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/standards/developer-testing.md b/docs/standards/developer-testing.md index 1b7219d9..04d9edb1 100644 --- a/docs/standards/developer-testing.md +++ b/docs/standards/developer-testing.md @@ -46,7 +46,7 @@ Testing can come in different forms, for example unit/integration/mutation/prope Writing tests early and often in a software development process reduces the amount of time spent debugging. Good tests written early helps improve the overall assurance and design quality of your application. -Test Driven Development (TDD) is a valuable development approach that can help towards achieving this. This development approach encourages you to think ahead on how your requirements can be met by thinking on how your solution can be tested first at unit level. Once a test is written first albeit failing due to non-existing solution, you can implement your solution to pass the test, and then refactor. "Testing early and often" in this case leads to "refactoring early and often", which continuously improves the quality of your application code and design. +Test Driven Development (TDD) is a valuable development approach that can help towards achieving this. This development approach encourages you to think ahead on how your requirements can be met by thinking on how your solution can be tested first at unit level. Once a test is written first albeit failing due to non-existing solution, you can implement your solution to pass the test, and then refactor. "Testing early and often" in this case leads to "refactoring early and often", which continuously improves the quality of your application code and design. This is more commonly referred to as the Red, Green, Refactor approach. ### You MUST automate tests