Merge pull request #281 from best-practice-and-impact/34-update-verif…

…ication-and-validation-for-the-aqua-book

34 update verification and validation for the aqua book

analysis.qmd

@@ -56,14 +56,14 @@ The Approver should be aware of the progress of the analysis and ensure that the
 
 ### Verification and validation
 
-Verification that the implemented methodology meets the intended plan should be incorporated as part the analysis. [Whitener and Balci (1989)](https://typeset.io/pdf/guidelines-for-selecting-and-using-simulation-model-143kzp6h5s.pdf) review verification techniques in relation to simulation modelling, but these techniques extend to analysis more broadly. These include:
+Verification that the implemented methodology meets the design requirements should be incorporated as part the analysis. [Whitener and Balci (1989)](https://typeset.io/pdf/guidelines-for-selecting-and-using-simulation-model-143kzp6h5s.pdf) reviewed verification techniques in relation to simulation modelling. These techniques extend to analysis more broadly. These include:
 
 - Informal analysis: techniques that rely on human reasoning and subjectivity.
 - Static analysis: tests that the implementation of the analysis before it is run. For example, checking that code adheres to code conventions, structural analysis of the code by examining graphs of control and data flows, .
 - Dynamic analysis: tests the behaviour of the system, model or code to find errors that arise during execution. This includes [unit testing](https://en.wikipedia.org/wiki/Unit_testing), [integration testing](https://en.wikipedia.org/wiki/Integration_testing) and [stress testing](https://en.wikipedia.org/wiki/Stress_testing_(computing))
 - Symbolic analysis:  particularly relevant to modelling and tests the transformation of symbolic proxies of model inputs into outputs during the execution of a model. Includes path tracing and cause-effect testing (see [Whitener and Balci (1989)](https://typeset.io/pdf/guidelines-for-selecting-and-using-simulation-model-143kzp6h5s.pdf) )
 - Constraint analysis: particularly relevant to modelling and tests the implementation of constraints during model execution. This includes checking the assertions of the model and boundary analysis.
-- Formal analysis: tests logical correctness through [formal verification](https://en.wikipedia.org/wiki/Formal_verification#Formal_verification_for_software) such as logic or mathematical proofs
+- Formal analysis: tests logical correctness through [formal verification](https://en.wikipedia.org/wiki/Formal_verification#Formal_verification_for_software) such as logic or mathematical proofs.
 
 
 

delivery_and_communication.qmd

@@ -20,15 +20,20 @@ This chapter provides information on the processes around assurance of communica
 
 ### The Commissioner's responsibilities during delivery, communication and sign-off
 
-The Commissioner shall use the analysis as specified at the start of the analytical cycle, applying any limitations to its use as described by the Analyst.
+The Commissioner shall 
+* confirm that the analysis is likely to meet their needs; 
+* use the analysis as specified; and, 
+* understand and apply any limitations to its use.
 
 
 ### The Analyst's responsibilities during delivery, communication and sign-off
 
-- The Analyst shall follow organisational governance procedures for delivery and sign-off, including, where appropriate, updating the [business-critical analysis register](#business-critical-analysis-register), and making the analysis [publicly available](#open-publishing-of-analysis).
-- The Analyst should ensure that communication meets audience requirements, including accessibility.
-- The Analyst should be prepared to respond to challenge from the Approver. For example, this challenge may arise through project or programme boards.
-- The Analyst may be required to communicate the assurance state to the Approver, if not done directly by the Assurer.
+The Analyst 
+- shall follow organisational governance procedures for delivery and sign-off, including, where appropriate, updating the [business-critical analysis register](#business-critical-analysis-register), and making the analysis [publicly available](#open-publishing-of-analysis);
+- shall decribe any limitations to using the analysis; 
+- should ensure that communication meets audience requirements e.g. on accessibility;
+- should be prepared to respond to challenge from the Approver e.g. scruitiny from project or programme boards;
+- may be required to communicate the assurance state to the Approver, if not done directly by the Assurer.
 
 
 ### The Assurer's responsibilities during delivery, communication and sign-off

design.qmd

@@ -80,15 +80,13 @@ In smaller projects, the Approver may not be heavily involved in the design stag
 
 
 
-Designing the assurance activities for subsequent analytical stages is an essential part of the design stage. The design of the assurance should take into account the principles of [proportionality](./proportionality.qmd). Assurance of data quality should be considered in the analytical plan.
-
 On completion of the design stage, the Assurer should be aware of the quality assurance tasks that will be required of them during the project lifetime and have assured the necessary elements of the analytical plan.
 
-The assurance of the design stage should consider whether the analytical plan:  
+The assurance of the design stage should consider whether the analytical plan is likely to:  
 
-* Delivers as intended
-* Adequately addresses the complexities of the customer issue for this purpose 
-* Is well-structured for the purpose, data driven, and reflects a robust overall design.
+* Address commissioner's requirements - validation; 
+* Deliver as intended - verification;
+* Be robust i.e. well-structured, data driven, with a sound overall design.
 
 The assurance of the design stage may be carried out by the Assurer. For more complex analysis, it is good practice to engage subject matter experts to provide independent assurance, and to ensure the accuracy and limitations of the chosen methods are understood, ideally with tests baselining their response against independent reference cases.