diff --git a/CHANGELOG.md b/CHANGELOG.md index bf857ac..58cad6a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,8 @@ +**09/11/2022** [Version 3.6.1] + - Allow defining a set of KPIs per TestCase + - Implement `/execution//kpis` endpoint + - Avoid exception when sending CSVs to InfluxDb on Python 3.11 + **10/10/2022** [Version 3.6.0] - Implemented Child tasks, flow control: diff --git a/docs/2-1_FACILITY_CONFIGURATION.md b/docs/2-1_FACILITY_CONFIGURATION.md index 895eab7..cf64263 100644 --- a/docs/2-1_FACILITY_CONFIGURATION.md +++ b/docs/2-1_FACILITY_CONFIGURATION.md @@ -91,6 +91,9 @@ Distributed: False Dashboard: {} ```` +V2 TestCases can also contain an optional `KPIs` field, which can be used to define a sub-set of results that are +considered *of interest*. See [TestCase Parameters](/docs/2-2_TESTCASE_PARAMETERS.md) for more information. + ##### - V1 TestCase (`Version: 1` or missing) TestCases using this format follow the same approach as for UE files. The following is an example V1 TestCase: diff --git a/docs/2-2_TESTCASE_PARAMETERS.md b/docs/2-2_TESTCASE_PARAMETERS.md index 3de5b7a..37c84e3 100644 --- a/docs/2-2_TESTCASE_PARAMETERS.md +++ b/docs/2-2_TESTCASE_PARAMETERS.md @@ -8,6 +8,19 @@ be added to the yaml description. These keys are: value is set to an empty list ('[]') the test case is considered public and will appear on the list of Custom experiments for all users of the Portal. If the list contains one or more email addresses, the test case will be visible only to the users with matching emails. + - `KPIs`: Optional dictionary that can be used for defining a sub-set of results, from all of the generated by the +TestCase, that can be considered *of interest*. The following is an example of the format: + ```yaml +"KPIs": + ping: # Each key refers to a 'measurement' (table) + - Success # Each key contains a list of 'kpi' (table columns) + - Delay + iPerf: + - Throughput + ``` +These values (as (`measurement`, `kpi`) pairs) can be retrieved, per experiment execution, by using the +`execution//kpis` endpoint. When multiple TestCases are included as part of an experiment the union +of all KPIs (duplicates removed) are returned. - `Parameters`: Dictionary of dictionaries, where each entry is defined as follows: ```yaml "": diff --git a/docs/A1_ENDPOINTS.md b/docs/A1_ENDPOINTS.md index 043dd12..3563e63 100644 --- a/docs/A1_ENDPOINTS.md +++ b/docs/A1_ENDPOINTS.md @@ -46,6 +46,14 @@ Returns a compressed file that includes the logs and all files generated by the Returns a copy of the Experiment Descriptor that was used to define the execution. +### [GET] '/execution//kpis' + +Returns a dictionary with a single `KPIs` key, containing a list of pairs (`measurement`, `kpi`) that are considered of +interest. + +> These values can be used as part of queries to the [Analytics Module](https://github.com/5genesis/Analytics), in order +> to extract a sub-set of important KPIs from all the generated measurements. + ### [DELETE] `/execution/` > *[GET] `/execution//cancel` (Deprecated)*