Added script for duplicate test id and summary check

In order to avoid duplicate test IDs and/or summaries, a
python script has been added and integrated into the CI
to check for this.

Signed-off-by: Michael Engel <[email protected]>

.github/workflows/integration-tests.yml

@@ -90,6 +90,12 @@ jobs:
           cd tests
           tmt test id --dry | grep "New id" && echo "Found integration tests with missing ID. Please generate Test-IDs." && exit 1
           cd ..
+      
+      - name: Check for duplicate test IDs and summaries
+        run: |
+          cd tests
+          ./scripts/duplicate-check.py
+          cd ..
 
       - name: Run integration tests
         run: |

tests/README.md

@@ -201,6 +201,21 @@ New id 'UUID' added to test '/tests/path_to_your_new_test'.
 ...
 ```
 
+### Checking for duplicate test IDs and summaries
+
+In addition to having a unique ID, the summaries of tests should be descriptive and unique as well. An automatic check will be performed by the CI. This check can also be invoked locally:
+
+```shell
+$ cd ~/bluechi/tests
+$ ./scripts/duplicate-check.py
+Duplicate ID '254b1fa2-7c9f-481c-85ac-c6e42c0226c8' detected for tests:
+        /tests/tier0/bluechi-anonymous-node
+        /tests/tier0/bluechi-controller-set-loglevel-invalid
+Duplicate summary detected for tests:
+        /tests/tier0/bluechi-agent-set-loglevel
+        /tests/tier0/bluechi-controller-set-loglevel
+```
+
 ## Usage of containers
 
 The integration tests rely on containers as separate compute entities. These containers are used to simulate BlueChi's

tests/scripts/duplicate-check.py

@@ -0,0 +1,61 @@
+#!/usr/bin/env python3
+#
+# Copyright Contributors to the Eclipse BlueChi project
+#
+# SPDX-License-Identifier: LGPL-2.1-or-later
+
+import os
+import sys
+
+import fmf
+
+
+# TODO:
+#   Remove when
+#   https://github.com/teemtee/tmt/issues/2939
+#   has been resolved and feature is in new release
+def has_duplicate_id(fmf_tree: fmf.Tree):
+    duplicate_ids = dict()
+    for leave in fmf_tree.climb():
+        id = leave.data.get("id")
+        if id is None:
+            continue
+        if id not in duplicate_ids:
+            duplicate_ids[id] = []
+        duplicate_ids[id].append(leave.name)
+
+    exit_code = 0
+    for id, tests in duplicate_ids.items():
+        if len(tests) > 1:
+            print(f"Duplicate ID '{id}' detected for tests:")
+            for test in tests:
+                print(f"\t{test}")
+            exit_code = 1
+    return exit_code
+
+
+def has_duplicate_summary(fmf_tree: fmf.Tree):
+    duplicate_summaries = dict()
+    for leave in fmf_tree.climb():
+        summary = leave.data.get("summary")
+        if summary is None:
+            continue
+        if summary not in duplicate_summaries:
+            duplicate_summaries[summary] = []
+        duplicate_summaries[summary].append(leave.name)
+
+    exit_code = 0
+    for summary, tests in duplicate_summaries.items():
+        if len(tests) > 1:
+            print("Duplicate summary detected for tests:")
+            for test in tests:
+                print(f"\t{test}")
+            exit_code = 1
+    return exit_code
+
+
+if __name__ == "__main__":
+    fmf_tree = fmf.Tree(os.getcwd())
+    res_id = has_duplicate_id(fmf_tree)
+    res_sum = has_duplicate_summary(fmf_tree)
+    sys.exit(res_id + res_sum)