adding integration testing

.pylintrc

@@ -307,8 +307,8 @@ min-public-methods=2
 [EXCEPTIONS]
 
 # Exceptions that will emit a warning when caught.
-overgeneral-exceptions=BaseException,
-                       Exception
+overgeneral-exceptions=builtins.BaseException',
+                       builtins.Exception'
 
 
 [FORMAT]

CONTRIBUTING.md

@@ -0,0 +1,18 @@
+# Contributing
+
+* report errors as [issues](https://github.com/ansibleguy/collection_nftables/issues)
+* test unstable modules and [report if they work as expected](https://github.com/ansibleguy/collection_nftables/discussions/new?category=general)
+* add [ansible-based tests](https://github.com/ansibleguy/collection_nftables/blob/latest/tests) for some error-case(s) you have encountered
+* extend or correct the [documentation](https://github.com/ansibleguy/collection_nftables/blob/latest/docs)
+* add missing inline documentation [as standardized](https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_documenting.html#documentation-block)
+  * should be placed in `<COLLECTION>/plugins/module_utils/inline_docs/<MODULE>.py` and then imported in the module
+* contribute code fixes or optimizations
+* implement additional modules
+
+## Module changes
+
+Whenever you change a module's code - you should run lint (`bash scripts/lint.sh`) and [its tests](https://github.com/ansibleguy/collection_nftables/blob/latest/tests/README.md)!
+
+TLDR:
+* Set up a VM or Container
+* Run the Module: `bash scripts/test.sh -e test_module=<MODULE>`

README.md

@@ -17,6 +17,14 @@ See: [Docs](https://nftables.ansibleguy.net)
 
 ----
 
+## Contribute
+
+Feel free to contribute to this project using [pull-requests](https://github.com/ansibleguy/collection_nftables/pulls), [issues](https://github.com/ansibleguy/collection_nftables/issues) and [discussions](https://github.com/ansibleguy/collection_nftables/discussions)!
+
+See also: [Contributing](https://github.com/ansibleguy/collection_nftables/blob/latest/CONTRIBUTING.md)
+
+----
+
 ## Modules
 
 not implemented => development => [testing](https://github.com/ansibleguy/collection_nftables/blob/latest/tests) => unstable (_practical testing_) => stable

plugins/module_utils/check.py

@@ -35,7 +35,7 @@ def check_dependencies() -> None:
 
         rc, _, _ = Nftables().cmd('list ruleset')
         if rc == -1:
-            raise SystemExit(
+            raise PermissionError(
                 'You need to run this module as root '
                 'so it can interact with NFTables!'
             )

plugins/module_utils/check_test.py

@@ -8,5 +8,6 @@
     ('nftables v1.0.6 (Lester Gooch #5)', True),  # extended version
 ])
 def test_version_check(raw_version: str, result: bool):
+    # pylint: disable=C0415
     from ansible_collections.ansibleguy.nftables.plugins.module_utils.check import _validate_version
     assert _validate_version(raw_version) is result

plugins/module_utils/helper/subps_test.py

@@ -6,11 +6,11 @@
     ('some-invalid-command', 1, None),  # soft cmd failure
 ])
 def test_process(cmd: str, rc: int, stdout: str):
+    # pylint: disable=C0415
     from ansible_collections.ansibleguy.nftables.plugins.module_utils.helper.subps import process
     result = process(cmd)
 
     assert result['rc'] == rc
 
     if stdout is not None:
         assert result['stdout'].strip() == stdout
-

plugins/module_utils/nft.py

@@ -15,7 +15,6 @@
 from ansible_collections.ansibleguy.nftables.plugins.module_utils.check import \
     check_dependencies
 
-check_dependencies()
 
 # pylint: disable=C0413
 from nftables import Nftables
@@ -29,6 +28,7 @@ class NFT:
     CHECK_MODE_CMDS = ['list ruleset']
 
     def __init__(self, module: AnsibleModule, result: dict):
+        check_dependencies()
         self.m = module
         self.r = result
         self.n = Nftables()

scripts/test.sh

@@ -1,6 +1,30 @@
 #!/usr/bin/env bash
 
-set -e
+set -eo pipefail
+
+echo ''
+echo '##### PREPARING #####'
 
 cd "$(dirname "$0")/.."
-python3 -m pytest
+COL_DIR="$(pwd)"
+TMP_DIR="/tmp/.nftables_test_$(date +%s)"
+TMP_COL_DIR="${TMP_DIR}/collections"
+
+mkdir -p "${TMP_COL_DIR}/ansible_collections/ansibleguy/"
+ln -s "$COL_DIR" "${TMP_COL_DIR}/ansible_collections/ansibleguy/nftables"
+
+export ANSIBLE_COLLECTIONS_PATH="$TMP_COL_DIR"
+export ANSIBLE_INVENTORY_UNPARSED_WARNING=False
+export ANSIBLE_LOCALHOST_WARNING=False
+cd "${COL_DIR}/tests/"
+
+echo ''
+echo '##### STARTING #####'
+
+ansible-playbook -k -K -i inventory/hosts.yml test.yml "$@"
+
+rm -rf "$TMP_DIR"
+
+echo ''
+echo '##### FINISHED #####'
+echo ''

scripts/unittest.sh

@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+cd "$(dirname "$0")/.."
+python3 -m pytest

tests/README.md

@@ -0,0 +1,70 @@
+# ansibleguy.nftables - Testing
+
+As NFTables behaves differently in containers, we are testing on a Linux VM and a Linux Container (_LXC > Docker_).
+
+Both must be reachable by SSH as **we are using Ansible directly for integration-testing**.
+
+----
+
+## Setup
+
+You can also only set-up one of these test-systems.
+
+But you will have to run Ansible by using the `--limit container` or `--limit vm` argument.
+
+### Virtual Machine
+
+We are using a [Debian 12 minimal]() installation.
+
+For a quick-start you could use [this VirtualBox image](https://sourceforge.net/projects/linuxvmimages/) provided by [linuxvmimages.com](https://www.linuxvmimages.com/images/debian-12/).
+
+### Container
+
+We are using a Debian 12 container.
+
+I would recommend using [a LXC](https://wiki.debian.org/LXC) if you have the needed system for it.
+
+* [Proxmox LXC](https://pve.proxmox.com/wiki/Linux_Container#pct_container_images):
+
+   ```bash
+   pveam update
+   pveam download local debian-12-standard_12.2-1_amd64.tar.zst  # exact version number could vary
+   ```
+
+* [Raw LXC](https://wiki.debian.org/LXC#Container_Creation)
+
+* Docker: `docker pull debian:12`
+
+### Config
+
+Add your test-system's IPs and users to the `inventory/host_vars/*.yml` files.
+
+A NFTables base-config might be added later on.
+
+----
+
+## Add/Modify
+
+When modifying tests you should run the lint-script: `bash scripts/lint.sh`
+
+Tests are placed under: `tests/tasks/` and should be named as the module they are testing.
+
+Example: `tests/tasks/list.yml` is testing `ansibleguy.nftables.list`
+
+Tests should always clean up after itself so the test-system is back to the state it was in before! Add those cleanup-tasks in `tests/tasks/<MODULE>_cleanup.yml`
+
+As the connection over SSH is needed for Ansible to work - tests should never deny/drop this connection.
+
+----
+
+## Run
+
+You can run the tests simply by running the script: `bash scripts/test.sh`
+
+Parameters you add to the test-script execution will be passed to `ansible-playbook`
+
+Examples:
+
+* Enable difference-mode: `bash scripts/test.sh -D`
+* Limit the execution: `bash scripts/test.sh --limit container`
+* Only test one module: `bash scripts/test.sh -e test_module=list`

tests/cleanup.yml

@@ -0,0 +1,29 @@
+---
+
+- name: Testing NFTables Modules - Cleanup
+  hosts: testing
+  become: true
+  gather_facts: true
+  vars:
+    test_module: 'all'
+
+  tasks:
+    - name: Cleanup ansibleguy.nftables.list
+      ansible.builtin.import_tasks: tasks/list_cleanup.yml
+      when: "test_module in ['all', 'list']"
+
+    - name: Cleanup ansibleguy.nftables.table
+      ansible.builtin.import_tasks: tasks/table_cleanup.yml
+      when: "test_module in ['all', 'table']"
+
+    - name: Cleanup ansibleguy.nftables.chain
+      ansible.builtin.import_tasks: tasks/chain_cleanup.yml
+      when: "test_module in ['all', 'chain']"
+
+    - name: Cleanup ansibleguy.nftables.rule_raw
+      ansible.builtin.import_tasks: tasks/rule_raw_cleanup.yml
+      when: "test_module in ['all', 'rule_raw']"
+
+    - name: Cleanup ansibleguy.nftables.rule
+      ansible.builtin.import_tasks: tasks/rule_cleanup.yml
+      when: "test_module in ['all', 'rule']"

tests/inventory/group_vars/all.yml

@@ -0,0 +1 @@
+---

tests/inventory/host_vars/container.yml

@@ -0,0 +1,5 @@
+---
+
+ansible_host: '0.0.0.0'  # ADD YOUR IP
+ansible_user: 'dummy'  # ADD YOUR USER
+ansible_port: 22

tests/inventory/host_vars/vm.yml

@@ -0,0 +1,5 @@
+---
+
+ansible_host: '0.0.0.0'  # ADD YOUR IP
+ansible_user: 'dummy'  # ADD YOUR USER
+ansible_port: 22

tests/inventory/hosts.yml

@@ -0,0 +1,12 @@
+---
+
+all:
+  hosts:
+    container:
+    vm:
+
+  children:
+    testing:
+      hosts:
+        container:
+        vm:

tests/tasks/chain.yml

@@ -0,0 +1 @@
+---

tests/tasks/chain_cleanup.yml

@@ -0,0 +1 @@
+---

tests/tasks/list.yml

@@ -0,0 +1,41 @@
+---
+
+- name: List | Pulling existing Tables
+  ansibleguy.nftables.list:
+    target: 'tables'
+  register: list_tables1
+
+# - ansible.builtin.debug:
+#     var: list_tables1.data
+
+- name: List | Checking existing Tables
+  ansible.builtin.assert:
+    that:
+      - "'data' in list_tables1"
+      - list_tables1.data | length == 1
+
+- name: List | Pulling existing Chains
+  ansibleguy.nftables.list:
+    target: 'chains'
+  register: list_chains1
+
+# - ansible.builtin.debug:
+#     var: list_chains1.data
+
+- name: List | Checking existing Chains
+  ansible.builtin.assert:
+    that:
+      - "'data' in list_chains1"
+
+- name: List | Pulling existing Rules
+  ansibleguy.nftables.list:
+    target: 'rules'
+  register: list_rules1
+
+# - ansible.builtin.debug:
+#     var: list_rules1.data
+
+- name: List | Checking existing Rules
+  ansible.builtin.assert:
+    that:
+      - "'data' in list_rules1"

tests/tasks/list_cleanup.yml

@@ -0,0 +1 @@
+---

tests/tasks/rule.yml

@@ -0,0 +1 @@
+---

tests/tasks/rule_cleanup.yml

@@ -0,0 +1 @@
+---

tests/tasks/rule_raw.yml

@@ -0,0 +1 @@
+---

tests/tasks/rule_raw_cleanup.yml

@@ -0,0 +1 @@
+---

tests/tasks/table.yml

@@ -0,0 +1 @@
+---

tests/tasks/table_cleanup.yml

@@ -0,0 +1 @@
+---

tests/test.yml

@@ -0,0 +1,32 @@
+---
+
+- name: Testing NFTables Modules
+  hosts: testing
+  become: true
+  gather_facts: true
+  vars:
+    test_module: 'all'
+
+  tasks:
+    - name: Testing ansibleguy.nftables.list
+      ansible.builtin.import_tasks: tasks/list.yml
+      when: "test_module in ['all', 'list']"
+
+    - name: Testing ansibleguy.nftables.table
+      ansible.builtin.import_tasks: tasks/table.yml
+      when: "test_module in ['all', 'table']"
+
+    - name: Testing ansibleguy.nftables.chain
+      ansible.builtin.import_tasks: tasks/chain.yml
+      when: "test_module in ['all', 'chain']"
+
+    - name: Testing ansibleguy.nftables.rule_raw
+      ansible.builtin.import_tasks: tasks/rule_raw.yml
+      when: "test_module in ['all', 'rule_raw']"
+
+    - name: Testing ansibleguy.nftables.rule
+      ansible.builtin.import_tasks: tasks/rule.yml
+      when: "test_module in ['all', 'rule']"
+
+- name: Testing NFTables Modules - Cleanup
+  ansible.builtin.import_playbook: cleanup.yml