From 06c1eedbb171c25397f803eeb1427143d0a0b658 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 21 Oct 2024 03:25:34 +0000 Subject: [PATCH 001/113] Bump the production-dependencies group with 13 updates Bumps the production-dependencies group with 13 updates: | Package | From | To | | --- | --- | --- | | [azure-keyvault-certificates](https://github.com/Azure/azure-sdk-for-python) | `4.8.0` | `4.9.0` | | [azure-keyvault-keys](https://github.com/Azure/azure-sdk-for-python) | `4.9.0` | `4.10.0` | | [azure-keyvault-secrets](https://github.com/Azure/azure-sdk-for-python) | `4.8.0` | `4.9.0` | | [cryptography](https://github.com/pyca/cryptography) | `43.0.1` | `43.0.3` | | [pulumi-azure-native](https://github.com/pulumi/pulumi-azure-native) | `2.66.0` | `2.67.0` | | [pulumi-azuread](https://github.com/pulumi/pulumi-azuread) | `6.0.0` | `6.0.1` | | [pulumi-random](https://github.com/pulumi/pulumi-random) | `4.16.6` | `4.16.7` | | [pulumi](https://github.com/pulumi/pulumi) | `3.136.1` | `3.137.0` | | [ansible-dev-tools](https://github.com/ansible/ansible-dev-tools) | `24.9.0` | `24.10.0` | | [mypy](https://github.com/python/mypy) | `1.11.2` | `1.12.1` | | [ruff](https://github.com/astral-sh/ruff) | `0.6.9` | `0.7.0` | | [types-requests](https://github.com/python/typeshed) | `2.32.0.20240914` | `2.32.0.20241016` | | [coverage](https://github.com/nedbat/coveragepy) | `7.6.3` | `7.6.4` | Updates `azure-keyvault-certificates` from 4.8.0 to 4.9.0 - [Release notes](https://github.com/Azure/azure-sdk-for-python/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md) - [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-keyvault-certificates_4.8.0...azure-keyvault-certificates_4.9.0) Updates `azure-keyvault-keys` from 4.9.0 to 4.10.0 - [Release notes](https://github.com/Azure/azure-sdk-for-python/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md) - [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-keyvault-keys_4.9.0...azure-keyvault-keys_4.10.0) Updates `azure-keyvault-secrets` from 4.8.0 to 4.9.0 - [Release notes](https://github.com/Azure/azure-sdk-for-python/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md) - [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-keyvault-secrets_4.8.0...azure-keyvault-secrets_4.9.0) Updates `cryptography` from 43.0.1 to 43.0.3 - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](https://github.com/pyca/cryptography/compare/43.0.1...43.0.3) Updates `pulumi-azure-native` from 2.66.0 to 2.67.0 - [Release notes](https://github.com/pulumi/pulumi-azure-native/releases) - [Changelog](https://github.com/pulumi/pulumi-azure-native/blob/master/CHANGELOG_OLD.md) - [Commits](https://github.com/pulumi/pulumi-azure-native/compare/v2.66.0...v2.67.0) Updates `pulumi-azuread` from 6.0.0 to 6.0.1 - [Release notes](https://github.com/pulumi/pulumi-azuread/releases) - [Changelog](https://github.com/pulumi/pulumi-azuread/blob/master/CHANGELOG_OLD.md) - [Commits](https://github.com/pulumi/pulumi-azuread/compare/v6.0.0...v6.0.1) Updates `pulumi-random` from 4.16.6 to 4.16.7 - [Release notes](https://github.com/pulumi/pulumi-random/releases) - [Changelog](https://github.com/pulumi/pulumi-random/blob/master/CHANGELOG_OLD.md) - [Commits](https://github.com/pulumi/pulumi-random/compare/v4.16.6...v4.16.7) Updates `pulumi` from 3.136.1 to 3.137.0 - [Release notes](https://github.com/pulumi/pulumi/releases) - [Changelog](https://github.com/pulumi/pulumi/blob/master/CHANGELOG.md) - [Commits](https://github.com/pulumi/pulumi/compare/v3.136.1...v3.137.0) Updates `ansible-dev-tools` from 24.9.0 to 24.10.0 - [Release notes](https://github.com/ansible/ansible-dev-tools/releases) - [Commits](https://github.com/ansible/ansible-dev-tools/compare/v24.9.0...v24.10.0) Updates `mypy` from 1.11.2 to 1.12.1 - [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md) - [Commits](https://github.com/python/mypy/compare/v1.11.2...v1.12.1) Updates `ruff` from 0.6.9 to 0.7.0 - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](https://github.com/astral-sh/ruff/compare/0.6.9...0.7.0) Updates `types-requests` from 2.32.0.20240914 to 2.32.0.20241016 - [Commits](https://github.com/python/typeshed/commits) Updates `coverage` from 7.6.3 to 7.6.4 - [Release notes](https://github.com/nedbat/coveragepy/releases) - [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst) - [Commits](https://github.com/nedbat/coveragepy/compare/7.6.3...7.6.4) --- updated-dependencies: - dependency-name: azure-keyvault-certificates dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: azure-keyvault-keys dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: azure-keyvault-secrets dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: cryptography dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: pulumi-azure-native dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: pulumi-azuread dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: pulumi-random dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: pulumi dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: ansible-dev-tools dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: mypy dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: ruff dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: types-requests dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: coverage dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] --- pyproject.toml | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index b12c64a3ed..1f396fa9eb 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -27,9 +27,9 @@ dependencies = [ "appdirs==1.4.4", "azure-core==1.31.0", "azure-identity==1.19.0", - "azure-keyvault-certificates==4.8.0", - "azure-keyvault-keys==4.9.0", - "azure-keyvault-secrets==4.8.0", + "azure-keyvault-certificates==4.9.0", + "azure-keyvault-keys==4.10.0", + "azure-keyvault-secrets==4.9.0", "azure-mgmt-compute==33.0.0", "azure-mgmt-containerinstance==10.1.0", "azure-mgmt-dns==8.1.0", @@ -42,13 +42,13 @@ dependencies = [ "azure-storage-file-datalake==12.17.0", "azure-storage-file-share==12.19.0", "chevron==0.14.0", - "cryptography==43.0.1", + "cryptography==43.0.3", "fqdn==1.5.1", "psycopg[binary]==3.2.3", - "pulumi-azure-native==2.66.0", - "pulumi-azuread==6.0.0", - "pulumi-random==4.16.6", - "pulumi==3.136.1", + "pulumi-azure-native==2.67.0", + "pulumi-azuread==6.0.1", + "pulumi-random==4.16.7", + "pulumi==3.137.0", "pydantic==2.9.2", "pyjwt[crypto]==2.9.0", "pytz==2024.2", @@ -73,21 +73,21 @@ docs = [ "sphinx==8.1.3", ] lint = [ - "ansible-dev-tools==24.9.0", + "ansible-dev-tools==24.10.0", "ansible==10.5.0", "black==24.10.0", - "mypy==1.11.2", + "mypy==1.12.1", "pandas-stubs==2.2.3.241009", "pydantic==2.9.2", - "ruff==0.6.9", + "ruff==0.7.0", "types-appdirs==1.4.3.5", "types-chevron==0.14.2.20240310", "types-pytz==2024.2.0.20241003", "types-pyyaml==6.0.12.20240917", - "types-requests==2.32.0.20240914", + "types-requests==2.32.0.20241016", ] test = [ - "coverage==7.6.3", + "coverage==7.6.4", "freezegun==1.5.1", "pytest-mock==3.14.0", "pytest==8.3.3", From 74afa8d34bab2e7587e185fa330f9faceda9e01b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" Date: Mon, 21 Oct 2024 03:32:13 +0000 Subject: [PATCH 002/113] [dependabot skip] :wrench: Update Python requirements files --- .hatch/requirements-docs.txt | 2 +- .hatch/requirements-lint.txt | 28 +++++++++++++------------- .hatch/requirements-test.txt | 38 ++++++++++++++++++------------------ .hatch/requirements.txt | 32 +++++++++++++++--------------- 4 files changed, 50 insertions(+), 50 deletions(-) diff --git a/.hatch/requirements-docs.txt b/.hatch/requirements-docs.txt index e4ac178af1..3f9d070931 100644 --- a/.hatch/requirements-docs.txt +++ b/.hatch/requirements-docs.txt @@ -42,7 +42,7 @@ markdown-it-py==3.0.0 # via # mdit-py-plugins # myst-parser -markupsafe==3.0.1 +markupsafe==3.0.2 # via jinja2 mdit-py-plugins==0.4.2 # via myst-parser diff --git a/.hatch/requirements-lint.txt b/.hatch/requirements-lint.txt index 7ae53e8876..d8dac4d24d 100644 --- a/.hatch/requirements-lint.txt +++ b/.hatch/requirements-lint.txt @@ -1,18 +1,18 @@ # # This file is autogenerated by hatch-pip-compile with Python 3.12 # -# - ansible-dev-tools==24.9.0 +# - ansible-dev-tools==24.10.0 # - ansible==10.5.0 # - black==24.10.0 -# - mypy==1.11.2 +# - mypy==1.12.1 # - pandas-stubs==2.2.3.241009 # - pydantic==2.9.2 -# - ruff==0.6.9 +# - ruff==0.7.0 # - types-appdirs==1.4.3.5 # - types-chevron==0.14.2.20240310 # - types-pytz==2024.2.0.20241003 # - types-pyyaml==6.0.12.20240917 -# - types-requests==2.32.0.20240914 +# - types-requests==2.32.0.20241016 # annotated-types==0.7.0 @@ -36,11 +36,11 @@ ansible-core==2.17.5 # ansible-lint # molecule # pytest-ansible -ansible-creator==24.10.0 +ansible-creator==24.9.0 # via ansible-dev-tools ansible-dev-environment==24.9.0 # via ansible-dev-tools -ansible-dev-tools==24.9.0 +ansible-dev-tools==24.10.0 # via hatch.envs.lint ansible-lint==24.9.2 # via @@ -81,7 +81,7 @@ click-help-colors==0.9.4 # via molecule colorama==0.4.6 # via tox -cryptography==43.0.1 +cryptography==43.0.3 # via ansible-core distlib==0.3.9 # via @@ -123,13 +123,13 @@ lockfile==0.12.2 # via python-daemon markdown-it-py==3.0.0 # via rich -markupsafe==3.0.1 +markupsafe==3.0.2 # via jinja2 mdurl==0.1.2 # via markdown-it-py molecule==24.9.0 # via ansible-dev-tools -mypy==1.11.2 +mypy==1.12.1 # via hatch.envs.lint mypy-extensions==1.0.0 # via @@ -233,16 +233,16 @@ rpds-py==0.20.0 # referencing ruamel-yaml==0.18.6 # via ansible-lint -ruamel-yaml-clib==0.2.8 +ruamel-yaml-clib==0.2.12 # via ruamel-yaml -ruff==0.6.9 +ruff==0.7.0 # via hatch.envs.lint subprocess-tee==0.4.2 # via # ansible-compat # ansible-dev-environment # ansible-lint -tox==4.21.2 +tox==4.23.0 # via tox-ansible tox-ansible==24.9.0 # via ansible-dev-tools @@ -256,7 +256,7 @@ types-pytz==2024.2.0.20241003 # pandas-stubs types-pyyaml==6.0.12.20240917 # via hatch.envs.lint -types-requests==2.32.0.20240914 +types-requests==2.32.0.20241016 # via hatch.envs.lint typing-extensions==4.12.2 # via @@ -267,7 +267,7 @@ tzdata==2024.2 # via ansible-navigator urllib3==2.2.3 # via types-requests -virtualenv==20.26.6 +virtualenv==20.27.0 # via tox wcmatch==10.0 # via diff --git a/.hatch/requirements-test.txt b/.hatch/requirements-test.txt index 7c2d561d5c..bb30ef91be 100644 --- a/.hatch/requirements-test.txt +++ b/.hatch/requirements-test.txt @@ -1,14 +1,14 @@ # # This file is autogenerated by hatch-pip-compile with Python 3.12 # -# [constraints] .hatch/requirements.txt (SHA256: 5fcd366dd7384a272955f8ae7546b65d5821df6f7381f315ec51998aee075503) +# [constraints] .hatch/requirements.txt (SHA256: b2e3d05a82b17891a9634de500f2ed665a5398f6f87734e27557b404b187a6e5) # # - appdirs==1.4.4 # - azure-core==1.31.0 # - azure-identity==1.19.0 -# - azure-keyvault-certificates==4.8.0 -# - azure-keyvault-keys==4.9.0 -# - azure-keyvault-secrets==4.8.0 +# - azure-keyvault-certificates==4.9.0 +# - azure-keyvault-keys==4.10.0 +# - azure-keyvault-secrets==4.9.0 # - azure-mgmt-compute==33.0.0 # - azure-mgmt-containerinstance==10.1.0 # - azure-mgmt-dns==8.1.0 @@ -21,13 +21,13 @@ # - azure-storage-file-datalake==12.17.0 # - azure-storage-file-share==12.19.0 # - chevron==0.14.0 -# - cryptography==43.0.1 +# - cryptography==43.0.3 # - fqdn==1.5.1 # - psycopg[binary]==3.2.3 -# - pulumi-azure-native==2.66.0 -# - pulumi-azuread==6.0.0 -# - pulumi-random==4.16.6 -# - pulumi==3.136.1 +# - pulumi-azure-native==2.67.0 +# - pulumi-azuread==6.0.1 +# - pulumi-random==4.16.7 +# - pulumi==3.137.0 # - pydantic==2.9.2 # - pyjwt[crypto]==2.9.0 # - pytz==2024.2 @@ -36,7 +36,7 @@ # - simple-acme-dns==3.1.0 # - typer==0.12.5 # - websocket-client==1.8.0 -# - coverage==7.6.3 +# - coverage==7.6.4 # - freezegun==1.5.1 # - pytest-mock==3.14.0 # - pytest==8.3.3 @@ -91,15 +91,15 @@ azure-identity==1.19.0 # via # -c .hatch/requirements.txt # hatch.envs.test -azure-keyvault-certificates==4.8.0 +azure-keyvault-certificates==4.9.0 # via # -c .hatch/requirements.txt # hatch.envs.test -azure-keyvault-keys==4.9.0 +azure-keyvault-keys==4.10.0 # via # -c .hatch/requirements.txt # hatch.envs.test -azure-keyvault-secrets==4.8.0 +azure-keyvault-secrets==4.9.0 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -180,9 +180,9 @@ click==8.1.7 # via # -c .hatch/requirements.txt # typer -coverage==7.6.3 +coverage==7.6.4 # via hatch.envs.test -cryptography==43.0.1 +cryptography==43.0.3 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -295,22 +295,22 @@ psycopg-binary==3.2.3 # via # -c .hatch/requirements.txt # psycopg -pulumi==3.136.1 +pulumi==3.137.0 # via # -c .hatch/requirements.txt # hatch.envs.test # pulumi-azure-native # pulumi-azuread # pulumi-random -pulumi-azure-native==2.66.0 +pulumi-azure-native==2.67.0 # via # -c .hatch/requirements.txt # hatch.envs.test -pulumi-azuread==6.0.0 +pulumi-azuread==6.0.1 # via # -c .hatch/requirements.txt # hatch.envs.test -pulumi-random==4.16.6 +pulumi-random==4.16.7 # via # -c .hatch/requirements.txt # hatch.envs.test diff --git a/.hatch/requirements.txt b/.hatch/requirements.txt index e2e683b43c..0be5d3ee49 100644 --- a/.hatch/requirements.txt +++ b/.hatch/requirements.txt @@ -4,9 +4,9 @@ # - appdirs==1.4.4 # - azure-core==1.31.0 # - azure-identity==1.19.0 -# - azure-keyvault-certificates==4.8.0 -# - azure-keyvault-keys==4.9.0 -# - azure-keyvault-secrets==4.8.0 +# - azure-keyvault-certificates==4.9.0 +# - azure-keyvault-keys==4.10.0 +# - azure-keyvault-secrets==4.9.0 # - azure-mgmt-compute==33.0.0 # - azure-mgmt-containerinstance==10.1.0 # - azure-mgmt-dns==8.1.0 @@ -19,13 +19,13 @@ # - azure-storage-file-datalake==12.17.0 # - azure-storage-file-share==12.19.0 # - chevron==0.14.0 -# - cryptography==43.0.1 +# - cryptography==43.0.3 # - fqdn==1.5.1 # - psycopg[binary]==3.2.3 -# - pulumi-azure-native==2.66.0 -# - pulumi-azuread==6.0.0 -# - pulumi-random==4.16.6 -# - pulumi==3.136.1 +# - pulumi-azure-native==2.67.0 +# - pulumi-azuread==6.0.1 +# - pulumi-random==4.16.7 +# - pulumi==3.137.0 # - pydantic==2.9.2 # - pyjwt[crypto]==2.9.0 # - pytz==2024.2 @@ -70,11 +70,11 @@ azure-core==1.31.0 # msrest azure-identity==1.19.0 # via hatch.envs.default -azure-keyvault-certificates==4.8.0 +azure-keyvault-certificates==4.9.0 # via hatch.envs.default -azure-keyvault-keys==4.9.0 +azure-keyvault-keys==4.10.0 # via hatch.envs.default -azure-keyvault-secrets==4.8.0 +azure-keyvault-secrets==4.9.0 # via hatch.envs.default azure-mgmt-compute==33.0.0 # via hatch.envs.default @@ -122,7 +122,7 @@ chevron==0.14.0 # via hatch.envs.default click==8.1.7 # via typer -cryptography==43.0.1 +cryptography==43.0.3 # via # hatch.envs.default # acme @@ -192,17 +192,17 @@ psycopg==3.2.3 # via hatch.envs.default psycopg-binary==3.2.3 # via psycopg -pulumi==3.136.1 +pulumi==3.137.0 # via # hatch.envs.default # pulumi-azure-native # pulumi-azuread # pulumi-random -pulumi-azure-native==2.66.0 +pulumi-azure-native==2.67.0 # via hatch.envs.default -pulumi-azuread==6.0.0 +pulumi-azuread==6.0.1 # via hatch.envs.default -pulumi-random==4.16.6 +pulumi-random==4.16.7 # via hatch.envs.default pycparser==2.22 # via cffi From b09e683e54b78d6edbf4586394d11447ff9422ab Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Mon, 21 Oct 2024 16:01:14 +0100 Subject: [PATCH 003/113] Allow data providers address to be 'Internet' --- data_safe_haven/config/config_sections.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/data_safe_haven/config/config_sections.py b/data_safe_haven/config/config_sections.py index 35b9570a7e..62bfec0833 100644 --- a/data_safe_haven/config/config_sections.py +++ b/data_safe_haven/config/config_sections.py @@ -57,7 +57,7 @@ class ConfigSectionSRE(BaseModel, validate_assignment=True): admin_email_address: EmailAddress admin_ip_addresses: list[IpAddress] = [] databases: UniqueList[DatabaseSystem] = [] - data_provider_ip_addresses: list[IpAddress] = [] + data_provider_ip_addresses: list[IpAddress] | AzureServiceTag = [] remote_desktop: ConfigSubsectionRemoteDesktopOpts research_user_ip_addresses: list[IpAddress] | AzureServiceTag = [] storage_quota_gb: ConfigSubsectionStorageQuotaGB @@ -67,8 +67,6 @@ class ConfigSectionSRE(BaseModel, validate_assignment=True): @field_validator( "admin_ip_addresses", - "data_provider_ip_addresses", - # "research_user_ip_addresses", mode="after", ) @classmethod @@ -81,6 +79,7 @@ def ensure_non_overlapping(cls, v: list[IpAddress]) -> list[IpAddress]: return v @field_validator( + "data_provider_ip_addresses", "research_user_ip_addresses", mode="after", ) From 413a2cc5932795fd816874d2ad70ff3f44943b17 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Mon, 21 Oct 2024 16:09:52 +0100 Subject: [PATCH 004/113] Add tests --- tests/config/test_config_sections.py | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/tests/config/test_config_sections.py b/tests/config/test_config_sections.py index 6528b130fa..7d9a0ba873 100644 --- a/tests/config/test_config_sections.py +++ b/tests/config/test_config_sections.py @@ -170,6 +170,24 @@ def test_all_databases_must_be_unique(self) -> None: databases=[DatabaseSystem.POSTGRESQL, DatabaseSystem.POSTGRESQL], ) + def test_data_provider_tag_internet( + self, + config_subsection_remote_desktop: ConfigSubsectionRemoteDesktopOpts, + config_subsection_storage_quota_gb: ConfigSubsectionStorageQuotaGB, + ): + sre_config = ConfigSectionSRE( + admin_email_address="admin@example.com", + remote_desktop=config_subsection_remote_desktop, + storage_quota_gb=config_subsection_storage_quota_gb, + data_provider_ip_addresses="Internet", + ) + assert isinstance(sre_config.data_provider_ip_addresses, AzureServiceTag) + assert sre_config.data_provider_ip_addresses == "Internet" + + def test_data_provider_tag_invalid(self): + with pytest.raises(ValueError, match="Input should be 'Internet'"): + ConfigSectionSRE(data_provider_ip_addresses="Not a tag") + def test_ip_overlap_admin(self): with pytest.raises(ValueError, match="IP addresses must not overlap."): ConfigSectionSRE( From 6f17ab0d7062f3a37c09ff4b9b0ea8822c33b124 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Mon, 21 Oct 2024 16:20:32 +0100 Subject: [PATCH 005/113] Correct ip address validator message --- data_safe_haven/validators/validators.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data_safe_haven/validators/validators.py b/data_safe_haven/validators/validators.py index 27507d26b4..dd4458ec57 100644 --- a/data_safe_haven/validators/validators.py +++ b/data_safe_haven/validators/validators.py @@ -124,7 +124,7 @@ def ip_address(ip_address: str) -> str: try: return str(ipaddress.ip_network(ip_address)) except Exception as exc: - msg = "Expected valid IPv4 address, for example '1.1.1.1', or 'Internet'." + msg = "Expected valid IPv4 address, for example '1.1.1.1'." raise ValueError(msg) from exc From 00aa68bf3477bce2b98db5fb3433eb051ed817a5 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Mon, 21 Oct 2024 16:29:41 +0100 Subject: [PATCH 006/113] Fix validator tests --- tests/validators/test_validators.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/validators/test_validators.py b/tests/validators/test_validators.py index 18d2fd31b5..c8447ab441 100644 --- a/tests/validators/test_validators.py +++ b/tests/validators/test_validators.py @@ -111,7 +111,7 @@ def test_ip_address(self, ip_address, output): def test_ip_address_fail(self, ip_address): with pytest.raises( ValueError, - match="Expected valid IPv4 address, for example '1.1.1.1', or 'Internet'.", + match="Expected valid IPv4 address, for example '1.1.1.1'.", ): validators.ip_address(ip_address) From 2cef813f74e0069fea9a5feb0421fb7991833e5a Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Wed, 23 Oct 2024 10:26:59 +0100 Subject: [PATCH 007/113] Bump pulumi-azure-native --- .hatch/requirements-lint.txt | 4 ++-- .hatch/requirements-test.txt | 6 +++--- .hatch/requirements.txt | 4 ++-- pyproject.toml | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/.hatch/requirements-lint.txt b/.hatch/requirements-lint.txt index d8dac4d24d..71c2c133cb 100644 --- a/.hatch/requirements-lint.txt +++ b/.hatch/requirements-lint.txt @@ -222,7 +222,7 @@ referencing==0.35.1 # jsonschema-specifications resolvelib==1.0.1 # via ansible-core -rich==13.9.2 +rich==13.9.3 # via # ansible-lint # enrich @@ -242,7 +242,7 @@ subprocess-tee==0.4.2 # ansible-compat # ansible-dev-environment # ansible-lint -tox==4.23.0 +tox==4.23.2 # via tox-ansible tox-ansible==24.9.0 # via ansible-dev-tools diff --git a/.hatch/requirements-test.txt b/.hatch/requirements-test.txt index bb30ef91be..ce51600060 100644 --- a/.hatch/requirements-test.txt +++ b/.hatch/requirements-test.txt @@ -1,7 +1,7 @@ # # This file is autogenerated by hatch-pip-compile with Python 3.12 # -# [constraints] .hatch/requirements.txt (SHA256: b2e3d05a82b17891a9634de500f2ed665a5398f6f87734e27557b404b187a6e5) +# [constraints] .hatch/requirements.txt (SHA256: 02ec09f022011bea4adeff6755ed03b7c3c1c847be7bcff73b746968aa5579a0) # # - appdirs==1.4.4 # - azure-core==1.31.0 @@ -24,7 +24,7 @@ # - cryptography==43.0.3 # - fqdn==1.5.1 # - psycopg[binary]==3.2.3 -# - pulumi-azure-native==2.67.0 +# - pulumi-azure-native==2.68.0 # - pulumi-azuread==6.0.1 # - pulumi-random==4.16.7 # - pulumi==3.137.0 @@ -302,7 +302,7 @@ pulumi==3.137.0 # pulumi-azure-native # pulumi-azuread # pulumi-random -pulumi-azure-native==2.67.0 +pulumi-azure-native==2.68.0 # via # -c .hatch/requirements.txt # hatch.envs.test diff --git a/.hatch/requirements.txt b/.hatch/requirements.txt index 0be5d3ee49..07020f85d4 100644 --- a/.hatch/requirements.txt +++ b/.hatch/requirements.txt @@ -22,7 +22,7 @@ # - cryptography==43.0.3 # - fqdn==1.5.1 # - psycopg[binary]==3.2.3 -# - pulumi-azure-native==2.67.0 +# - pulumi-azure-native==2.68.0 # - pulumi-azuread==6.0.1 # - pulumi-random==4.16.7 # - pulumi==3.137.0 @@ -198,7 +198,7 @@ pulumi==3.137.0 # pulumi-azure-native # pulumi-azuread # pulumi-random -pulumi-azure-native==2.67.0 +pulumi-azure-native==2.68.0 # via hatch.envs.default pulumi-azuread==6.0.1 # via hatch.envs.default diff --git a/pyproject.toml b/pyproject.toml index 1f396fa9eb..cd7dd999b0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -45,7 +45,7 @@ dependencies = [ "cryptography==43.0.3", "fqdn==1.5.1", "psycopg[binary]==3.2.3", - "pulumi-azure-native==2.67.0", + "pulumi-azure-native==2.68.0", "pulumi-azuread==6.0.1", "pulumi-random==4.16.7", "pulumi==3.137.0", From 423ffc5e081eaee7dcb179d1708050f3840a844d Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Wed, 23 Oct 2024 15:19:57 +0000 Subject: [PATCH 008/113] Prevent SHM teardown if SRE is deployed --- data_safe_haven/infrastructure/programs/imperative_shm.py | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/data_safe_haven/infrastructure/programs/imperative_shm.py b/data_safe_haven/infrastructure/programs/imperative_shm.py index 9b748bbdd1..0af2e6f542 100644 --- a/data_safe_haven/infrastructure/programs/imperative_shm.py +++ b/data_safe_haven/infrastructure/programs/imperative_shm.py @@ -1,4 +1,4 @@ -from data_safe_haven.config import Context, SHMConfig +from data_safe_haven.config import Context, SHMConfig, DSHPulumiConfig from data_safe_haven.exceptions import ( DataSafeHavenAzureError, DataSafeHavenMicrosoftGraphError, @@ -172,6 +172,12 @@ def teardown(self) -> None: DataSafeHavenAzureError if any resources cannot be destroyed """ logger = get_logger() + pulumi_config = DSHPulumiConfig.from_remote(self.context) + deployed = pulumi_config.project_names + if deployed: + logger.info(f"Found deployed Pulumi SREs: {deployed}.") + msg = f"Deployed SREs must be torn down before the SHM can be torn down." + raise DataSafeHavenAzureError(msg) try: logger.info( f"Removing [green]{self.context.description}[/] resource group {self.context.resource_group_name}." From 600f931de2587175c02e250c388c78d69d6ea34d Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Wed, 23 Oct 2024 15:21:21 +0000 Subject: [PATCH 009/113] Fix linting --- data_safe_haven/infrastructure/programs/imperative_shm.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/infrastructure/programs/imperative_shm.py b/data_safe_haven/infrastructure/programs/imperative_shm.py index 0af2e6f542..7c8eab3f98 100644 --- a/data_safe_haven/infrastructure/programs/imperative_shm.py +++ b/data_safe_haven/infrastructure/programs/imperative_shm.py @@ -1,4 +1,4 @@ -from data_safe_haven.config import Context, SHMConfig, DSHPulumiConfig +from data_safe_haven.config import Context, DSHPulumiConfig, SHMConfig from data_safe_haven.exceptions import ( DataSafeHavenAzureError, DataSafeHavenMicrosoftGraphError, @@ -176,7 +176,7 @@ def teardown(self) -> None: deployed = pulumi_config.project_names if deployed: logger.info(f"Found deployed Pulumi SREs: {deployed}.") - msg = f"Deployed SREs must be torn down before the SHM can be torn down." + msg = "Deployed SREs must be torn down before the SHM can be torn down." raise DataSafeHavenAzureError(msg) try: logger.info( From 0b4bd25bca3d8cc023e7df9ed9348851ef2c0395 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Thu, 24 Oct 2024 13:33:09 +0100 Subject: [PATCH 010/113] docs: update @JimMadge as a contributor --- .all-contributorsrc | 3 ++- README.md | 2 +- docs/source/contributing/index.md | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/.all-contributorsrc b/.all-contributorsrc index 20c0996d58..c25c29e8cd 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -75,7 +75,8 @@ "question", "review", "security", - "test" + "test", + "talk" ] }, { diff --git a/README.md b/README.md index 622f76facd..42085c21aa 100644 --- a/README.md +++ b/README.md @@ -73,7 +73,7 @@ See our [Code of Conduct](CODE_OF_CONDUCT.md) and our [Contributor Guide](CONTRI James Robinson
James Robinson
💻 🖋 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ 📢 - Jim Madge
Jim Madge
💻 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ + Jim Madge
Jim Madge
💻 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ 📢 Josh Everett
Josh Everett
🐛 Jules M
Jules M
📖 🤔 🐛 🖋 Kirstie Whitaker
Kirstie Whitaker
🖋 📖 🐛 🤔 🔍 📋 📆 📣 📢 📓 diff --git a/docs/source/contributing/index.md b/docs/source/contributing/index.md index 37f5e26f9d..1d1f6bcd92 100644 --- a/docs/source/contributing/index.md +++ b/docs/source/contributing/index.md @@ -36,7 +36,7 @@ James Robinson
James Robinson
💻 🖋 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ 📢 - Jim Madge
Jim Madge
💻 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ + Jim Madge
Jim Madge
💻 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ 📢 Josh Everett
Josh Everett
🐛 Jules M
Jules M
📖 🤔 🐛 🖋 Kirstie Whitaker
Kirstie Whitaker
🖋 📖 🐛 🤔 🔍 📋 📆 📣 📢 📓 From c6ea61a6f6d3344d6217ce72b362ecd506585c81 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Thu, 24 Oct 2024 13:36:25 +0100 Subject: [PATCH 011/113] docs: update @cptanalatriste as a contributor --- .all-contributorsrc | 3 ++- README.md | 2 +- docs/source/contributing/index.md | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/.all-contributorsrc b/.all-contributorsrc index c25c29e8cd..2a87b4be43 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -579,7 +579,8 @@ "avatar_url": "https://avatars.githubusercontent.com/u/1616531?v=4", "profile": "https://carlos.gavidia.me/", "contributions": [ - "bug" + "bug", + "ideas" ] }, { diff --git a/README.md b/README.md index 42085c21aa..51d8d71a23 100644 --- a/README.md +++ b/README.md @@ -47,7 +47,7 @@ See our [Code of Conduct](CODE_OF_CONDUCT.md) and our [Contributor Guide](CONTRI Alvaro Cabrejas Egea
Alvaro Cabrejas Egea
💻 🖋 Callum Mole
Callum Mole
🐛 💻 - Carlos Gavidia-Calderon
Carlos Gavidia-Calderon
🐛 + Carlos Gavidia-Calderon
Carlos Gavidia-Calderon
🐛 🤔 Catalina Vallejos
Catalina Vallejos
🖋 Christopher Edsall
Christopher Edsall
💻 📖 🐛 DDelbarre
DDelbarre
🐛 diff --git a/docs/source/contributing/index.md b/docs/source/contributing/index.md index 1d1f6bcd92..1f2034d7dd 100644 --- a/docs/source/contributing/index.md +++ b/docs/source/contributing/index.md @@ -10,7 +10,7 @@ Alvaro Cabrejas Egea
Alvaro Cabrejas Egea
💻 🖋 Callum Mole
Callum Mole
🐛 💻 - Carlos Gavidia-Calderon
Carlos Gavidia-Calderon
🐛 + Carlos Gavidia-Calderon
Carlos Gavidia-Calderon
🐛 🤔 Catalina Vallejos
Catalina Vallejos
🖋 Christopher Edsall
Christopher Edsall
💻 📖 🐛 DDelbarre
DDelbarre
🐛 From 5f6298778f54ea68554a556dee517aa47132341f Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Thu, 24 Oct 2024 13:36:51 +0100 Subject: [PATCH 012/113] docs: update @helendduncan as a contributor --- .all-contributorsrc | 3 ++- README.md | 2 +- docs/source/contributing/index.md | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/.all-contributorsrc b/.all-contributorsrc index 2a87b4be43..d52f1df237 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -590,7 +590,8 @@ "profile": "https://github.com/helendduncan", "contributions": [ "bug", - "review" + "review", + "ideas" ] }, { diff --git a/README.md b/README.md index 51d8d71a23..e9936d3bad 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ See our [Code of Conduct](CODE_OF_CONDUCT.md) and our [Contributor Guide](CONTRI Guillaume Noell
Guillaume Noell
📖 🐛 🤔 - Helen D Little
Helen D Little
🐛 👀 + Helen D Little
Helen D Little
🐛 👀 🤔 Helen Sherwood-Taylor
Helen Sherwood-Taylor
🤔 🖋 Jack Roberts
Jack Roberts
💻 🐛 James Cunningham
James Cunningham
💻 📖 🐛 🤔 🖋 diff --git a/docs/source/contributing/index.md b/docs/source/contributing/index.md index 1f2034d7dd..043c0e9506 100644 --- a/docs/source/contributing/index.md +++ b/docs/source/contributing/index.md @@ -27,7 +27,7 @@ Guillaume Noell
Guillaume Noell
📖 🐛 🤔 - Helen D Little
Helen D Little
🐛 👀 + Helen D Little
Helen D Little
🐛 👀 🤔 Helen Sherwood-Taylor
Helen Sherwood-Taylor
🤔 🖋 Jack Roberts
Jack Roberts
💻 🐛 James Cunningham
James Cunningham
💻 📖 🐛 🤔 🖋 From d2830de1c3fb2349e9c6a56fa56277dbe2ac0cc4 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Thu, 24 Oct 2024 13:37:11 +0100 Subject: [PATCH 013/113] docs: update @dsj976 as a contributor --- .all-contributorsrc | 3 ++- README.md | 2 +- docs/source/contributing/index.md | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/.all-contributorsrc b/.all-contributorsrc index d52f1df237..7be5bd0da9 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -610,7 +610,8 @@ "profile": "https://github.com/dsj976", "contributions": [ "bug", - "doc" + "doc", + "ideas" ] }, { diff --git a/README.md b/README.md index e9936d3bad..edba208fa4 100644 --- a/README.md +++ b/README.md @@ -55,7 +55,7 @@ See our [Code of Conduct](CODE_OF_CONDUCT.md) and our [Contributor Guide](CONTRI David Beavan
David Beavan
📖 🖋 - David Salvador Jasin
David Salvador Jasin
🐛 📖 + David Salvador Jasin
David Salvador Jasin
🐛 📖 🤔 Diego Arenas
Diego Arenas
💻 🤔 🖋 Ed Chalstrey
Ed Chalstrey
💻 📖 🐛 🤔 📋 👀 ⚠️ Evelina Gabasova
Evelina Gabasova
🖋 diff --git a/docs/source/contributing/index.md b/docs/source/contributing/index.md index 043c0e9506..40e74bf53f 100644 --- a/docs/source/contributing/index.md +++ b/docs/source/contributing/index.md @@ -18,7 +18,7 @@ David Beavan
David Beavan
📖 🖋 - David Salvador Jasin
David Salvador Jasin
🐛 📖 + David Salvador Jasin
David Salvador Jasin
🐛 📖 🤔 Diego Arenas
Diego Arenas
💻 🤔 🖋 Ed Chalstrey
Ed Chalstrey
💻 📖 🐛 🤔 📋 👀 ⚠️ Evelina Gabasova
Evelina Gabasova
🖋 From 22901cd4c6a8e39c275046a3c9354b8c89a97766 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Thu, 24 Oct 2024 14:10:51 +0100 Subject: [PATCH 014/113] Update requirements files --- .hatch/requirements-lint.txt | 2 +- .hatch/requirements-test.txt | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/.hatch/requirements-lint.txt b/.hatch/requirements-lint.txt index cf536a5fc9..71c2c133cb 100644 --- a/.hatch/requirements-lint.txt +++ b/.hatch/requirements-lint.txt @@ -36,7 +36,7 @@ ansible-core==2.17.5 # ansible-lint # molecule # pytest-ansible -ansible-creator==24.10.1 +ansible-creator==24.9.0 # via ansible-dev-tools ansible-dev-environment==24.9.0 # via ansible-dev-tools diff --git a/.hatch/requirements-test.txt b/.hatch/requirements-test.txt index 75d3ee6341..8e331e9c35 100644 --- a/.hatch/requirements-test.txt +++ b/.hatch/requirements-test.txt @@ -1,6 +1,7 @@ # # This file is autogenerated by hatch-pip-compile with Python 3.12 # +# [constraints] .hatch/requirements.txt (SHA256: e9ada52b8f6ba0b8f072b9c1a47b577eeb19f863ee4e75afc8a24ad948bf35eb) # # - appdirs==1.4.4 # - azure-core==1.31.0 From 3ec9cee3a072d366d8f4fbfb331b0cf2c9af5d2e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 24 Oct 2024 13:26:42 +0000 Subject: [PATCH 015/113] Bump the production-dependencies group with 5 updates Bumps the production-dependencies group with 5 updates: | Package | From | To | | --- | --- | --- | | [azure-mgmt-dns](https://github.com/Azure/azure-sdk-for-python) | `8.1.0` | `8.2.0` | | [azure-mgmt-resource](https://github.com/Azure/azure-sdk-for-python) | `23.1.1` | `23.2.0` | | [rich](https://github.com/Textualize/rich) | `13.9.2` | `13.9.3` | | [pydata-sphinx-theme](https://github.com/pydata/pydata-sphinx-theme) | `0.15.4` | `0.16.0` | | [mypy](https://github.com/python/mypy) | `1.12.1` | `1.13.0` | Updates `azure-mgmt-dns` from 8.1.0 to 8.2.0 - [Release notes](https://github.com/Azure/azure-sdk-for-python/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md) - [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-mgmt-dns_8.1.0...azure-mgmt-dns_8.2.0) Updates `azure-mgmt-resource` from 23.1.1 to 23.2.0 - [Release notes](https://github.com/Azure/azure-sdk-for-python/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md) - [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-mgmt-resource_23.1.1...azure-mgmt-resource_23.2.0) Updates `rich` from 13.9.2 to 13.9.3 - [Release notes](https://github.com/Textualize/rich/releases) - [Changelog](https://github.com/Textualize/rich/blob/master/CHANGELOG.md) - [Commits](https://github.com/Textualize/rich/compare/v13.9.2...v13.9.3) Updates `pydata-sphinx-theme` from 0.15.4 to 0.16.0 - [Release notes](https://github.com/pydata/pydata-sphinx-theme/releases) - [Changelog](https://github.com/pydata/pydata-sphinx-theme/blob/main/RELEASE.md) - [Commits](https://github.com/pydata/pydata-sphinx-theme/compare/v0.15.4...v0.16.0) Updates `mypy` from 1.12.1 to 1.13.0 - [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md) - [Commits](https://github.com/python/mypy/compare/v1.12.1...v1.13.0) --- updated-dependencies: - dependency-name: azure-mgmt-dns dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: azure-mgmt-resource dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: rich dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: pydata-sphinx-theme dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: mypy dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] --- pyproject.toml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index 97461b4d32..50d2ad2378 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -32,11 +32,11 @@ dependencies = [ "azure-keyvault-secrets==4.9.0", "azure-mgmt-compute==33.0.0", "azure-mgmt-containerinstance==10.1.0", - "azure-mgmt-dns==8.1.0", + "azure-mgmt-dns==8.2.0", "azure-mgmt-keyvault==10.3.1", "azure-mgmt-msi==7.0.0", "azure-mgmt-rdbms==10.1.0", - "azure-mgmt-resource==23.1.1", + "azure-mgmt-resource==23.2.0", "azure-mgmt-storage==21.2.1", "azure-storage-blob==12.23.1", "azure-storage-file-datalake==12.17.0", @@ -53,7 +53,7 @@ dependencies = [ "pyjwt[crypto]==2.9.0", "pytz==2024.2", "pyyaml==6.0.2", - "rich==13.9.2", + "rich==13.9.3", "simple-acme-dns==3.1.0", "typer==0.12.5", "websocket-client==1.8.0", @@ -68,7 +68,7 @@ Source = "https://github.com/alan-turing-institute/data-safe-haven" docs = [ "emoji==2.14.0", "myst-parser==4.0.0", - "pydata-sphinx-theme==0.15.4", + "pydata-sphinx-theme==0.16.0", "sphinx-togglebutton==0.3.2", "sphinx==8.1.3", ] @@ -76,7 +76,7 @@ lint = [ "ansible-dev-tools==24.10.0", "ansible==10.5.0", "black==24.10.0", - "mypy==1.12.1", + "mypy==1.13.0", "pandas-stubs==2.2.3.241009", "pydantic==2.9.2", "ruff==0.7.0", From 0bbbe42722ec7a0a4d716a84f6d7a902838d8b92 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" Date: Thu, 24 Oct 2024 13:33:28 +0000 Subject: [PATCH 016/113] [dependabot skip] :wrench: Update Python requirements files --- .hatch/requirements-docs.txt | 8 +++----- .hatch/requirements-lint.txt | 4 ++-- .hatch/requirements-test.txt | 16 +++++++++------- .hatch/requirements.txt | 14 ++++++++------ 4 files changed, 22 insertions(+), 20 deletions(-) diff --git a/.hatch/requirements-docs.txt b/.hatch/requirements-docs.txt index 3f9d070931..d93aeb3bbf 100644 --- a/.hatch/requirements-docs.txt +++ b/.hatch/requirements-docs.txt @@ -3,7 +3,7 @@ # # - emoji==2.14.0 # - myst-parser==4.0.0 -# - pydata-sphinx-theme==0.15.4 +# - pydata-sphinx-theme==0.16.0 # - sphinx-togglebutton==0.3.2 # - sphinx==8.1.3 # @@ -51,10 +51,8 @@ mdurl==0.1.2 myst-parser==4.0.0 # via hatch.envs.docs packaging==24.1 - # via - # pydata-sphinx-theme - # sphinx -pydata-sphinx-theme==0.15.4 + # via sphinx +pydata-sphinx-theme==0.16.0 # via hatch.envs.docs pygments==2.18.0 # via diff --git a/.hatch/requirements-lint.txt b/.hatch/requirements-lint.txt index 71c2c133cb..969e9c3b47 100644 --- a/.hatch/requirements-lint.txt +++ b/.hatch/requirements-lint.txt @@ -4,7 +4,7 @@ # - ansible-dev-tools==24.10.0 # - ansible==10.5.0 # - black==24.10.0 -# - mypy==1.12.1 +# - mypy==1.13.0 # - pandas-stubs==2.2.3.241009 # - pydantic==2.9.2 # - ruff==0.7.0 @@ -129,7 +129,7 @@ mdurl==0.1.2 # via markdown-it-py molecule==24.9.0 # via ansible-dev-tools -mypy==1.12.1 +mypy==1.13.0 # via hatch.envs.lint mypy-extensions==1.0.0 # via diff --git a/.hatch/requirements-test.txt b/.hatch/requirements-test.txt index 8e331e9c35..f224befaa0 100644 --- a/.hatch/requirements-test.txt +++ b/.hatch/requirements-test.txt @@ -1,7 +1,7 @@ # # This file is autogenerated by hatch-pip-compile with Python 3.12 # -# [constraints] .hatch/requirements.txt (SHA256: e9ada52b8f6ba0b8f072b9c1a47b577eeb19f863ee4e75afc8a24ad948bf35eb) +# [constraints] .hatch/requirements.txt (SHA256: 7ad151976b72bb564d491ec5c0ead3b24dcd4d259b99700df8e92cc8a0cd07ed) # # - appdirs==1.4.4 # - azure-core==1.31.0 @@ -11,11 +11,11 @@ # - azure-keyvault-secrets==4.9.0 # - azure-mgmt-compute==33.0.0 # - azure-mgmt-containerinstance==10.1.0 -# - azure-mgmt-dns==8.1.0 +# - azure-mgmt-dns==8.2.0 # - azure-mgmt-keyvault==10.3.1 # - azure-mgmt-msi==7.0.0 # - azure-mgmt-rdbms==10.1.0 -# - azure-mgmt-resource==23.1.1 +# - azure-mgmt-resource==23.2.0 # - azure-mgmt-storage==21.2.1 # - azure-storage-blob==12.23.1 # - azure-storage-file-datalake==12.17.0 @@ -32,7 +32,7 @@ # - pyjwt[crypto]==2.9.0 # - pytz==2024.2 # - pyyaml==6.0.2 -# - rich==13.9.2 +# - rich==13.9.3 # - simple-acme-dns==3.1.0 # - typer==0.12.5 # - websocket-client==1.8.0 @@ -122,7 +122,7 @@ azure-mgmt-core==1.4.0 # azure-mgmt-rdbms # azure-mgmt-resource # azure-mgmt-storage -azure-mgmt-dns==8.1.0 +azure-mgmt-dns==8.2.0 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -138,7 +138,7 @@ azure-mgmt-rdbms==10.1.0 # via # -c .hatch/requirements.txt # hatch.envs.test -azure-mgmt-resource==23.1.1 +azure-mgmt-resource==23.2.0 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -378,7 +378,7 @@ requests-oauthlib==2.0.0 # via # -c .hatch/requirements.txt # msrest -rich==13.9.2 +rich==13.9.3 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -417,7 +417,9 @@ typing-extensions==4.12.2 # azure-keyvault-keys # azure-keyvault-secrets # azure-mgmt-compute + # azure-mgmt-dns # azure-mgmt-keyvault + # azure-mgmt-resource # azure-storage-blob # azure-storage-file-datalake # azure-storage-file-share diff --git a/.hatch/requirements.txt b/.hatch/requirements.txt index 7354e9da99..e9e357bfc0 100644 --- a/.hatch/requirements.txt +++ b/.hatch/requirements.txt @@ -9,11 +9,11 @@ # - azure-keyvault-secrets==4.9.0 # - azure-mgmt-compute==33.0.0 # - azure-mgmt-containerinstance==10.1.0 -# - azure-mgmt-dns==8.1.0 +# - azure-mgmt-dns==8.2.0 # - azure-mgmt-keyvault==10.3.1 # - azure-mgmt-msi==7.0.0 # - azure-mgmt-rdbms==10.1.0 -# - azure-mgmt-resource==23.1.1 +# - azure-mgmt-resource==23.2.0 # - azure-mgmt-storage==21.2.1 # - azure-storage-blob==12.23.1 # - azure-storage-file-datalake==12.17.0 @@ -30,7 +30,7 @@ # - pyjwt[crypto]==2.9.0 # - pytz==2024.2 # - pyyaml==6.0.2 -# - rich==13.9.2 +# - rich==13.9.3 # - simple-acme-dns==3.1.0 # - typer==0.12.5 # - websocket-client==1.8.0 @@ -90,7 +90,7 @@ azure-mgmt-core==1.4.0 # azure-mgmt-rdbms # azure-mgmt-resource # azure-mgmt-storage -azure-mgmt-dns==8.1.0 +azure-mgmt-dns==8.2.0 # via hatch.envs.default azure-mgmt-keyvault==10.3.1 # via hatch.envs.default @@ -98,7 +98,7 @@ azure-mgmt-msi==7.0.0 # via hatch.envs.default azure-mgmt-rdbms==10.1.0 # via hatch.envs.default -azure-mgmt-resource==23.1.1 +azure-mgmt-resource==23.2.0 # via hatch.envs.default azure-mgmt-storage==21.2.1 # via hatch.envs.default @@ -240,7 +240,7 @@ requests==2.32.3 # requests-oauthlib requests-oauthlib==2.0.0 # via msrest -rich==13.9.2 +rich==13.9.3 # via # hatch.envs.default # typer @@ -268,7 +268,9 @@ typing-extensions==4.12.2 # azure-keyvault-keys # azure-keyvault-secrets # azure-mgmt-compute + # azure-mgmt-dns # azure-mgmt-keyvault + # azure-mgmt-resource # azure-storage-blob # azure-storage-file-datalake # azure-storage-file-share From 640ab2a50ed8151a8d6271c41d5f8e19a2b62033 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Fri, 25 Oct 2024 10:17:35 +0100 Subject: [PATCH 017/113] :wrench: Update contributors names --- .all-contributorsrc | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/.all-contributorsrc b/.all-contributorsrc index 7be5bd0da9..a36e18670c 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -152,7 +152,7 @@ }, { "login": "miguelmorin", - "name": "miguelmorin", + "name": "Miguel Morin", "avatar_url": "https://avatars.githubusercontent.com/u/32396311?v=4", "profile": "https://github.com/miguelmorin", "contributions": [ @@ -164,7 +164,7 @@ }, { "login": "cathiest", - "name": "cathiest", + "name": "Catherine Lawrence", "avatar_url": "https://avatars.githubusercontent.com/u/38755168?v=4", "profile": "https://github.com/cathiest", "contributions": [ @@ -176,7 +176,7 @@ }, { "login": "bw-faststream", - "name": "bw-faststream", + "name": "Benjamin Walden", "avatar_url": "https://avatars.githubusercontent.com/u/54804128?v=4", "profile": "https://github.com/bw-faststream", "contributions": [ @@ -189,7 +189,7 @@ }, { "login": "oforrest", - "name": "oforrest", + "name": "Oliver Forrest", "avatar_url": "https://avatars.githubusercontent.com/u/49275282?v=4", "profile": "https://github.com/oforrest", "contributions": [ @@ -218,7 +218,7 @@ }, { "login": "warwick26", - "name": "warwick26", + "name": "Warwick Wood", "avatar_url": "https://avatars.githubusercontent.com/u/33690673?v=4", "profile": "https://github.com/warwick26", "contributions": [ @@ -258,7 +258,7 @@ }, { "login": "ens-george-holmes", - "name": "ens-george-holmes", + "name": "George Holmes", "avatar_url": "https://avatars.githubusercontent.com/u/62715301?v=4", "profile": "https://github.com/ens-george-holmes", "contributions": [ @@ -292,7 +292,7 @@ }, { "login": "rwinstanley1", - "name": "rwinstanley1", + "name": "Rachel Winstanley", "avatar_url": "https://avatars.githubusercontent.com/u/56362072?v=4", "profile": "https://github.com/rwinstanley1", "contributions": [ @@ -304,7 +304,7 @@ }, { "login": "sysdan", - "name": "Daniel", + "name": "Daniel Allen", "avatar_url": "https://avatars.githubusercontent.com/u/49038294?v=4", "profile": "https://github.com/sysdan", "contributions": [ @@ -324,7 +324,7 @@ }, { "login": "harisood", - "name": "harisood", + "name": "Hari Sood", "avatar_url": "https://avatars.githubusercontent.com/u/67151373?v=4", "profile": "https://github.com/harisood", "contributions": [ @@ -343,7 +343,7 @@ }, { "login": "getcarter21", - "name": "getcarter21", + "name": "Ian Carter", "avatar_url": "https://avatars.githubusercontent.com/u/34555297?v=4", "profile": "https://github.com/getcarter21", "contributions": [ @@ -364,7 +364,7 @@ }, { "login": "ens-brett-todd", - "name": "ens-brett-todd", + "name": "Brett Todd", "avatar_url": "https://avatars.githubusercontent.com/u/62715658?v=4", "profile": "https://github.com/ens-brett-todd", "contributions": [ @@ -385,7 +385,7 @@ }, { "login": "kevinxufs", - "name": "kevinxufs", + "name": "Kevin Xu", "avatar_url": "https://avatars.githubusercontent.com/u/48526846?v=4", "profile": "https://github.com/kevinxufs", "contributions": [ @@ -396,7 +396,7 @@ }, { "login": "vollmersj", - "name": "vollmersj", + "name": "Sebastian Vollmer", "avatar_url": "https://avatars.githubusercontent.com/u/12613127?v=4", "profile": "https://github.com/vollmersj", "contributions": [ @@ -418,7 +418,7 @@ }, { "login": "JulesMarz", - "name": "Jules M", + "name": "Jules Manser", "avatar_url": "https://avatars.githubusercontent.com/u/40864686?v=4", "profile": "https://github.com/JulesMarz", "contributions": [ @@ -484,7 +484,7 @@ }, { "login": "DDelbarre", - "name": "DDelbarre", + "name": "Daniel Delbarre", "avatar_url": "https://avatars.githubusercontent.com/u/108824056?v=4", "profile": "https://github.com/DDelbarre", "contributions": [ @@ -549,7 +549,7 @@ }, { "login": "Arielle-Bennett", - "name": "arielle-bennett", + "name": "Arielle Bennett", "avatar_url": "https://avatars.githubusercontent.com/u/74651964?v=4", "profile": "https://github.com/Arielle-Bennett", "contributions": [ @@ -560,7 +560,7 @@ }, { "login": "Davsarper", - "name": "davsarper", + "name": "David Sarmiento Perez", "avatar_url": "https://avatars.githubusercontent.com/u/118986872?v=4", "profile": "https://github.com/Davsarper", "contributions": [ @@ -585,7 +585,7 @@ }, { "login": "helendduncan", - "name": "Helen D Little", + "name": "Helen Duncan Little", "avatar_url": "https://avatars.githubusercontent.com/u/46891265?v=4", "profile": "https://github.com/helendduncan", "contributions": [ @@ -616,7 +616,7 @@ }, { "login": "mattwestby", - "name": "mattwestby", + "name": "Matt Westby", "avatar_url": "https://avatars.githubusercontent.com/u/91054185?v=4", "profile": "https://github.com/mattwestby", "contributions": [ From c438588f6b9fbe9172c6b972c7f4af5e40f5d002 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Fri, 25 Oct 2024 11:35:38 +0100 Subject: [PATCH 018/113] Update tables --- README.md | 54 +++++++++++++++---------------- docs/source/contributing/index.md | 54 +++++++++++++++---------------- 2 files changed, 54 insertions(+), 54 deletions(-) diff --git a/README.md b/README.md index edba208fa4..79659201f0 100644 --- a/README.md +++ b/README.md @@ -46,69 +46,69 @@ See our [Code of Conduct](CODE_OF_CONDUCT.md) and our [Contributor Guide](CONTRI Alvaro Cabrejas Egea
Alvaro Cabrejas Egea
💻 🖋 + Arielle Bennett
Arielle Bennett
🔍 🤔 📆 + Benjamin Walden
Benjamin Walden
📖 🤔 🐛 📆 📓 + Brett Todd
Brett Todd
💻 🤔 Callum Mole
Callum Mole
🐛 💻 Carlos Gavidia-Calderon
Carlos Gavidia-Calderon
🐛 🤔 Catalina Vallejos
Catalina Vallejos
🖋 - Christopher Edsall
Christopher Edsall
💻 📖 🐛 - DDelbarre
DDelbarre
🐛 - Daniel
Daniel
💻 🐛 + Catherine Lawrence
Catherine Lawrence
🖋 📖 🐛 🤔 + Christopher Edsall
Christopher Edsall
💻 📖 🐛 + Daniel Allen
Daniel Allen
💻 🐛 + Daniel Delbarre
Daniel Delbarre
🐛 David Beavan
David Beavan
📖 🖋 David Salvador Jasin
David Salvador Jasin
🐛 📖 🤔 + David Sarmiento Perez
David Sarmiento Perez
📖 📋 🔍 🤔 📆 📣 📢 + + Diego Arenas
Diego Arenas
💻 🤔 🖋 Ed Chalstrey
Ed Chalstrey
💻 📖 🐛 🤔 📋 👀 ⚠️ Evelina Gabasova
Evelina Gabasova
🖋 Federico Nanni
Federico Nanni
💻 🐛 📖 🤔 Franz Király
Franz Király
🖋 + George Holmes
George Holmes
💻 🤔 + Guillaume Noell
Guillaume Noell
📖 🐛 🤔 - Guillaume Noell
Guillaume Noell
📖 🐛 🤔 - Helen D Little
Helen D Little
🐛 👀 🤔 + Hari Sood
Hari Sood
📖 🐛 🤔 🔍 📋 📆 📣 💬 📢 🛡️ 📓 + Helen Duncan Little
Helen Duncan Little
🐛 👀 🤔 Helen Sherwood-Taylor
Helen Sherwood-Taylor
🤔 🖋 + Ian Carter
Ian Carter
💻 🖋 Jack Roberts
Jack Roberts
💻 🐛 James Cunningham
James Cunningham
💻 📖 🐛 🤔 🖋 James Geddes
James Geddes
🖋 - James Hetherington
James Hetherington
📖 🐛 🤔 🔍 📆 📣 📢 🖋 + James Hetherington
James Hetherington
📖 🐛 🤔 🔍 📆 📣 📢 🖋 James Robinson
James Robinson
💻 🖋 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ 📢 Jim Madge
Jim Madge
💻 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ 📢 Josh Everett
Josh Everett
🐛 - Jules M
Jules M
📖 🤔 🐛 🖋 + Jules Manser
Jules Manser
📖 🤔 🐛 🖋 + Kevin Xu
Kevin Xu
📖 🤔 🛡️ Kirstie Whitaker
Kirstie Whitaker
🖋 📖 🐛 🤔 🔍 📋 📆 📣 📢 📓 + + Martin O'Reilly
Martin O'Reilly
💻 🖋 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ 📢 Matt Craddock
Matt Craddock
💻 📖 🐛 🤔 🔍 📋 📣 💬 👀 🛡️ ⚠️ + Matt Westby
Matt Westby
🐛 + Miguel Morin
Miguel Morin
💻 📖 🤔 ⚠️ + Oliver Forrest
Oliver Forrest
📖 🤔 📆 📣 🖋 + Oscar T Giles
Oscar T Giles
💻 📖 🤔 + Rachel Winstanley
Rachel Winstanley
📖 🤔 📆 🛡️ - Oscar T Giles
Oscar T Giles
💻 📖 🤔 Radka Jersakova
Radka Jersakova
🖋 Rob Clarke
Rob Clarke
🤔 🐛 💻 📖 🖋 + Sebastian Vollmer
Sebastian Vollmer
📖 🐛 🤔 🖋 Steven Carlysle-Davies
Steven Carlysle-Davies
💻 🖋 🤔 Tim Hobson
Tim Hobson
💻 🐛 📖 🤔 Tom Doel
Tom Doel
💻 📖 🐛 🤔 🖋 Tomas Lazauskas
Tomas Lazauskas
💻 📖 🐛 🤔 - arielle-bennett
arielle-bennett
🔍 🤔 📆 - bw-faststream
bw-faststream
📖 🤔 🐛 📆 📓 - cathiest
cathiest
🖋 📖 🐛 🤔 - davsarper
davsarper
📖 📋 🔍 🤔 📆 📣 📢 - ens-brett-todd
ens-brett-todd
💻 🤔 - ens-george-holmes
ens-george-holmes
💻 🤔 - getcarter21
getcarter21
💻 🖋 - - - harisood
harisood
📖 🐛 🤔 🔍 📋 📆 📣 💬 📢 🛡️ 📓 - kevinxufs
kevinxufs
📖 🤔 🛡️ - mattwestby
mattwestby
🐛 - miguelmorin
miguelmorin
💻 📖 🤔 ⚠️ - oforrest
oforrest
📖 🤔 📆 📣 🖋 - rwinstanley1
rwinstanley1
📖 🤔 📆 🛡️ - vollmersj
vollmersj
📖 🐛 🤔 🖋 - - - warwick26
warwick26
💻 🤔 + Warwick Wood
Warwick Wood
💻 🤔 diff --git a/docs/source/contributing/index.md b/docs/source/contributing/index.md index 40e74bf53f..e5f99fd3a0 100644 --- a/docs/source/contributing/index.md +++ b/docs/source/contributing/index.md @@ -9,69 +9,69 @@ Alvaro Cabrejas Egea
Alvaro Cabrejas Egea
💻 🖋 + Arielle Bennett
Arielle Bennett
🔍 🤔 📆 + Benjamin Walden
Benjamin Walden
📖 🤔 🐛 📆 📓 + Brett Todd
Brett Todd
💻 🤔 Callum Mole
Callum Mole
🐛 💻 Carlos Gavidia-Calderon
Carlos Gavidia-Calderon
🐛 🤔 Catalina Vallejos
Catalina Vallejos
🖋 - Christopher Edsall
Christopher Edsall
💻 📖 🐛 - DDelbarre
DDelbarre
🐛 - Daniel
Daniel
💻 🐛 + Catherine Lawrence
Catherine Lawrence
🖋 📖 🐛 🤔 + Christopher Edsall
Christopher Edsall
💻 📖 🐛 + Daniel Allen
Daniel Allen
💻 🐛 + Daniel Delbarre
Daniel Delbarre
🐛 David Beavan
David Beavan
📖 🖋 David Salvador Jasin
David Salvador Jasin
🐛 📖 🤔 + David Sarmiento Perez
David Sarmiento Perez
📖 📋 🔍 🤔 📆 📣 📢 + + Diego Arenas
Diego Arenas
💻 🤔 🖋 Ed Chalstrey
Ed Chalstrey
💻 📖 🐛 🤔 📋 👀 ⚠️ Evelina Gabasova
Evelina Gabasova
🖋 Federico Nanni
Federico Nanni
💻 🐛 📖 🤔 Franz Király
Franz Király
🖋 + George Holmes
George Holmes
💻 🤔 + Guillaume Noell
Guillaume Noell
📖 🐛 🤔 - Guillaume Noell
Guillaume Noell
📖 🐛 🤔 - Helen D Little
Helen D Little
🐛 👀 🤔 + Hari Sood
Hari Sood
📖 🐛 🤔 🔍 📋 📆 📣 💬 📢 🛡️ 📓 + Helen Duncan Little
Helen Duncan Little
🐛 👀 🤔 Helen Sherwood-Taylor
Helen Sherwood-Taylor
🤔 🖋 + Ian Carter
Ian Carter
💻 🖋 Jack Roberts
Jack Roberts
💻 🐛 James Cunningham
James Cunningham
💻 📖 🐛 🤔 🖋 James Geddes
James Geddes
🖋 - James Hetherington
James Hetherington
📖 🐛 🤔 🔍 📆 📣 📢 🖋 + James Hetherington
James Hetherington
📖 🐛 🤔 🔍 📆 📣 📢 🖋 James Robinson
James Robinson
💻 🖋 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ 📢 Jim Madge
Jim Madge
💻 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ 📢 Josh Everett
Josh Everett
🐛 - Jules M
Jules M
📖 🤔 🐛 🖋 + Jules Manser
Jules Manser
📖 🤔 🐛 🖋 + Kevin Xu
Kevin Xu
📖 🤔 🛡️ Kirstie Whitaker
Kirstie Whitaker
🖋 📖 🐛 🤔 🔍 📋 📆 📣 📢 📓 + + Martin O'Reilly
Martin O'Reilly
💻 🖋 📖 🐛 🤔 🔍 📋 🚇 📆 📣 💬 👀 🛡️ ⚠️ 📢 Matt Craddock
Matt Craddock
💻 📖 🐛 🤔 🔍 📋 📣 💬 👀 🛡️ ⚠️ + Matt Westby
Matt Westby
🐛 + Miguel Morin
Miguel Morin
💻 📖 🤔 ⚠️ + Oliver Forrest
Oliver Forrest
📖 🤔 📆 📣 🖋 + Oscar T Giles
Oscar T Giles
💻 📖 🤔 + Rachel Winstanley
Rachel Winstanley
📖 🤔 📆 🛡️ - Oscar T Giles
Oscar T Giles
💻 📖 🤔 Radka Jersakova
Radka Jersakova
🖋 Rob Clarke
Rob Clarke
🤔 🐛 💻 📖 🖋 + Sebastian Vollmer
Sebastian Vollmer
📖 🐛 🤔 🖋 Steven Carlysle-Davies
Steven Carlysle-Davies
💻 🖋 🤔 Tim Hobson
Tim Hobson
💻 🐛 📖 🤔 Tom Doel
Tom Doel
💻 📖 🐛 🤔 🖋 Tomas Lazauskas
Tomas Lazauskas
💻 📖 🐛 🤔 - arielle-bennett
arielle-bennett
🔍 🤔 📆 - bw-faststream
bw-faststream
📖 🤔 🐛 📆 📓 - cathiest
cathiest
🖋 📖 🐛 🤔 - davsarper
davsarper
📖 📋 🔍 🤔 📆 📣 📢 - ens-brett-todd
ens-brett-todd
💻 🤔 - ens-george-holmes
ens-george-holmes
💻 🤔 - getcarter21
getcarter21
💻 🖋 - - - harisood
harisood
📖 🐛 🤔 🔍 📋 📆 📣 💬 📢 🛡️ 📓 - kevinxufs
kevinxufs
📖 🤔 🛡️ - mattwestby
mattwestby
🐛 - miguelmorin
miguelmorin
💻 📖 🤔 ⚠️ - oforrest
oforrest
📖 🤔 📆 📣 🖋 - rwinstanley1
rwinstanley1
📖 🤔 📆 🛡️ - vollmersj
vollmersj
📖 🐛 🤔 🖋 - - - warwick26
warwick26
💻 🤔 + Warwick Wood
Warwick Wood
💻 🤔 From 6b70d0b832857f501d3de78e02f36a853b06592c Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 28 Oct 2024 03:55:26 +0000 Subject: [PATCH 019/113] Bump ruff from 0.7.0 to 0.7.1 in the production-dependencies group Bumps the production-dependencies group with 1 update: [ruff](https://github.com/astral-sh/ruff). Updates `ruff` from 0.7.0 to 0.7.1 - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](https://github.com/astral-sh/ruff/compare/0.7.0...0.7.1) --- updated-dependencies: - dependency-name: ruff dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] --- pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index 50d2ad2378..8bcd84a1ca 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -79,7 +79,7 @@ lint = [ "mypy==1.13.0", "pandas-stubs==2.2.3.241009", "pydantic==2.9.2", - "ruff==0.7.0", + "ruff==0.7.1", "types-appdirs==1.4.3.5", "types-chevron==0.14.2.20240310", "types-pytz==2024.2.0.20241003", From c309fc520493a7bb763280c5e69c765b5c020ff1 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" Date: Mon, 28 Oct 2024 04:02:11 +0000 Subject: [PATCH 020/113] [dependabot skip] :wrench: Update Python requirements files --- .hatch/requirements-lint.txt | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/.hatch/requirements-lint.txt b/.hatch/requirements-lint.txt index 969e9c3b47..4bd0234a71 100644 --- a/.hatch/requirements-lint.txt +++ b/.hatch/requirements-lint.txt @@ -7,7 +7,7 @@ # - mypy==1.13.0 # - pandas-stubs==2.2.3.241009 # - pydantic==2.9.2 -# - ruff==0.7.0 +# - ruff==0.7.1 # - types-appdirs==1.4.3.5 # - types-chevron==0.14.2.20240310 # - types-pytz==2024.2.0.20241003 @@ -89,8 +89,6 @@ distlib==0.3.9 # virtualenv distro==1.9.0 # via bindep -docutils==0.21.2 - # via python-daemon enrich==1.2.7 # via molecule execnet==2.1.1 @@ -199,7 +197,7 @@ pytest-ansible==24.9.0 # tox-ansible pytest-xdist==3.6.1 # via tox-ansible -python-daemon==3.0.1 +python-daemon==3.1.0 # via ansible-runner python-gnupg==0.5.3 # via ansible-sign @@ -235,7 +233,7 @@ ruamel-yaml==0.18.6 # via ansible-lint ruamel-yaml-clib==0.2.12 # via ruamel-yaml -ruff==0.7.0 +ruff==0.7.1 # via hatch.envs.lint subprocess-tee==0.4.2 # via From 621707dc24070ac365d4ed326ab13c966812cada Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 13:32:30 +0000 Subject: [PATCH 021/113] update example config yaml --- docs/source/deployment/deploy_sre.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index 2a1e5511a7..e6ada6fc42 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -46,6 +46,7 @@ $ dsh config template --file PATH_YOU_WANT_TO_SAVE_YOUR_YAML_FILE_TO \ :::{code} yaml azure: + location: # Azure location where SRE resources will be deployed subscription_id: # ID of the Azure subscription that the TRE will be deployed to tenant_id: # Home tenant for the Azure account used to deploy infrastructure: `az account show` description: # A free-text description of your SRE deployment @@ -61,8 +62,13 @@ sre: remote_desktop: allow_copy: # True/False: whether to allow copying text out of the environment allow_paste: # True/False: whether to allow pasting text into the environment - research_user_ip_addresses: # List of IP addresses belonging to users + research_user_ip_addresses: + - # List of IP addresses belonging to users + - # You can also use the tag 'Internet' instead of a list software_packages: # Which Python/R packages to allow users to install: [any/pre-approved/none] + storage_quota_gb: + home: # Total size in GiB across all home directories . + shared: #Total size in GiB for the shared directories . timezone: # Timezone in pytz format (eg. Europe/London) workspace_skus: # List of Azure VM SKUs that will be used for data analysis. ::: From 519744f3dbc08b460e455707d4ef861d91a81d5a Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 13:44:13 +0000 Subject: [PATCH 022/113] Reorganize deployment page and add details about copy and paste --- docs/source/deployment/deploy_sre.md | 32 ++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index e6ada6fc42..c0fdb99540 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -75,14 +75,18 @@ sre: :::: -:::{admonition} Supported Azure regions -:class: dropdown important +### Configuration guidance + +#### Choosing an Azure region Some of the SRE resources are not available in all Azure regions. - Workspace virtual machines use zone redundant storage managed disks which have [limited regional availability](https://learn.microsoft.com/en-us/azure/virtual-machines/disks-redundancy). - Some shares mounted on workspace virtual machines require premium file shares which have [limited regional availability](https://learn.microsoft.com/en-us/azure/storage/files/redundancy-premium-file-shares). +:::{admonition} Supported Azure regions +:class: dropdown important + The regions which satisfy all requirements are, - Australia East @@ -117,6 +121,8 @@ The regions which satisfy all requirements are, ::: +#### Choosing a VM SKU + :::{hint} See [here](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/) for a full list of valid Azure VM SKUs. ::: @@ -170,6 +176,28 @@ As some general recommendations, ::: +### Copy and paste + +The [Guacamole clipboard](https://guacamole.apache.org/doc/gug/using-guacamole.html#using-the-clipboard) provides an interface between the local clipboard and the clipboard on the remote workspaces. +Only text is allowed to be passed through the Guacamole clipboard. + +Copy and pasting of text to or from SRE workspaces via the Guacamole clipboard can be configured with the `allow_copy` and `allow_paste` settings. + +`allow_copy` allows users to copy text from an SRE workspace to the Guacamole clipboard. + +`allow_paste` allows users to paste text into an SRE workspace from the Guacamole clipboard. + +These options have no impact on the ability to use copy and paste within a workspace. + +The impact of setting each of these options is detailed in the below table. + +| allow_copy | allow_paste | Copy/paste within workspace | Copy/paste between workspaces | Copy to local machine | Paste from local machine | +|--------|--------|--------|--|---|--| +| true | true | yes | yes | yes | yes | +| true | false | yes | no | yes | no | +| false | true | yes | no | no | yes| +| false | false | yes | no | no | no | + ## Upload the configuration file - Upload the config to Azure. This will validate your file and report any problems. From 2a3e076a1ef1e3e50c70d448e88e063d15ab126b Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 14:02:19 +0000 Subject: [PATCH 023/113] Add some description of copy and pasting to the user guide --- docs/source/roles/researcher/using_the_sre.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/source/roles/researcher/using_the_sre.md b/docs/source/roles/researcher/using_the_sre.md index 51a7b60abf..368bf1d820 100644 --- a/docs/source/roles/researcher/using_the_sre.md +++ b/docs/source/roles/researcher/using_the_sre.md @@ -703,3 +703,13 @@ The data can be read into a dataframe for local analysis. head(df, 3) ::: :::: + +## Copy and paste + +It is always possible to use copy and paste as normal within an SRE workspace. + +However, the ability to copy and paste text to or from an SRE workspace depends on the specific configuration of the SRE. + +The {ref}`system manager ` can configure the SRE workspaces to allow copying text from a workspace, pasting text into a workspace, both, or neither. + +Copy and paste of anything other than text to or from a workspace is always disabled. From 31f200c3d47c0e92eeba63dbc55bc579bc87a526 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 14:10:47 +0000 Subject: [PATCH 024/113] Move copy and paste section in user guide --- docs/source/deployment/deploy_sre.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index c0fdb99540..8326130997 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -192,7 +192,7 @@ These options have no impact on the ability to use copy and paste within a works The impact of setting each of these options is detailed in the below table. | allow_copy | allow_paste | Copy/paste within workspace | Copy/paste between workspaces | Copy to local machine | Paste from local machine | -|--------|--------|--------|--|---|--| +|-----|-----|-----|-----|-----|-----| | true | true | yes | yes | yes | yes | | true | false | yes | no | yes | no | | false | true | yes | no | no | yes| From de075dcbef5960f9272caa27fda1c42e0d4d3535 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 14:19:59 +0000 Subject: [PATCH 025/113] Move and clarify copy and paste user information --- docs/source/roles/researcher/using_the_sre.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/source/roles/researcher/using_the_sre.md b/docs/source/roles/researcher/using_the_sre.md index 368bf1d820..50b78f9126 100644 --- a/docs/source/roles/researcher/using_the_sre.md +++ b/docs/source/roles/researcher/using_the_sre.md @@ -48,6 +48,16 @@ You can make the process as easy as possible by providing as much information as For instance, describing in detail what a dataset contains and how it will be use will help speed up decision making. ::: +## {{scissors}} Copy and paste + +It is always possible to use copy and paste as normal within an SRE workspace. + +However, the ability to copy and paste text to or from an SRE workspace depends on the specific configuration of the SRE. + +The {ref}`system manager ` can configure the SRE workspaces to allow copying text from a workspace, pasting text into a workspace, both, or neither. + +Copy and paste of anything other than text to or from a workspace is always disabled. + ## {{books}} Maintaining an archive of the project SREs are designed to be ephemeral and only deployed for as long as necessary. @@ -703,13 +713,3 @@ The data can be read into a dataframe for local analysis. head(df, 3) ::: :::: - -## Copy and paste - -It is always possible to use copy and paste as normal within an SRE workspace. - -However, the ability to copy and paste text to or from an SRE workspace depends on the specific configuration of the SRE. - -The {ref}`system manager ` can configure the SRE workspaces to allow copying text from a workspace, pasting text into a workspace, both, or neither. - -Copy and paste of anything other than text to or from a workspace is always disabled. From a055dd56ea70c0a1abbdaa95f618f687b2307ef0 Mon Sep 17 00:00:00 2001 From: Matt Craddock Date: Mon, 28 Oct 2024 14:48:16 +0000 Subject: [PATCH 026/113] Update docs/source/deployment/deploy_sre.md Co-authored-by: Jim Madge --- docs/source/deployment/deploy_sre.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index 8326130997..836eca111e 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -67,8 +67,8 @@ sre: - # You can also use the tag 'Internet' instead of a list software_packages: # Which Python/R packages to allow users to install: [any/pre-approved/none] storage_quota_gb: - home: # Total size in GiB across all home directories . - shared: #Total size in GiB for the shared directories . + home: # Total size in GiB across all home directories + shared: #Total size in GiB for the shared directories timezone: # Timezone in pytz format (eg. Europe/London) workspace_skus: # List of Azure VM SKUs that will be used for data analysis. ::: From 9d478010c3cf4bd2f2fbc1d48e8b2a7eeb77718e Mon Sep 17 00:00:00 2001 From: Matt Craddock Date: Mon, 28 Oct 2024 14:50:15 +0000 Subject: [PATCH 027/113] Update docs/source/deployment/deploy_sre.md Co-authored-by: Jim Madge --- docs/source/deployment/deploy_sre.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index 836eca111e..9913e71c4c 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -176,7 +176,7 @@ As some general recommendations, ::: -### Copy and paste +#### Copy and paste The [Guacamole clipboard](https://guacamole.apache.org/doc/gug/using-guacamole.html#using-the-clipboard) provides an interface between the local clipboard and the clipboard on the remote workspaces. Only text is allowed to be passed through the Guacamole clipboard. From 81ead78d5f76ad6c42b5c7c3c289eceab951fb26 Mon Sep 17 00:00:00 2001 From: Matt Craddock Date: Mon, 28 Oct 2024 14:50:44 +0000 Subject: [PATCH 028/113] Update docs/source/deployment/deploy_sre.md Co-authored-by: Jim Madge --- docs/source/deployment/deploy_sre.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index 9913e71c4c..a6da5aad6f 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -181,12 +181,9 @@ As some general recommendations, The [Guacamole clipboard](https://guacamole.apache.org/doc/gug/using-guacamole.html#using-the-clipboard) provides an interface between the local clipboard and the clipboard on the remote workspaces. Only text is allowed to be passed through the Guacamole clipboard. -Copy and pasting of text to or from SRE workspaces via the Guacamole clipboard can be configured with the `allow_copy` and `allow_paste` settings. - +The ability to copy and paste text to or from SRE workspaces via the Guacamole clipboard can be controlled with the DSH configuration parameters `allow_copy` and `allow_paste`. `allow_copy` allows users to copy text from an SRE workspace to the Guacamole clipboard. - `allow_paste` allows users to paste text into an SRE workspace from the Guacamole clipboard. - These options have no impact on the ability to use copy and paste within a workspace. The impact of setting each of these options is detailed in the below table. From 2a61c58338b482b5efbf3a024348ca63f3e15079 Mon Sep 17 00:00:00 2001 From: Matt Craddock Date: Mon, 28 Oct 2024 14:51:21 +0000 Subject: [PATCH 029/113] Update docs/source/roles/researcher/using_the_sre.md Co-authored-by: Jim Madge --- docs/source/roles/researcher/using_the_sre.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/source/roles/researcher/using_the_sre.md b/docs/source/roles/researcher/using_the_sre.md index 50b78f9126..1620a8f717 100644 --- a/docs/source/roles/researcher/using_the_sre.md +++ b/docs/source/roles/researcher/using_the_sre.md @@ -51,12 +51,9 @@ For instance, describing in detail what a dataset contains and how it will be us ## {{scissors}} Copy and paste It is always possible to use copy and paste as normal within an SRE workspace. - However, the ability to copy and paste text to or from an SRE workspace depends on the specific configuration of the SRE. - The {ref}`system manager ` can configure the SRE workspaces to allow copying text from a workspace, pasting text into a workspace, both, or neither. - -Copy and paste of anything other than text to or from a workspace is always disabled. +Copy and paste of anything other than text to or from a workspace is not possible. ## {{books}} Maintaining an archive of the project From b281eb99cd0d0ccdce020ea6fa51611d87a3cafc Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 15:07:26 +0000 Subject: [PATCH 030/113] Use table directive --- docs/source/deployment/deploy_sre.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index a6da5aad6f..20a41c52e8 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -188,12 +188,14 @@ These options have no impact on the ability to use copy and paste within a works The impact of setting each of these options is detailed in the below table. +:::{table} **Configuration of copy and paste** | allow_copy | allow_paste | Copy/paste within workspace | Copy/paste between workspaces | Copy to local machine | Paste from local machine | |-----|-----|-----|-----|-----|-----| | true | true | yes | yes | yes | yes | | true | false | yes | no | yes | no | | false | true | yes | no | no | yes| | false | false | yes | no | no | no | +::: ## Upload the configuration file From 754faf28690906726b1879fb0a7a7af782297be9 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 15:34:51 +0000 Subject: [PATCH 031/113] Add confirmation step to SHM teardown --- data_safe_haven/commands/shm.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/data_safe_haven/commands/shm.py b/data_safe_haven/commands/shm.py index 522609b8ff..9fe35692f1 100644 --- a/data_safe_haven/commands/shm.py +++ b/data_safe_haven/commands/shm.py @@ -142,6 +142,12 @@ def teardown() -> None: try: config = SHMConfig.from_remote(context) shm_infra = ImperativeSHM(context, config) + console.print( + "Tearing down the Safe Haven Management environment will permanently delete all associated resources, including remotely stored configurations." + ) + if not console.confirm("Do you wish to continue?", default_to_yes=True): + logger.info("SHM teardown cancelled by user.") + raise typer.Exit(0) shm_infra.teardown() except DataSafeHavenError as exc: logger.critical("Could not teardown Safe Haven Management environment.") From f2c1fa160b86e9fae20582275cc373f4f70895fb Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 15:35:40 +0000 Subject: [PATCH 032/113] Ask for confirmation before doing anything --- data_safe_haven/commands/shm.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/commands/shm.py b/data_safe_haven/commands/shm.py index 9fe35692f1..7a56c5c8a5 100644 --- a/data_safe_haven/commands/shm.py +++ b/data_safe_haven/commands/shm.py @@ -140,14 +140,14 @@ def teardown() -> None: # Teardown Data Safe Haven SHM infrastructure. try: - config = SHMConfig.from_remote(context) - shm_infra = ImperativeSHM(context, config) console.print( "Tearing down the Safe Haven Management environment will permanently delete all associated resources, including remotely stored configurations." ) if not console.confirm("Do you wish to continue?", default_to_yes=True): logger.info("SHM teardown cancelled by user.") raise typer.Exit(0) + config = SHMConfig.from_remote(context) + shm_infra = ImperativeSHM(context, config) shm_infra.teardown() except DataSafeHavenError as exc: logger.critical("Could not teardown Safe Haven Management environment.") From c1e488f589498be06db3a8c7495b2c679eacd725 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 16:10:40 +0000 Subject: [PATCH 033/113] Add confirmation step to SRE teardown --- data_safe_haven/commands/sre.py | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/data_safe_haven/commands/sre.py b/data_safe_haven/commands/sre.py index f03f0cc53e..c4519d573e 100644 --- a/data_safe_haven/commands/sre.py +++ b/data_safe_haven/commands/sre.py @@ -4,6 +4,7 @@ import typer +from data_safe_haven import console from data_safe_haven.config import ContextManager, DSHPulumiConfig, SHMConfig, SREConfig from data_safe_haven.exceptions import DataSafeHavenConfigError, DataSafeHavenError from data_safe_haven.external import AzureSdk, GraphApi @@ -183,6 +184,7 @@ def teardown( """Tear down a deployed a Secure Research Environment.""" logger = get_logger() try: + # Load context and SHM config context = ContextManager.from_file().assert_context() shm_config = SHMConfig.from_remote(context) @@ -197,6 +199,15 @@ def teardown( pulumi_config = DSHPulumiConfig.from_remote(context) sre_config = SREConfig.from_remote_by_name(context, name) + console.print( + "Tearing down the Secure Research Environment will permanently delete all associated resources, " + "including all data stored in the environment.\n" + "Ensure that any desired outputs have been extracted before continuing." + ) + if not console.confirm("Do you wish to continue tearing down the SRE?", default_to_yes=True): + logger.info("SRE teardown cancelled by user.") + raise typer.Exit(0) + # Check whether current IP address is authorised to take administrator actions if not ip_address_in_list(sre_config.sre.admin_ip_addresses): logger.warning( From 567273687657668eb9fd315d561b87344c9fd421 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 16:11:09 +0000 Subject: [PATCH 034/113] Move SHM teardown confirmation after config retrieval --- data_safe_haven/commands/shm.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/data_safe_haven/commands/shm.py b/data_safe_haven/commands/shm.py index 7a56c5c8a5..b202711989 100644 --- a/data_safe_haven/commands/shm.py +++ b/data_safe_haven/commands/shm.py @@ -140,14 +140,14 @@ def teardown() -> None: # Teardown Data Safe Haven SHM infrastructure. try: + config = SHMConfig.from_remote(context) + shm_infra = ImperativeSHM(context, config) console.print( "Tearing down the Safe Haven Management environment will permanently delete all associated resources, including remotely stored configurations." ) - if not console.confirm("Do you wish to continue?", default_to_yes=True): + if not console.confirm("Do you wish to continue tearing down the SHM?", default_to_yes=True): logger.info("SHM teardown cancelled by user.") raise typer.Exit(0) - config = SHMConfig.from_remote(context) - shm_infra = ImperativeSHM(context, config) shm_infra.teardown() except DataSafeHavenError as exc: logger.critical("Could not teardown Safe Haven Management environment.") From 59ca7b09b46cad235dd72d2a6e017735e572ad7c Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 16:12:09 +0000 Subject: [PATCH 035/113] Fix linting --- data_safe_haven/commands/shm.py | 4 +++- data_safe_haven/commands/sre.py | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/commands/shm.py b/data_safe_haven/commands/shm.py index b202711989..ff73884db9 100644 --- a/data_safe_haven/commands/shm.py +++ b/data_safe_haven/commands/shm.py @@ -145,7 +145,9 @@ def teardown() -> None: console.print( "Tearing down the Safe Haven Management environment will permanently delete all associated resources, including remotely stored configurations." ) - if not console.confirm("Do you wish to continue tearing down the SHM?", default_to_yes=True): + if not console.confirm( + "Do you wish to continue tearing down the SHM?", default_to_yes=True + ): logger.info("SHM teardown cancelled by user.") raise typer.Exit(0) shm_infra.teardown() diff --git a/data_safe_haven/commands/sre.py b/data_safe_haven/commands/sre.py index c4519d573e..453794d525 100644 --- a/data_safe_haven/commands/sre.py +++ b/data_safe_haven/commands/sre.py @@ -204,7 +204,9 @@ def teardown( "including all data stored in the environment.\n" "Ensure that any desired outputs have been extracted before continuing." ) - if not console.confirm("Do you wish to continue tearing down the SRE?", default_to_yes=True): + if not console.confirm( + "Do you wish to continue tearing down the SRE?", default_to_yes=True + ): logger.info("SRE teardown cancelled by user.") raise typer.Exit(0) From 20a8722345928cf70d73caad3ce33b4a7ec21ca0 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 28 Oct 2024 16:17:52 +0000 Subject: [PATCH 036/113] remove superfluous pulumi reference --- data_safe_haven/infrastructure/programs/imperative_shm.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data_safe_haven/infrastructure/programs/imperative_shm.py b/data_safe_haven/infrastructure/programs/imperative_shm.py index 7c8eab3f98..5cbb679b73 100644 --- a/data_safe_haven/infrastructure/programs/imperative_shm.py +++ b/data_safe_haven/infrastructure/programs/imperative_shm.py @@ -175,7 +175,7 @@ def teardown(self) -> None: pulumi_config = DSHPulumiConfig.from_remote(self.context) deployed = pulumi_config.project_names if deployed: - logger.info(f"Found deployed Pulumi SREs: {deployed}.") + logger.info(f"Found deployed SREs: {deployed}.") msg = "Deployed SREs must be torn down before the SHM can be torn down." raise DataSafeHavenAzureError(msg) try: From 41e3ac40e58d8fddcba3928b09dbf39fe3da7da4 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Fri, 18 Oct 2024 22:24:40 +0100 Subject: [PATCH 037/113] :wrench: Add GUID for Directory.ReadWrite.All permission --- data_safe_haven/external/api/graph_api.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/data_safe_haven/external/api/graph_api.py b/data_safe_haven/external/api/graph_api.py index 66cf139a65..ea78f928c3 100644 --- a/data_safe_haven/external/api/graph_api.py +++ b/data_safe_haven/external/api/graph_api.py @@ -34,6 +34,7 @@ class GraphApi: "Application.ReadWrite.All": "1bfefb4e-e0b5-418b-a88f-73c46d2cc8e9", "AppRoleAssignment.ReadWrite.All": "06b708a9-e830-4db3-a914-8e69da51d44f", "Directory.Read.All": "7ab1d382-f21e-4acd-a863-ba3e13f7da61", + "Directory.ReadWrite.All": "19dbc75e-c2e2-444c-a770-ec69d8559fc7", "Domain.Read.All": "dbb9058a-0e50-45d7-ae91-66909b5d4664", "Group.Read.All": "5b567255-7703-4780-807c-7be8301ae99b", "Group.ReadWrite.All": "62a82d76-70ea-41e2-9197-370581804d09", @@ -589,9 +590,9 @@ def grant_application_role_permissions( f"Assigning application role '[green]{application_role_name}[/]' to '{application_name}'...", ) request_json = { + "appRoleId": app_role_id, "principalId": application_sp["id"], "resourceId": microsoft_graph_sp["id"], - "appRoleId": app_role_id, } self.http_post( f"{self.base_endpoint}/servicePrincipals/{microsoft_graph_sp['id']}/appRoleAssignments", From dc14741d8fbc4c551928c2d7008587f7275a4471 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Fri, 18 Oct 2024 22:25:44 +0100 Subject: [PATCH 038/113] :sparkles: Add pulumi-azuread EntraApplication component --- .../components/composite/entra_application.py | 97 +++++++++++++++++++ .../programs/declarative_sre.py | 2 + .../infrastructure/programs/imperative_shm.py | 7 +- .../infrastructure/programs/sre/entra.py | 61 +++++++++++- 4 files changed, 161 insertions(+), 6 deletions(-) create mode 100644 data_safe_haven/infrastructure/components/composite/entra_application.py diff --git a/data_safe_haven/infrastructure/components/composite/entra_application.py b/data_safe_haven/infrastructure/components/composite/entra_application.py new file mode 100644 index 0000000000..0121828d94 --- /dev/null +++ b/data_safe_haven/infrastructure/components/composite/entra_application.py @@ -0,0 +1,97 @@ +"""Pulumi component for an Entra Application resource""" + +from collections.abc import Mapping + +import pulumi_azuread as entra +from pulumi import ComponentResource, Input, Output, ResourceOptions + +from data_safe_haven.functions import replace_separators + + +class EntraApplicationProps: + """Properties for EntraApplicationComponent""" + + def __init__( + self, + application_name: Input[str], + application_permissions: list[tuple[str, str]], + msgraph_service_principal: Input[entra.ServicePrincipal], + ) -> None: + self.application_name = application_name + self.application_permissions = application_permissions + self.msgraph_client_id = msgraph_service_principal.client_id + self.msgraph_object_id = msgraph_service_principal.object_id + + # Construct a mapping of all the available application permissions + self.msgraph_permissions: Output[dict[str, Mapping[str, str]]] = Output.all( + application=msgraph_service_principal.app_role_ids, + delegated=msgraph_service_principal.oauth2_permission_scope_ids, + ).apply( + lambda kwargs: { + # 'Role' permissions belong to the application + "Role": kwargs["application"], + # 'Scope' permissions are delegated to users + "Scope": kwargs["delegated"], + } + ) + + +class EntraApplicationComponent(ComponentResource): + """Deploy an Entra application with Pulumi""" + + def __init__( + self, + name: str, + props: EntraApplicationProps, + opts: ResourceOptions | None = None, + ) -> None: + super().__init__("dsh:common:EntraApplicationComponent", name, {}, opts) + + # Create the application + self.application = entra.Application( + f"{self._name}_application", + display_name=props.application_name, + sign_in_audience="AzureADMyOrg", + public_client={"redirectUris": ["urn:ietf:wg:oauth:2.0:oob"]}, + required_resource_accesses=[ + entra.ApplicationRequiredResourceAccessArgs( + resource_accesses=[ + entra.ApplicationRequiredResourceAccessResourceAccessArgs( + id=props.msgraph_permissions[scope][permission], + type=scope, + ) + for scope, permission in props.application_permissions + ], + resource_app_id=props.msgraph_client_id, + ) + ], + ) + + # Get the service principal for this application + self.application_service_principal = entra.ServicePrincipal( + f"{self._name}_application_service_principal", + client_id=self.application.client_id, + ) + + # Grant admin approval for requested application permissions + for scope, permission in props.application_permissions: + if scope == "application": + entra.AppRoleAssignment( + replace_separators( + f"{self._name}_application_grant_{scope}_{permission}".lower(), + "_", + ), + app_role_id=props.msgraph_permissions[scope][permission], + principal_object_id=self.application_service_principal.object_id, + resource_object_id=props.msgraph_object_id, + ) + if scope == "delegated": + entra.ServicePrincipalDelegatedPermissionGrant( + replace_separators( + f"{self._name}_application_delegated_grant_{scope}_{permission}".lower(), + "_", + ), + claim_values=[permission], + resource_service_principal_object_id=props.msgraph_object_id, + service_principal_object_id=self.application_service_principal.object_id, + ) diff --git a/data_safe_haven/infrastructure/programs/declarative_sre.py b/data_safe_haven/infrastructure/programs/declarative_sre.py index ce678dbb4a..8be5086976 100644 --- a/data_safe_haven/infrastructure/programs/declarative_sre.py +++ b/data_safe_haven/infrastructure/programs/declarative_sre.py @@ -117,6 +117,8 @@ def __call__(self) -> None: "sre_entra", SREEntraProps( group_names=ldap_group_names, + shm_name=self.context.name, + sre_name=self.config.name, ), ) diff --git a/data_safe_haven/infrastructure/programs/imperative_shm.py b/data_safe_haven/infrastructure/programs/imperative_shm.py index 9b748bbdd1..1095f26db2 100644 --- a/data_safe_haven/infrastructure/programs/imperative_shm.py +++ b/data_safe_haven/infrastructure/programs/imperative_shm.py @@ -147,7 +147,12 @@ def deploy(self) -> None: try: graph_api.create_application( self.context.entra_application_name, - application_scopes=["Group.ReadWrite.All"], + application_scopes=[ + "Application.ReadWrite.All", + "AppRoleAssignment.ReadWrite.All", + "Directory.ReadWrite.All", + "Group.ReadWrite.All", + ], delegated_scopes=[], request_json={ "displayName": self.context.entra_application_name, diff --git a/data_safe_haven/infrastructure/programs/sre/entra.py b/data_safe_haven/infrastructure/programs/sre/entra.py index 1f44995f9f..d6ea97e068 100644 --- a/data_safe_haven/infrastructure/programs/sre/entra.py +++ b/data_safe_haven/infrastructure/programs/sre/entra.py @@ -2,10 +2,14 @@ from collections.abc import Mapping -from pulumi import ComponentResource, ResourceOptions -from pulumi_azuread import Group +import pulumi_azuread as entra +from pulumi import ComponentResource, Input, Output, ResourceOptions from data_safe_haven.functions import replace_separators +from data_safe_haven.infrastructure.components.composite.entra_application import ( + EntraApplicationComponent, + EntraApplicationProps, +) class SREEntraProps: @@ -14,8 +18,12 @@ class SREEntraProps: def __init__( self, group_names: Mapping[str, str], + shm_name: Input[str], + sre_name: Input[str], ) -> None: self.group_names = group_names + self.shm_name = shm_name + self.sre_name = sre_name class SREEntraComponent(ComponentResource): @@ -28,13 +36,56 @@ def __init__( opts: ResourceOptions | None = None, ) -> None: super().__init__("dsh:sre:EntraComponent", name, {}, opts) + child_opts = ResourceOptions.merge(opts, ResourceOptions(parent=self)) - for group_id, group_description in props.group_names.items(): - Group( - replace_separators(f"{self._name}_group_{group_id}", "_"), + # Create Entra groups + for group_name, group_description in props.group_names.items(): + entra.Group( + replace_separators(f"{self._name}_group_{group_name}", "_"), description=group_description, display_name=group_description, mail_enabled=False, prevent_duplicate_names=True, security_enabled=True, ) + + # Get the Microsoft Graph service principal + well_known = entra.get_application_published_app_ids_output() + msgraph_service_principal = entra.ServicePrincipal( + f"{self._name}_microsoft_graph_service_principal", + client_id=well_known.apply( + lambda app_ids: app_ids.result["MicrosoftGraph"] + ), + use_existing=True, + ) + + # Identity application + # - needs read-only permissions for users/groups + # - needs delegated permission to read users (for validating log-in attempts) + # - needs an application secret for authentication + self.identity_application = EntraApplicationComponent( + f"{self._name}_identity", + EntraApplicationProps( + application_name=Output.concat( + "Data Safe Haven (", + props.shm_name, + " - ", + props.sre_name, + ") Identity Service Principal", + ), + application_permissions=[ + ("Role", "User.Read.All"), + ("Role", "GroupMember.Read.All"), + ("Scope", "User.Read.All"), + ], + msgraph_service_principal=msgraph_service_principal, + ), + opts=child_opts, + ) + + # Add an application password + self.identity_application_secret = entra.ApplicationPassword( + f"{self._name}_identity_application_secret", + application_id=self.identity_application.application.id, + display_name="Apricot Authentication Secret", + ) From f533b0e913b703a5b57aa3ef968ad0f4b1b3add5 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 22 Oct 2024 10:13:54 +0100 Subject: [PATCH 039/113] :sparkles: Add support for web applications as well as desktop applications --- .../components/composite/entra_application.py | 86 ++++++++++++++++--- .../programs/declarative_sre.py | 21 ++--- .../infrastructure/programs/sre/entra.py | 30 ++++++- 3 files changed, 112 insertions(+), 25 deletions(-) diff --git a/data_safe_haven/infrastructure/components/composite/entra_application.py b/data_safe_haven/infrastructure/components/composite/entra_application.py index 0121828d94..1e234bbbce 100644 --- a/data_safe_haven/infrastructure/components/composite/entra_application.py +++ b/data_safe_haven/infrastructure/components/composite/entra_application.py @@ -1,6 +1,7 @@ """Pulumi component for an Entra Application resource""" from collections.abc import Mapping +from typing import Any import pulumi_azuread as entra from pulumi import ComponentResource, Input, Output, ResourceOptions @@ -16,11 +17,13 @@ def __init__( application_name: Input[str], application_permissions: list[tuple[str, str]], msgraph_service_principal: Input[entra.ServicePrincipal], + application_kwargs: Mapping[str, Any], ) -> None: self.application_name = application_name self.application_permissions = application_permissions self.msgraph_client_id = msgraph_service_principal.client_id self.msgraph_object_id = msgraph_service_principal.object_id + self.application_kwargs = application_kwargs # Construct a mapping of all the available application permissions self.msgraph_permissions: Output[dict[str, Mapping[str, str]]] = Output.all( @@ -36,6 +39,58 @@ def __init__( ) +class EntraDesktopApplicationProps(EntraApplicationProps): + """ + Properties for a desktop EntraApplicationComponent. + See https://learn.microsoft.com/en-us/entra/identity-platform/msal-client-applications) + """ + + def __init__( + self, + application_name: Input[str], + application_permissions: list[tuple[str, str]], + msgraph_service_principal: Input[entra.ServicePrincipal], + ): + super().__init__( + application_name=application_name, + application_kwargs={ + "public_client": entra.ApplicationPublicClientArgs( + redirect_uris=["urn:ietf:wg:oauth:2.0:oob"] + ) + }, + application_permissions=application_permissions, + msgraph_service_principal=msgraph_service_principal, + ) + + +class EntraWebApplicationProps(EntraApplicationProps): + """ + Properties for a web EntraApplicationComponent. + See https://learn.microsoft.com/en-us/entra/identity-platform/msal-client-applications) + """ + + def __init__( + self, + application_name: Input[str], + application_permissions: list[tuple[str, str]], + msgraph_service_principal: Input[entra.ServicePrincipal], + redirect_url: Input[str], + ): + super().__init__( + application_name=application_name, + application_kwargs={ + "web": entra.ApplicationWebArgs( + redirect_uris=[redirect_url], + implicit_grant=entra.ApplicationWebImplicitGrantArgs( + id_token_issuance_enabled=True, + ), + ) + }, + application_permissions=application_permissions, + msgraph_service_principal=msgraph_service_principal, + ) + + class EntraApplicationComponent(ComponentResource): """Deploy an Entra application with Pulumi""" @@ -51,20 +106,25 @@ def __init__( self.application = entra.Application( f"{self._name}_application", display_name=props.application_name, + prevent_duplicate_names=True, + required_resource_accesses=( + [ + entra.ApplicationRequiredResourceAccessArgs( + resource_accesses=[ + entra.ApplicationRequiredResourceAccessResourceAccessArgs( + id=props.msgraph_permissions[scope][permission], + type=scope, + ) + for scope, permission in props.application_permissions + ], + resource_app_id=props.msgraph_client_id, + ) + ] + if props.application_permissions + else [] + ), sign_in_audience="AzureADMyOrg", - public_client={"redirectUris": ["urn:ietf:wg:oauth:2.0:oob"]}, - required_resource_accesses=[ - entra.ApplicationRequiredResourceAccessArgs( - resource_accesses=[ - entra.ApplicationRequiredResourceAccessResourceAccessArgs( - id=props.msgraph_permissions[scope][permission], - type=scope, - ) - for scope, permission in props.application_permissions - ], - resource_app_id=props.msgraph_client_id, - ) - ], + **props.application_kwargs, ) # Get the service principal for this application diff --git a/data_safe_haven/infrastructure/programs/declarative_sre.py b/data_safe_haven/infrastructure/programs/declarative_sre.py index 8be5086976..9a3885d068 100644 --- a/data_safe_haven/infrastructure/programs/declarative_sre.py +++ b/data_safe_haven/infrastructure/programs/declarative_sre.py @@ -112,16 +112,6 @@ def __call__(self) -> None: ] ) - # Deploy Entra resources - SREEntraComponent( - "sre_entra", - SREEntraProps( - group_names=ldap_group_names, - shm_name=self.context.name, - sre_name=self.config.name, - ), - ) - # Deploy resource group resource_group = resources.ResourceGroup( "sre_resource_group", @@ -164,6 +154,17 @@ def __call__(self) -> None: tags=self.tags, ) + # Deploy Entra resources + SREEntraComponent( + "sre_entra", + SREEntraProps( + group_names=ldap_group_names, + shm_name=self.context.name, + sre_name=self.config.name, + remote_desktop_fqdn=networking.sre_fqdn, + ), + ) + # Deploy SRE firewall SREFirewallComponent( "sre_firewall", diff --git a/data_safe_haven/infrastructure/programs/sre/entra.py b/data_safe_haven/infrastructure/programs/sre/entra.py index d6ea97e068..7ce1a003ad 100644 --- a/data_safe_haven/infrastructure/programs/sre/entra.py +++ b/data_safe_haven/infrastructure/programs/sre/entra.py @@ -8,7 +8,8 @@ from data_safe_haven.functions import replace_separators from data_safe_haven.infrastructure.components.composite.entra_application import ( EntraApplicationComponent, - EntraApplicationProps, + EntraDesktopApplicationProps, + EntraWebApplicationProps, ) @@ -18,10 +19,14 @@ class SREEntraProps: def __init__( self, group_names: Mapping[str, str], + remote_desktop_fqdn: Input[str], shm_name: Input[str], sre_name: Input[str], ) -> None: self.group_names = group_names + self.remote_desktop_url = Output.from_input(remote_desktop_fqdn).apply( + lambda fqdn: f"https://{fqdn.strip('/')}/" + ) self.shm_name = shm_name self.sre_name = sre_name @@ -65,7 +70,7 @@ def __init__( # - needs an application secret for authentication self.identity_application = EntraApplicationComponent( f"{self._name}_identity", - EntraApplicationProps( + EntraDesktopApplicationProps( application_name=Output.concat( "Data Safe Haven (", props.shm_name, @@ -89,3 +94,24 @@ def __init__( application_id=self.identity_application.application.id, display_name="Apricot Authentication Secret", ) + + # Remote desktop application + # - only used as part of the OAuth 2.0 authorization flow + # - does not need any application permissions + # - does not need an application secret + self.remote_desktop_application = EntraApplicationComponent( + f"{self._name}_remote_desktop", + EntraWebApplicationProps( + application_name=Output.concat( + "Data Safe Haven (", + props.shm_name, + " - ", + props.sre_name, + ") Remote Desktop Service Principal", + ), + application_permissions=[], + msgraph_service_principal=msgraph_service_principal, + redirect_url=props.remote_desktop_url, + ), + opts=child_opts, + ) From a6ddc981f73931177635709d98fd51f5e8fe902e Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 22 Oct 2024 10:52:58 +0100 Subject: [PATCH 040/113] :recycle: Use composite EntraApplication component for identity server --- .../programs/declarative_sre.py | 6 ++-- .../infrastructure/programs/sre/entra.py | 7 +++++ .../infrastructure/programs/sre/identity.py | 28 ++++--------------- 3 files changed, 16 insertions(+), 25 deletions(-) diff --git a/data_safe_haven/infrastructure/programs/declarative_sre.py b/data_safe_haven/infrastructure/programs/declarative_sre.py index 9a3885d068..38598cf740 100644 --- a/data_safe_haven/infrastructure/programs/declarative_sre.py +++ b/data_safe_haven/infrastructure/programs/declarative_sre.py @@ -155,7 +155,7 @@ def __call__(self) -> None: ) # Deploy Entra resources - SREEntraComponent( + entra = SREEntraComponent( "sre_entra", SREEntraProps( group_names=ldap_group_names, @@ -251,8 +251,8 @@ def __call__(self) -> None: SREIdentityProps( dns_server_ip=dns.ip_address, dockerhub_credentials=dockerhub_credentials, - entra_application_name=f"sre-{self.config.name}-apricot", - entra_auth_token=self.graph_api_token, + entra_application_id=entra.identity_application_id, + entra_application_secret=entra.identity_application_secret, entra_tenant_id=shm_entra_tenant_id, location=self.config.azure.location, resource_group_name=resource_group.name, diff --git a/data_safe_haven/infrastructure/programs/sre/entra.py b/data_safe_haven/infrastructure/programs/sre/entra.py index 7ce1a003ad..9bf14d1ce8 100644 --- a/data_safe_haven/infrastructure/programs/sre/entra.py +++ b/data_safe_haven/infrastructure/programs/sre/entra.py @@ -115,3 +115,10 @@ def __init__( ), opts=child_opts, ) + + # Register outputs + self.identity_application_id = self.identity_application.application.id + self.identity_application_secret = self.identity_application_secret.value + self.remote_desktop_application_id = ( + self.remote_desktop_application.application.id + ) diff --git a/data_safe_haven/infrastructure/programs/sre/identity.py b/data_safe_haven/infrastructure/programs/sre/identity.py index 0196fe7e39..7839853384 100644 --- a/data_safe_haven/infrastructure/programs/sre/identity.py +++ b/data_safe_haven/infrastructure/programs/sre/identity.py @@ -11,8 +11,6 @@ get_ip_address_from_container_group, ) from data_safe_haven.infrastructure.components import ( - EntraApplication, - EntraApplicationProps, LocalDnsRecordComponent, LocalDnsRecordProps, ) @@ -25,8 +23,8 @@ def __init__( self, dns_server_ip: Input[str], dockerhub_credentials: DockerHubCredentials, - entra_application_name: Input[str], - entra_auth_token: str, + entra_application_id: Input[str], + entra_application_secret: Input[str], entra_tenant_id: Input[str], location: Input[str], resource_group_name: Input[str], @@ -38,8 +36,8 @@ def __init__( ) -> None: self.dns_server_ip = dns_server_ip self.dockerhub_credentials = dockerhub_credentials - self.entra_application_name = entra_application_name - self.entra_auth_token = entra_auth_token + self.entra_application_id = entra_application_id + self.entra_application_secret = entra_application_secret self.entra_tenant_id = entra_tenant_id self.location = location self.resource_group_name = resource_group_name @@ -82,20 +80,6 @@ def __init__( opts=child_opts, ) - # Define Entra ID application - entra_application = EntraApplication( - f"{self._name}_entra_application", - EntraApplicationProps( - application_name=props.entra_application_name, - application_role_assignments=["User.Read.All", "GroupMember.Read.All"], - application_secret_name="Apricot Authentication Secret", - delegated_role_assignments=["User.Read.All"], - public_client_redirect_uri="urn:ietf:wg:oauth:2.0:oob", - ), - auth_token=props.entra_auth_token, - opts=child_opts, - ) - # Define the LDAP server container group with Apricot container_group = containerinstance.ContainerGroup( f"{self._name}_container_group", @@ -111,11 +95,11 @@ def __init__( ), containerinstance.EnvironmentVariableArgs( name="CLIENT_ID", - value=entra_application.application_id, + value=props.entra_application_id, ), containerinstance.EnvironmentVariableArgs( name="CLIENT_SECRET", - secure_value=entra_application.application_secret, + secure_value=props.entra_application_secret, ), containerinstance.EnvironmentVariableArgs( name="DEBUG", From ba1f5b3b7395d62f90033c9cfba31749c5b0e8ff Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 22 Oct 2024 10:56:43 +0100 Subject: [PATCH 041/113] :truck: Consolidate remote desktop URL construction in Entra component --- .../infrastructure/programs/declarative_sre.py | 4 ++-- data_safe_haven/infrastructure/programs/sre/entra.py | 11 ++++++----- .../infrastructure/programs/sre/remote_desktop.py | 4 ++-- 3 files changed, 10 insertions(+), 9 deletions(-) diff --git a/data_safe_haven/infrastructure/programs/declarative_sre.py b/data_safe_haven/infrastructure/programs/declarative_sre.py index 38598cf740..04017e8b22 100644 --- a/data_safe_haven/infrastructure/programs/declarative_sre.py +++ b/data_safe_haven/infrastructure/programs/declarative_sre.py @@ -160,8 +160,8 @@ def __call__(self) -> None: SREEntraProps( group_names=ldap_group_names, shm_name=self.context.name, + sre_fqdn=networking.sre_fqdn, sre_name=self.config.name, - remote_desktop_fqdn=networking.sre_fqdn, ), ) @@ -291,7 +291,7 @@ def __call__(self) -> None: database_password=data.password_user_database_admin, dns_server_ip=dns.ip_address, dockerhub_credentials=dockerhub_credentials, - entra_application_fqdn=networking.sre_fqdn, + entra_application_url=entra.remote_desktop_url, entra_application_name=f"sre-{self.config.name}-guacamole", entra_auth_token=self.graph_api_token, entra_tenant_id=shm_entra_tenant_id, diff --git a/data_safe_haven/infrastructure/programs/sre/entra.py b/data_safe_haven/infrastructure/programs/sre/entra.py index 9bf14d1ce8..b2d9e03821 100644 --- a/data_safe_haven/infrastructure/programs/sre/entra.py +++ b/data_safe_haven/infrastructure/programs/sre/entra.py @@ -19,15 +19,13 @@ class SREEntraProps: def __init__( self, group_names: Mapping[str, str], - remote_desktop_fqdn: Input[str], + sre_fqdn: Input[str], shm_name: Input[str], sre_name: Input[str], ) -> None: self.group_names = group_names - self.remote_desktop_url = Output.from_input(remote_desktop_fqdn).apply( - lambda fqdn: f"https://{fqdn.strip('/')}/" - ) self.shm_name = shm_name + self.sre_fqdn = sre_fqdn self.sre_name = sre_name @@ -99,6 +97,9 @@ def __init__( # - only used as part of the OAuth 2.0 authorization flow # - does not need any application permissions # - does not need an application secret + self.remote_desktop_url = Output.from_input(props.sre_fqdn).apply( + lambda fqdn: f"https://{str(fqdn).strip('/')}/" + ) self.remote_desktop_application = EntraApplicationComponent( f"{self._name}_remote_desktop", EntraWebApplicationProps( @@ -111,7 +112,7 @@ def __init__( ), application_permissions=[], msgraph_service_principal=msgraph_service_principal, - redirect_url=props.remote_desktop_url, + redirect_url=self.remote_desktop_url, ), opts=child_opts, ) diff --git a/data_safe_haven/infrastructure/programs/sre/remote_desktop.py b/data_safe_haven/infrastructure/programs/sre/remote_desktop.py index 3be1207c77..9566367255 100644 --- a/data_safe_haven/infrastructure/programs/sre/remote_desktop.py +++ b/data_safe_haven/infrastructure/programs/sre/remote_desktop.py @@ -32,7 +32,7 @@ def __init__( database_password: Input[str], dns_server_ip: Input[str], dockerhub_credentials: DockerHubCredentials, - entra_application_fqdn: Input[str], + entra_application_url: Input[str], entra_application_name: Input[str], entra_auth_token: str, entra_tenant_id: Input[str], @@ -59,7 +59,7 @@ def __init__( self.dns_server_ip = dns_server_ip self.dockerhub_credentials = dockerhub_credentials self.entra_application_name = entra_application_name - self.entra_application_url = Output.concat("https://", entra_application_fqdn) + self.entra_application_url = entra_application_url self.entra_auth_token = entra_auth_token self.entra_tenant_id = entra_tenant_id self.ldap_group_filter = ldap_group_filter From 5df24d7bb65157c7e80343127cde314b5c4e073e Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 22 Oct 2024 11:09:47 +0100 Subject: [PATCH 042/113] :recycle: Use composite EntraApplication component for remote desktop server --- .../programs/declarative_sre.py | 2 +- .../programs/sre/remote_desktop.py | 19 +++---------------- 2 files changed, 4 insertions(+), 17 deletions(-) diff --git a/data_safe_haven/infrastructure/programs/declarative_sre.py b/data_safe_haven/infrastructure/programs/declarative_sre.py index 04017e8b22..1c81e8324e 100644 --- a/data_safe_haven/infrastructure/programs/declarative_sre.py +++ b/data_safe_haven/infrastructure/programs/declarative_sre.py @@ -291,8 +291,8 @@ def __call__(self) -> None: database_password=data.password_user_database_admin, dns_server_ip=dns.ip_address, dockerhub_credentials=dockerhub_credentials, + entra_application_id=entra.remote_desktop_application_id, entra_application_url=entra.remote_desktop_url, - entra_application_name=f"sre-{self.config.name}-guacamole", entra_auth_token=self.graph_api_token, entra_tenant_id=shm_entra_tenant_id, ldap_group_filter=ldap_group_filter, diff --git a/data_safe_haven/infrastructure/programs/sre/remote_desktop.py b/data_safe_haven/infrastructure/programs/sre/remote_desktop.py index 9566367255..c6cf26ffdb 100644 --- a/data_safe_haven/infrastructure/programs/sre/remote_desktop.py +++ b/data_safe_haven/infrastructure/programs/sre/remote_desktop.py @@ -11,8 +11,6 @@ get_id_from_subnet, ) from data_safe_haven.infrastructure.components import ( - EntraApplication, - EntraApplicationProps, FileShareFile, FileShareFileProps, PostgresqlDatabaseComponent, @@ -32,8 +30,8 @@ def __init__( database_password: Input[str], dns_server_ip: Input[str], dockerhub_credentials: DockerHubCredentials, + entra_application_id: Input[str], entra_application_url: Input[str], - entra_application_name: Input[str], entra_auth_token: str, entra_tenant_id: Input[str], ldap_group_filter: Input[str], @@ -58,7 +56,7 @@ def __init__( self.disable_paste = not allow_paste self.dns_server_ip = dns_server_ip self.dockerhub_credentials = dockerhub_credentials - self.entra_application_name = entra_application_name + self.entra_application_id = entra_application_id self.entra_application_url = entra_application_url self.entra_auth_token = entra_auth_token self.entra_tenant_id = entra_tenant_id @@ -119,17 +117,6 @@ def __init__( child_opts = ResourceOptions.merge(opts, ResourceOptions(parent=self)) child_tags = {"component": "remote desktop"} | (tags if tags else {}) - # Define Entra ID application - entra_application = EntraApplication( - f"{self._name}_entra_application", - EntraApplicationProps( - application_name=props.entra_application_name, - web_redirect_url=props.entra_application_url, - ), - auth_token=props.entra_auth_token, - opts=child_opts, - ) - # Define configuration file shares file_share = storage.FileShare( f"{self._name}_file_share", @@ -224,7 +211,7 @@ def __init__( ), containerinstance.EnvironmentVariableArgs( name="OPENID_CLIENT_ID", - value=entra_application.application_id, + value=props.entra_application_id, ), containerinstance.EnvironmentVariableArgs( name="OPENID_ISSUER", From c43708b6f371a20962abc3d1eaac09fe32380886 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 22 Oct 2024 13:20:05 +0100 Subject: [PATCH 043/113] :coffin: Drop dynamic EntraApplication component --- .../infrastructure/components/__init__.py | 4 - .../components/dynamic/__init__.py | 3 - .../components/dynamic/entra_application.py | 191 ------------------ 3 files changed, 198 deletions(-) delete mode 100644 data_safe_haven/infrastructure/components/dynamic/entra_application.py diff --git a/data_safe_haven/infrastructure/components/__init__.py b/data_safe_haven/infrastructure/components/__init__.py index 7531491bf8..5ff8b92baf 100644 --- a/data_safe_haven/infrastructure/components/__init__.py +++ b/data_safe_haven/infrastructure/components/__init__.py @@ -13,8 +13,6 @@ from .dynamic import ( BlobContainerAcl, BlobContainerAclProps, - EntraApplication, - EntraApplicationProps, FileShareFile, FileShareFileProps, SSLCertificate, @@ -28,8 +26,6 @@ __all__ = [ "BlobContainerAcl", "BlobContainerAclProps", - "EntraApplication", - "EntraApplicationProps", "FileShareFile", "FileShareFileProps", "LinuxVMComponentProps", diff --git a/data_safe_haven/infrastructure/components/dynamic/__init__.py b/data_safe_haven/infrastructure/components/dynamic/__init__.py index 429fc8470d..78ecfbcef1 100644 --- a/data_safe_haven/infrastructure/components/dynamic/__init__.py +++ b/data_safe_haven/infrastructure/components/dynamic/__init__.py @@ -1,13 +1,10 @@ from .blob_container_acl import BlobContainerAcl, BlobContainerAclProps -from .entra_application import EntraApplication, EntraApplicationProps from .file_share_file import FileShareFile, FileShareFileProps from .ssl_certificate import SSLCertificate, SSLCertificateProps __all__ = [ "BlobContainerAcl", "BlobContainerAclProps", - "EntraApplication", - "EntraApplicationProps", "FileShareFile", "FileShareFileProps", "SSLCertificate", diff --git a/data_safe_haven/infrastructure/components/dynamic/entra_application.py b/data_safe_haven/infrastructure/components/dynamic/entra_application.py deleted file mode 100644 index fd2d233137..0000000000 --- a/data_safe_haven/infrastructure/components/dynamic/entra_application.py +++ /dev/null @@ -1,191 +0,0 @@ -"""Pulumi dynamic component for Entra applications.""" - -from contextlib import suppress -from typing import Any - -from pulumi import Input, Output, ResourceOptions -from pulumi.dynamic import CreateResult, DiffResult, Resource, UpdateResult - -from data_safe_haven.exceptions import DataSafeHavenMicrosoftGraphError -from data_safe_haven.external import GraphApi - -from .dsh_resource_provider import DshResourceProvider - - -class EntraApplicationProps: - """Props for the EntraApplication class""" - - def __init__( - self, - application_name: Input[str], - application_role_assignments: Input[list[str]] | None = None, - application_secret_name: Input[str] | None = None, - delegated_role_assignments: Input[list[str]] | None = None, - public_client_redirect_uri: Input[str] | None = None, - web_redirect_url: Input[str] | None = None, - ) -> None: - self.application_name = application_name - self.application_role_assignments = application_role_assignments - self.application_secret_name = application_secret_name - self.delegated_role_assignments = delegated_role_assignments - self.public_client_redirect_uri = public_client_redirect_uri - self.web_redirect_url = web_redirect_url - - -class EntraApplicationProvider(DshResourceProvider): - def __init__(self, auth_token: str): - self.auth_token = auth_token - super().__init__() - - def create(self, props: dict[str, Any]) -> CreateResult: - """Create new Entra application.""" - outs = dict(**props) - try: - graph_api = GraphApi.from_token(self.auth_token, disable_logging=True) - request_json = { - "displayName": props["application_name"], - "signInAudience": "AzureADMyOrg", - } - # Add a web redirection URL if requested - if props.get("web_redirect_url", None): - request_json["web"] = { - "redirectUris": [props["web_redirect_url"]], - "implicitGrantSettings": {"enableIdTokenIssuance": True}, - } - # Add a public client redirection URL if requested - if props.get("public_client_redirect_uri", None): - request_json["publicClient"] = { - "redirectUris": [props["public_client_redirect_uri"]], - } - json_response = graph_api.create_application( - props["application_name"], - application_scopes=props.get("application_role_assignments", []), - delegated_scopes=props.get("delegated_role_assignments", []), - request_json=request_json, - ) - outs["object_id"] = json_response["id"] - outs["application_id"] = json_response["appId"] - - # Grant requested role permissions - graph_api.grant_role_permissions( - outs["application_name"], - application_role_assignments=props.get( - "application_role_assignments", [] - ), - delegated_role_assignments=props.get("delegated_role_assignments", []), - ) - - # Attach an application secret if requested - outs["application_secret"] = ( - graph_api.create_application_secret( - props["application_name"], - props["application_secret_name"], - ) - if props.get("application_secret_name", None) - else "" - ) - except Exception as exc: - msg = f"Failed to create application '{props['application_name']}' in Entra ID." - raise DataSafeHavenMicrosoftGraphError(msg) from exc - return CreateResult( - f"EntraApplication-{props['application_name']}", - outs=outs, - ) - - def delete(self, id_: str, props: dict[str, Any]) -> None: - """Delete an Entra application.""" - # Use `id` as a no-op to avoid ARG002 while maintaining function signature - id(id_) - try: - graph_api = GraphApi.from_token(self.auth_token, disable_logging=True) - graph_api.delete_application(props["application_name"]) - except Exception as exc: - msg = f"Failed to delete application '{props['application_name']}' from Entra ID." - raise DataSafeHavenMicrosoftGraphError(msg) from exc - - def diff( - self, - id_: str, - old_props: dict[str, Any], - new_props: dict[str, Any], - ) -> DiffResult: - """Calculate diff between old and new state""" - # Use `id` as a no-op to avoid ARG002 while maintaining function signature - id(id_) - # We exclude '__provider' from the diff. This is a Base64-encoded pickle of this - # EntraApplicationProvider instance. This means that it contains self.auth_token - # and would otherwise trigger a diff each time the auth_token changes. Note that - # ignoring '__provider' could cause issues if the structure of this class - # changes in any other way, but this could be fixed by manually deleting the - # application in the Entra directory. - return self.partial_diff(old_props, new_props, excluded_props=["__provider"]) - - def refresh(self, props: dict[str, Any]) -> dict[str, Any]: - try: - outs = dict(**props) - with suppress(DataSafeHavenMicrosoftGraphError, KeyError): - graph_api = GraphApi.from_token(self.auth_token, disable_logging=True) - if json_response := graph_api.get_application_by_name( - outs["application_name"] - ): - outs["object_id"] = json_response["id"] - outs["application_id"] = json_response["appId"] - - # Ensure that requested role permissions have been granted - graph_api.grant_role_permissions( - outs["application_name"], - application_role_assignments=props.get( - "application_role_assignments", [] - ), - delegated_role_assignments=props.get( - "delegated_role_assignments", [] - ), - ) - return outs - except Exception as exc: - msg = f"Failed to refresh application '{props['application_name']}' in Entra ID." - raise DataSafeHavenMicrosoftGraphError(msg) from exc - - def update( - self, - id_: str, - old_props: dict[str, Any], - new_props: dict[str, Any], - ) -> UpdateResult: - """Updating is deleting followed by creating.""" - try: - # Delete the old application, using the auth token from new_props - old_props_ = {**old_props} - self.delete(id_, old_props_) - # Create a new application - updated = self.create(new_props) - return UpdateResult(outs=updated.outs) - except Exception as exc: - msg = f"Failed to update application '{new_props['application_name']}' in Entra ID." - raise DataSafeHavenMicrosoftGraphError(msg) from exc - - -class EntraApplication(Resource): - application_id: Output[str] - application_secret: Output[str] - object_id: Output[str] - _resource_type_name = "dsh:common:EntraApplication" # set resource type - - def __init__( - self, - name: str, - props: EntraApplicationProps, - auth_token: str, - opts: ResourceOptions | None = None, - ): - super().__init__( - EntraApplicationProvider(auth_token), - name, - { - "application_id": None, - "application_secret": None, - "object_id": None, - **vars(props), - }, - opts, - ) From c0d985756238c885300c64e0b8d620b1e0f0b853 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Wed, 23 Oct 2024 13:19:00 +0100 Subject: [PATCH 044/113] :bug: Use client_id instead of id for Entra applications --- data_safe_haven/infrastructure/programs/sre/entra.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/infrastructure/programs/sre/entra.py b/data_safe_haven/infrastructure/programs/sre/entra.py index b2d9e03821..70ee803d17 100644 --- a/data_safe_haven/infrastructure/programs/sre/entra.py +++ b/data_safe_haven/infrastructure/programs/sre/entra.py @@ -118,8 +118,8 @@ def __init__( ) # Register outputs - self.identity_application_id = self.identity_application.application.id + self.identity_application_id = self.identity_application.application.client_id self.identity_application_secret = self.identity_application_secret.value self.remote_desktop_application_id = ( - self.remote_desktop_application.application.id + self.remote_desktop_application.application.client_id ) From 4f9aa977fe542b9e82602a449c8461db83f981f1 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Wed, 23 Oct 2024 12:20:35 +0100 Subject: [PATCH 045/113] :recycle: Add an EntraAppPermissionType enum to ensure that Role/Scope are used consistently --- data_safe_haven/external/api/graph_api.py | 7 ++-- .../components/composite/entra_application.py | 37 ++++++++++--------- .../infrastructure/programs/sre/entra.py | 7 ++-- data_safe_haven/types/__init__.py | 2 + data_safe_haven/types/enums.py | 6 +++ 5 files changed, 35 insertions(+), 24 deletions(-) diff --git a/data_safe_haven/external/api/graph_api.py b/data_safe_haven/external/api/graph_api.py index ea78f928c3..8e9b4a8d73 100644 --- a/data_safe_haven/external/api/graph_api.py +++ b/data_safe_haven/external/api/graph_api.py @@ -17,6 +17,7 @@ DataSafeHavenValueError, ) from data_safe_haven.logging import get_logger, get_null_logger +from data_safe_haven.types import EntraAppPermissionType from .credentials import DeferredCredential, GraphApiCredential @@ -193,7 +194,6 @@ def create_application( if not request_json: request_json = { "displayName": application_name, - "signInAudience": "AzureADMyOrg", "passwordCredentials": [], "publicClient": { "redirectUris": [ @@ -201,18 +201,19 @@ def create_application( "urn:ietf:wg:oauth:2.0:oob", ] }, + "signInAudience": "AzureADMyOrg", } # Add scopes if there are any scopes = [ { "id": self.uuid_application[application_scope], - "type": "Role", # 'Role' is the type for application permissions + "type": EntraAppPermissionType.APPLICATION.value, } for application_scope in application_scopes ] + [ { "id": self.uuid_delegated[delegated_scope], - "type": "Scope", # 'Scope' is the type for delegated permissions + "type": EntraAppPermissionType.DELEGATED.value, } for delegated_scope in delegated_scopes ] diff --git a/data_safe_haven/infrastructure/components/composite/entra_application.py b/data_safe_haven/infrastructure/components/composite/entra_application.py index 1e234bbbce..52e977c1f5 100644 --- a/data_safe_haven/infrastructure/components/composite/entra_application.py +++ b/data_safe_haven/infrastructure/components/composite/entra_application.py @@ -7,6 +7,7 @@ from pulumi import ComponentResource, Input, Output, ResourceOptions from data_safe_haven.functions import replace_separators +from data_safe_haven.types import EntraAppPermissionType class EntraApplicationProps: @@ -15,7 +16,7 @@ class EntraApplicationProps: def __init__( self, application_name: Input[str], - application_permissions: list[tuple[str, str]], + application_permissions: list[tuple[EntraAppPermissionType, str]], msgraph_service_principal: Input[entra.ServicePrincipal], application_kwargs: Mapping[str, Any], ) -> None: @@ -31,10 +32,8 @@ def __init__( delegated=msgraph_service_principal.oauth2_permission_scope_ids, ).apply( lambda kwargs: { - # 'Role' permissions belong to the application - "Role": kwargs["application"], - # 'Scope' permissions are delegated to users - "Scope": kwargs["delegated"], + EntraAppPermissionType.APPLICATION: kwargs["application"], + EntraAppPermissionType.DELEGATED: kwargs["delegated"], } ) @@ -48,7 +47,7 @@ class EntraDesktopApplicationProps(EntraApplicationProps): def __init__( self, application_name: Input[str], - application_permissions: list[tuple[str, str]], + application_permissions: list[tuple[EntraAppPermissionType, str]], msgraph_service_principal: Input[entra.ServicePrincipal], ): super().__init__( @@ -72,7 +71,7 @@ class EntraWebApplicationProps(EntraApplicationProps): def __init__( self, application_name: Input[str], - application_permissions: list[tuple[str, str]], + application_permissions: list[tuple[EntraAppPermissionType, str]], msgraph_service_principal: Input[entra.ServicePrincipal], redirect_url: Input[str], ): @@ -112,10 +111,12 @@ def __init__( entra.ApplicationRequiredResourceAccessArgs( resource_accesses=[ entra.ApplicationRequiredResourceAccessResourceAccessArgs( - id=props.msgraph_permissions[scope][permission], - type=scope, + id=props.msgraph_permissions[permission_type][ + permission + ], + type=permission_type.value, ) - for scope, permission in props.application_permissions + for permission_type, permission in props.application_permissions ], resource_app_id=props.msgraph_client_id, ) @@ -134,23 +135,23 @@ def __init__( ) # Grant admin approval for requested application permissions - for scope, permission in props.application_permissions: - if scope == "application": + for permission_type, permission in props.application_permissions: + if permission_type == EntraAppPermissionType.APPLICATION: entra.AppRoleAssignment( replace_separators( - f"{self._name}_application_grant_{scope}_{permission}".lower(), + f"{self._name}_application_role_grant_{permission_type.value}_{permission}", "_", - ), - app_role_id=props.msgraph_permissions[scope][permission], + ).lower(), + app_role_id=props.msgraph_permissions[permission_type][permission], principal_object_id=self.application_service_principal.object_id, resource_object_id=props.msgraph_object_id, ) - if scope == "delegated": + if permission_type == EntraAppPermissionType.DELEGATED: entra.ServicePrincipalDelegatedPermissionGrant( replace_separators( - f"{self._name}_application_delegated_grant_{scope}_{permission}".lower(), + f"{self._name}_application_delegated_grant_{permission_type.value}_{permission}", "_", - ), + ).lower(), claim_values=[permission], resource_service_principal_object_id=props.msgraph_object_id, service_principal_object_id=self.application_service_principal.object_id, diff --git a/data_safe_haven/infrastructure/programs/sre/entra.py b/data_safe_haven/infrastructure/programs/sre/entra.py index 70ee803d17..b460113189 100644 --- a/data_safe_haven/infrastructure/programs/sre/entra.py +++ b/data_safe_haven/infrastructure/programs/sre/entra.py @@ -11,6 +11,7 @@ EntraDesktopApplicationProps, EntraWebApplicationProps, ) +from data_safe_haven.types import EntraAppPermissionType class SREEntraProps: @@ -77,9 +78,9 @@ def __init__( ") Identity Service Principal", ), application_permissions=[ - ("Role", "User.Read.All"), - ("Role", "GroupMember.Read.All"), - ("Scope", "User.Read.All"), + (EntraAppPermissionType.APPLICATION, "User.Read.All"), + (EntraAppPermissionType.APPLICATION, "GroupMember.Read.All"), + (EntraAppPermissionType.DELEGATED, "User.Read.All"), ], msgraph_service_principal=msgraph_service_principal, ), diff --git a/data_safe_haven/types/__init__.py b/data_safe_haven/types/__init__.py index 4f2f89b3be..185a39ab10 100644 --- a/data_safe_haven/types/__init__.py +++ b/data_safe_haven/types/__init__.py @@ -17,6 +17,7 @@ AzureSdkCredentialScope, AzureServiceTag, DatabaseSystem, + EntraAppPermissionType, FirewallPriorities, ForbiddenDomains, NetworkingPriorities, @@ -36,6 +37,7 @@ "AzureVmSku", "DatabaseSystem", "EmailAddress", + "EntraAppPermissionType", "EntraGroupName", "FirewallPriorities", "ForbiddenDomains", diff --git a/data_safe_haven/types/enums.py b/data_safe_haven/types/enums.py index 35465f260e..85c5b4c15b 100644 --- a/data_safe_haven/types/enums.py +++ b/data_safe_haven/types/enums.py @@ -37,6 +37,12 @@ class DatabaseSystem(str, Enum): POSTGRESQL = "postgresql" +@verify(UNIQUE) +class EntraAppPermissionType(str, Enum): + APPLICATION = "Role" + DELEGATED = "Scope" + + @verify(UNIQUE) class FirewallPriorities(int, Enum): """Priorities for firewall rules.""" From 89d966215661b915707db0f34fdf813029eb57d3 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Wed, 23 Oct 2024 19:48:46 +0100 Subject: [PATCH 046/113] :truck: Simplify import path for Entra components --- data_safe_haven/infrastructure/components/__init__.py | 6 ++++++ .../infrastructure/components/composite/__init__.py | 8 ++++++++ data_safe_haven/infrastructure/programs/sre/entra.py | 2 +- 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/data_safe_haven/infrastructure/components/__init__.py b/data_safe_haven/infrastructure/components/__init__.py index 5ff8b92baf..2b3dd67e7a 100644 --- a/data_safe_haven/infrastructure/components/__init__.py +++ b/data_safe_haven/infrastructure/components/__init__.py @@ -1,4 +1,7 @@ from .composite import ( + EntraApplicationComponent, + EntraDesktopApplicationProps, + EntraWebApplicationProps, LinuxVMComponentProps, LocalDnsRecordComponent, LocalDnsRecordProps, @@ -26,6 +29,9 @@ __all__ = [ "BlobContainerAcl", "BlobContainerAclProps", + "EntraApplicationComponent", + "EntraDesktopApplicationProps", + "EntraWebApplicationProps", "FileShareFile", "FileShareFileProps", "LinuxVMComponentProps", diff --git a/data_safe_haven/infrastructure/components/composite/__init__.py b/data_safe_haven/infrastructure/components/composite/__init__.py index e4254a50ed..bc09bc18a8 100644 --- a/data_safe_haven/infrastructure/components/composite/__init__.py +++ b/data_safe_haven/infrastructure/components/composite/__init__.py @@ -1,3 +1,8 @@ +from .entra_application import ( + EntraApplicationComponent, + EntraDesktopApplicationProps, + EntraWebApplicationProps, +) from .local_dns_record import LocalDnsRecordComponent, LocalDnsRecordProps from .microsoft_sql_database import ( MicrosoftSQLDatabaseComponent, @@ -8,6 +13,9 @@ from .virtual_machine import LinuxVMComponentProps, VMComponent __all__ = [ + "EntraApplicationComponent", + "EntraDesktopApplicationProps", + "EntraWebApplicationProps", "LinuxVMComponentProps", "LocalDnsRecordComponent", "LocalDnsRecordProps", diff --git a/data_safe_haven/infrastructure/programs/sre/entra.py b/data_safe_haven/infrastructure/programs/sre/entra.py index b460113189..a3f2bb78da 100644 --- a/data_safe_haven/infrastructure/programs/sre/entra.py +++ b/data_safe_haven/infrastructure/programs/sre/entra.py @@ -6,7 +6,7 @@ from pulumi import ComponentResource, Input, Output, ResourceOptions from data_safe_haven.functions import replace_separators -from data_safe_haven.infrastructure.components.composite.entra_application import ( +from data_safe_haven.infrastructure.components import ( EntraApplicationComponent, EntraDesktopApplicationProps, EntraWebApplicationProps, From 463f4909af5420dae51c86931668b9edaa339498 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Thu, 24 Oct 2024 11:05:25 +0100 Subject: [PATCH 047/113] :coffin: Drop requirements for GraphAPI everywhere in the SRE where they are used to seed the pulumi-azuread provider --- data_safe_haven/commands/pulumi.py | 15 +-------------- data_safe_haven/commands/sre.py | 12 +----------- .../infrastructure/programs/declarative_sre.py | 3 --- .../infrastructure/programs/sre/remote_desktop.py | 2 -- data_safe_haven/infrastructure/project_manager.py | 4 +--- .../provisioning/sre_provisioning_manager.py | 3 --- tests/commands/conftest.py | 5 ----- tests/commands/test_pulumi.py | 3 --- tests/commands/test_shm.py | 1 - tests/commands/test_sre.py | 4 ---- 10 files changed, 3 insertions(+), 49 deletions(-) diff --git a/data_safe_haven/commands/pulumi.py b/data_safe_haven/commands/pulumi.py index 7ad9506f0b..13f0795de0 100644 --- a/data_safe_haven/commands/pulumi.py +++ b/data_safe_haven/commands/pulumi.py @@ -6,8 +6,7 @@ import typer from data_safe_haven import console -from data_safe_haven.config import ContextManager, DSHPulumiConfig, SHMConfig, SREConfig -from data_safe_haven.external import GraphApi +from data_safe_haven.config import ContextManager, DSHPulumiConfig, SREConfig from data_safe_haven.infrastructure import SREProjectManager pulumi_command_group = typer.Typer() @@ -33,24 +32,12 @@ def run( """Run arbitrary Pulumi commands in a DSH project""" context = ContextManager.from_file().assert_context() pulumi_config = DSHPulumiConfig.from_remote(context) - shm_config = SHMConfig.from_remote(context) sre_config = SREConfig.from_remote_by_name(context, sre_name) - graph_api = GraphApi.from_scopes( - scopes=[ - "Application.ReadWrite.All", - "AppRoleAssignment.ReadWrite.All", - "Directory.ReadWrite.All", - "Group.ReadWrite.All", - ], - tenant_id=shm_config.shm.entra_tenant_id, - ) - project = SREProjectManager( context=context, config=sre_config, pulumi_config=pulumi_config, - graph_api_token=graph_api.token, ) stdout = project.run_pulumi_command(command) diff --git a/data_safe_haven/commands/sre.py b/data_safe_haven/commands/sre.py index f03f0cc53e..6762716f78 100644 --- a/data_safe_haven/commands/sre.py +++ b/data_safe_haven/commands/sre.py @@ -66,7 +66,6 @@ def deploy( config=sre_config, pulumi_config=pulumi_config, create_project=True, - graph_api_token=graph_api.token, ) # Set Azure options stack.add_option( @@ -153,7 +152,6 @@ def deploy( # Provision SRE with anything that could not be done in Pulumi manager = SREProvisioningManager( - graph_api_token=graph_api.token, location=sre_config.azure.location, sre_name=sre_config.name, sre_stack=stack, @@ -183,15 +181,8 @@ def teardown( """Tear down a deployed a Secure Research Environment.""" logger = get_logger() try: - # Load context and SHM config + # Load context context = ContextManager.from_file().assert_context() - shm_config = SHMConfig.from_remote(context) - - # Load GraphAPI as this may require user-interaction - graph_api = GraphApi.from_scopes( - scopes=["Application.ReadWrite.All", "Group.ReadWrite.All"], - tenant_id=shm_config.shm.entra_tenant_id, - ) # Load Pulumi and SRE configs pulumi_config = DSHPulumiConfig.from_remote(context) @@ -212,7 +203,6 @@ def teardown( context=context, config=sre_config, pulumi_config=pulumi_config, - graph_api_token=graph_api.token, create_project=True, ) stack.teardown(force=force) diff --git a/data_safe_haven/infrastructure/programs/declarative_sre.py b/data_safe_haven/infrastructure/programs/declarative_sre.py index 1c81e8324e..15989bbe7b 100644 --- a/data_safe_haven/infrastructure/programs/declarative_sre.py +++ b/data_safe_haven/infrastructure/programs/declarative_sre.py @@ -35,11 +35,9 @@ def __init__( self, context: Context, config: SREConfig, - graph_api_token: str, ) -> None: self.context = context self.config = config - self.graph_api_token = graph_api_token self.stack_name = replace_separators( f"shm-{context.name}-sre-{config.name}", "-" ) @@ -293,7 +291,6 @@ def __call__(self) -> None: dockerhub_credentials=dockerhub_credentials, entra_application_id=entra.remote_desktop_application_id, entra_application_url=entra.remote_desktop_url, - entra_auth_token=self.graph_api_token, entra_tenant_id=shm_entra_tenant_id, ldap_group_filter=ldap_group_filter, ldap_group_search_base=ldap_group_search_base, diff --git a/data_safe_haven/infrastructure/programs/sre/remote_desktop.py b/data_safe_haven/infrastructure/programs/sre/remote_desktop.py index c6cf26ffdb..e2df83ede5 100644 --- a/data_safe_haven/infrastructure/programs/sre/remote_desktop.py +++ b/data_safe_haven/infrastructure/programs/sre/remote_desktop.py @@ -32,7 +32,6 @@ def __init__( dockerhub_credentials: DockerHubCredentials, entra_application_id: Input[str], entra_application_url: Input[str], - entra_auth_token: str, entra_tenant_id: Input[str], ldap_group_filter: Input[str], ldap_group_search_base: Input[str], @@ -58,7 +57,6 @@ def __init__( self.dockerhub_credentials = dockerhub_credentials self.entra_application_id = entra_application_id self.entra_application_url = entra_application_url - self.entra_auth_token = entra_auth_token self.entra_tenant_id = entra_tenant_id self.ldap_group_filter = ldap_group_filter self.ldap_group_search_base = ldap_group_search_base diff --git a/data_safe_haven/infrastructure/project_manager.py b/data_safe_haven/infrastructure/project_manager.py index a6d5af805b..eca352b28b 100644 --- a/data_safe_haven/infrastructure/project_manager.py +++ b/data_safe_haven/infrastructure/project_manager.py @@ -446,14 +446,12 @@ def __init__( pulumi_config: DSHPulumiConfig, *, create_project: bool = False, - graph_api_token: str | None = None, ) -> None: """Constructor""" - token = graph_api_token or "" super().__init__( context, pulumi_config, config.name, - DeclarativeSRE(context, config, token), + DeclarativeSRE(context, config), create_project=create_project, ) diff --git a/data_safe_haven/provisioning/sre_provisioning_manager.py b/data_safe_haven/provisioning/sre_provisioning_manager.py index 7c39046b86..b269779d8f 100644 --- a/data_safe_haven/provisioning/sre_provisioning_manager.py +++ b/data_safe_haven/provisioning/sre_provisioning_manager.py @@ -7,7 +7,6 @@ AzureContainerInstance, AzurePostgreSQLDatabase, AzureSdk, - GraphApi, ) from data_safe_haven.infrastructure import SREProjectManager from data_safe_haven.logging import get_logger @@ -19,7 +18,6 @@ class SREProvisioningManager: def __init__( self, - graph_api_token: str, location: AzureLocation, sre_name: str, sre_stack: SREProjectManager, @@ -28,7 +26,6 @@ def __init__( ): self._available_vm_skus: dict[str, dict[str, Any]] | None = None self.location = location - self.graph_api = GraphApi.from_token(graph_api_token) self.logger = get_logger() self.sre_name = sre_name self.subscription_name = subscription_name diff --git a/tests/commands/conftest.py b/tests/commands/conftest.py index dab10adb7b..dc6811ff9f 100644 --- a/tests/commands/conftest.py +++ b/tests/commands/conftest.py @@ -48,11 +48,6 @@ def mock_graph_api_get_application_by_name(mocker, request): ) -@fixture -def mock_graph_api_token(mocker): - mocker.patch.object(GraphApi, "token", return_value="dummy-token") - - @fixture def mock_imperative_shm_deploy(mocker): mocker.patch.object( diff --git a/tests/commands/test_pulumi.py b/tests/commands/test_pulumi.py index fefb4615fc..b55d2974ec 100644 --- a/tests/commands/test_pulumi.py +++ b/tests/commands/test_pulumi.py @@ -6,7 +6,6 @@ def test_run_sre( self, runner, local_project_settings, # noqa: ARG002 - mock_graph_api_token, # noqa: ARG002 mock_install_plugins, # noqa: ARG002 mock_key_vault_key, # noqa: ARG002 mock_pulumi_config_no_key_from_remote, # noqa: ARG002 @@ -30,7 +29,6 @@ def test_run_sre_invalid_command( self, runner, local_project_settings, # noqa: ARG002 - mock_graph_api_token, # noqa: ARG002 mock_install_plugins, # noqa: ARG002 mock_key_vault_key, # noqa: ARG002 mock_pulumi_config_no_key_from_remote, # noqa: ARG002 @@ -48,7 +46,6 @@ def test_run_sre_invalid_name( self, runner, local_project_settings, # noqa: ARG002 - mock_graph_api_token, # noqa: ARG002 mock_install_plugins, # noqa: ARG002 mock_key_vault_key, # noqa: ARG002 mock_pulumi_config_no_key_from_remote, # noqa: ARG002 diff --git a/tests/commands/test_shm.py b/tests/commands/test_shm.py index 8258d2a16a..6480a9531f 100644 --- a/tests/commands/test_shm.py +++ b/tests/commands/test_shm.py @@ -7,7 +7,6 @@ def test_infrastructure_deploy( runner, mock_imperative_shm_deploy_then_exit, # noqa: ARG002 mock_graph_api_add_custom_domain, # noqa: ARG002 - mock_graph_api_token, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 mock_shm_config_remote_exists, # noqa: ARG002 mock_shm_config_upload, # noqa: ARG002 diff --git a/tests/commands/test_sre.py b/tests/commands/test_sre.py index 6c4a13f545..1e5f0aabfc 100644 --- a/tests/commands/test_sre.py +++ b/tests/commands/test_sre.py @@ -13,7 +13,6 @@ def test_deploy( self, runner: CliRunner, mock_azuresdk_get_subscription_name, # noqa: ARG002 - mock_graph_api_token, # noqa: ARG002 mock_contextmanager_assert_context, # noqa: ARG002 mock_ip_1_2_3_4, # noqa: ARG002 mock_pulumi_config_from_remote_or_create, # noqa: ARG002 @@ -34,7 +33,6 @@ def test_no_application( runner: CliRunner, mock_azuresdk_get_subscription_name, # noqa: ARG002 mock_contextmanager_assert_context, # noqa: ARG002 - mock_graph_api_token, # noqa: ARG002 mock_ip_1_2_3_4, # noqa: ARG002 mock_pulumi_config_from_remote_or_create, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 @@ -56,7 +54,6 @@ def test_no_application_secret( mocker: MockerFixture, mock_azuresdk_get_subscription_name, # noqa: ARG002 mock_graph_api_get_application_by_name, # noqa: ARG002 - mock_graph_api_token, # noqa: ARG002 mock_ip_1_2_3_4, # noqa: ARG002 mock_pulumi_config_from_remote_or_create, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 @@ -104,7 +101,6 @@ class TestTeardownSRE: def test_teardown( self, runner: CliRunner, - mock_graph_api_token, # noqa: ARG002 mock_ip_1_2_3_4, # noqa: ARG002 mock_pulumi_config_from_remote, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 From 8942257b70cb08670183a6ac8b82a52a5b46900f Mon Sep 17 00:00:00 2001 From: James Robinson Date: Fri, 25 Oct 2024 10:28:04 +0100 Subject: [PATCH 048/113] :alien: Do not replace azuread:clientId or azuread:tenantId in stack options since these set the provider information in the state file and cannot be changed --- data_safe_haven/commands/sre.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/commands/sre.py b/data_safe_haven/commands/sre.py index 6762716f78..e7b781cdbf 100644 --- a/data_safe_haven/commands/sre.py +++ b/data_safe_haven/commands/sre.py @@ -98,7 +98,9 @@ def deploy( if not application: msg = f"No Entra application '{context.entra_application_name}' was found. Please redeploy your SHM." raise DataSafeHavenConfigError(msg) - stack.add_option("azuread:clientId", application.get("appId", ""), replace=True) + stack.add_option( + "azuread:clientId", application.get("appId", ""), replace=False + ) if not context.entra_application_secret: msg = f"No Entra application secret '{context.entra_application_secret_name}' was found. Please redeploy your SHM." raise DataSafeHavenConfigError(msg) @@ -106,7 +108,7 @@ def deploy( "azuread:clientSecret", context.entra_application_secret, replace=True ) stack.add_option( - "azuread:tenantId", shm_config.shm.entra_tenant_id, replace=True + "azuread:tenantId", shm_config.shm.entra_tenant_id, replace=False ) # Load SHM outputs stack.add_option( From 23918b9b4a3e6a4a8f85cb9aa49e5212955f340a Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 28 Oct 2024 14:48:26 +0000 Subject: [PATCH 049/113] :wrench: Add an EntraSignInAudienceType enum --- data_safe_haven/external/api/graph_api.py | 4 ++-- .../components/composite/entra_application.py | 4 ++-- data_safe_haven/infrastructure/programs/imperative_shm.py | 3 ++- data_safe_haven/types/__init__.py | 2 ++ data_safe_haven/types/enums.py | 8 ++++++++ 5 files changed, 16 insertions(+), 5 deletions(-) diff --git a/data_safe_haven/external/api/graph_api.py b/data_safe_haven/external/api/graph_api.py index 8e9b4a8d73..ca717d3ee7 100644 --- a/data_safe_haven/external/api/graph_api.py +++ b/data_safe_haven/external/api/graph_api.py @@ -17,7 +17,7 @@ DataSafeHavenValueError, ) from data_safe_haven.logging import get_logger, get_null_logger -from data_safe_haven.types import EntraAppPermissionType +from data_safe_haven.types import EntraAppPermissionType, EntraSignInAudienceType from .credentials import DeferredCredential, GraphApiCredential @@ -201,7 +201,7 @@ def create_application( "urn:ietf:wg:oauth:2.0:oob", ] }, - "signInAudience": "AzureADMyOrg", + "signInAudience": EntraSignInAudienceType.THIS_TENANT.value, } # Add scopes if there are any scopes = [ diff --git a/data_safe_haven/infrastructure/components/composite/entra_application.py b/data_safe_haven/infrastructure/components/composite/entra_application.py index 52e977c1f5..4dd68da425 100644 --- a/data_safe_haven/infrastructure/components/composite/entra_application.py +++ b/data_safe_haven/infrastructure/components/composite/entra_application.py @@ -7,7 +7,7 @@ from pulumi import ComponentResource, Input, Output, ResourceOptions from data_safe_haven.functions import replace_separators -from data_safe_haven.types import EntraAppPermissionType +from data_safe_haven.types import EntraAppPermissionType, EntraSignInAudienceType class EntraApplicationProps: @@ -124,7 +124,7 @@ def __init__( if props.application_permissions else [] ), - sign_in_audience="AzureADMyOrg", + sign_in_audience=EntraSignInAudienceType.THIS_TENANT.value, **props.application_kwargs, ) diff --git a/data_safe_haven/infrastructure/programs/imperative_shm.py b/data_safe_haven/infrastructure/programs/imperative_shm.py index 1095f26db2..796026d735 100644 --- a/data_safe_haven/infrastructure/programs/imperative_shm.py +++ b/data_safe_haven/infrastructure/programs/imperative_shm.py @@ -5,6 +5,7 @@ ) from data_safe_haven.external import AzureSdk, GraphApi from data_safe_haven.logging import get_logger +from data_safe_haven.types import EntraSignInAudienceType class ImperativeSHM: @@ -156,7 +157,7 @@ def deploy(self) -> None: delegated_scopes=[], request_json={ "displayName": self.context.entra_application_name, - "signInAudience": "AzureADMyOrg", + "signInAudience": EntraSignInAudienceType.THIS_TENANT.value, }, ) # Always recreate the application secret. diff --git a/data_safe_haven/types/__init__.py b/data_safe_haven/types/__init__.py index 185a39ab10..cf631b0229 100644 --- a/data_safe_haven/types/__init__.py +++ b/data_safe_haven/types/__init__.py @@ -18,6 +18,7 @@ AzureServiceTag, DatabaseSystem, EntraAppPermissionType, + EntraSignInAudienceType, FirewallPriorities, ForbiddenDomains, NetworkingPriorities, @@ -39,6 +40,7 @@ "EmailAddress", "EntraAppPermissionType", "EntraGroupName", + "EntraSignInAudienceType", "FirewallPriorities", "ForbiddenDomains", "Fqdn", diff --git a/data_safe_haven/types/enums.py b/data_safe_haven/types/enums.py index 85c5b4c15b..7096e478ce 100644 --- a/data_safe_haven/types/enums.py +++ b/data_safe_haven/types/enums.py @@ -43,6 +43,14 @@ class EntraAppPermissionType(str, Enum): DELEGATED = "Scope" +@verify(UNIQUE) +class EntraSignInAudienceType(str, Enum): + ANY_TENANT = "AzureADMultipleOrgs" + ANY_TENANT_OR_PERSONAL = "AzureADandPersonalMicrosoftAccount" + PERSONAL = "PersonalMicrosoftAccount" + THIS_TENANT = "AzureADMyOrg" + + @verify(UNIQUE) class FirewallPriorities(int, Enum): """Priorities for firewall rules.""" From e740ba0749ca82ad74e9f3b4c0bd0e05bead7906 Mon Sep 17 00:00:00 2001 From: James Robinson Date: Mon, 28 Oct 2024 14:52:51 +0000 Subject: [PATCH 050/113] :loud_sound: Add comments explaining application scope requirements --- data_safe_haven/infrastructure/programs/imperative_shm.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/data_safe_haven/infrastructure/programs/imperative_shm.py b/data_safe_haven/infrastructure/programs/imperative_shm.py index 796026d735..d08fe891b4 100644 --- a/data_safe_haven/infrastructure/programs/imperative_shm.py +++ b/data_safe_haven/infrastructure/programs/imperative_shm.py @@ -149,10 +149,10 @@ def deploy(self) -> None: graph_api.create_application( self.context.entra_application_name, application_scopes=[ - "Application.ReadWrite.All", - "AppRoleAssignment.ReadWrite.All", - "Directory.ReadWrite.All", - "Group.ReadWrite.All", + "Application.ReadWrite.All", # For creating applications + "AppRoleAssignment.ReadWrite.All", # For application permissions + "Directory.ReadWrite.All", # For creating/deleting groups + "Group.ReadWrite.All", # For creating/deleting groups ], delegated_scopes=[], request_json={ From 7ef3d6cbbeac2915ce6a65e4430aa662a541f4cb Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 29 Oct 2024 09:41:44 +0000 Subject: [PATCH 051/113] :art: Add an enum for Entra application IDs --- data_safe_haven/external/api/graph_api.py | 11 ++++++----- data_safe_haven/infrastructure/programs/sre/entra.py | 7 ++----- data_safe_haven/types/__init__.py | 2 ++ data_safe_haven/types/enums.py | 5 +++++ 4 files changed, 15 insertions(+), 10 deletions(-) diff --git a/data_safe_haven/external/api/graph_api.py b/data_safe_haven/external/api/graph_api.py index ca717d3ee7..7d3b088672 100644 --- a/data_safe_haven/external/api/graph_api.py +++ b/data_safe_haven/external/api/graph_api.py @@ -17,7 +17,11 @@ DataSafeHavenValueError, ) from data_safe_haven.logging import get_logger, get_null_logger -from data_safe_haven.types import EntraAppPermissionType, EntraSignInAudienceType +from data_safe_haven.types import ( + EntraApplicationId, + EntraAppPermissionType, + EntraSignInAudienceType, +) from .credentials import DeferredCredential, GraphApiCredential @@ -25,9 +29,6 @@ class GraphApi: """Interface to the Microsoft Graph REST API""" - application_ids: ClassVar[dict[str, str]] = { - "Microsoft Graph": "00000003-0000-0000-c000-000000000000", - } role_template_ids: ClassVar[dict[str, str]] = { "Global Administrator": "62e90394-69f5-4237-9190-012177145e10" } @@ -220,7 +221,7 @@ def create_application( if scopes: request_json["requiredResourceAccess"] = [ { - "resourceAppId": self.application_ids["Microsoft Graph"], + "resourceAppId": EntraApplicationId.MICROSOFT_GRAPH.value, "resourceAccess": scopes, } ] diff --git a/data_safe_haven/infrastructure/programs/sre/entra.py b/data_safe_haven/infrastructure/programs/sre/entra.py index a3f2bb78da..abc0241070 100644 --- a/data_safe_haven/infrastructure/programs/sre/entra.py +++ b/data_safe_haven/infrastructure/programs/sre/entra.py @@ -11,7 +11,7 @@ EntraDesktopApplicationProps, EntraWebApplicationProps, ) -from data_safe_haven.types import EntraAppPermissionType +from data_safe_haven.types import EntraApplicationId, EntraAppPermissionType class SREEntraProps: @@ -54,12 +54,9 @@ def __init__( ) # Get the Microsoft Graph service principal - well_known = entra.get_application_published_app_ids_output() msgraph_service_principal = entra.ServicePrincipal( f"{self._name}_microsoft_graph_service_principal", - client_id=well_known.apply( - lambda app_ids: app_ids.result["MicrosoftGraph"] - ), + client_id=EntraApplicationId.MICROSOFT_GRAPH.value, use_existing=True, ) diff --git a/data_safe_haven/types/__init__.py b/data_safe_haven/types/__init__.py index cf631b0229..e12c76bdc6 100644 --- a/data_safe_haven/types/__init__.py +++ b/data_safe_haven/types/__init__.py @@ -17,6 +17,7 @@ AzureSdkCredentialScope, AzureServiceTag, DatabaseSystem, + EntraApplicationId, EntraAppPermissionType, EntraSignInAudienceType, FirewallPriorities, @@ -38,6 +39,7 @@ "AzureVmSku", "DatabaseSystem", "EmailAddress", + "EntraApplicationId", "EntraAppPermissionType", "EntraGroupName", "EntraSignInAudienceType", diff --git a/data_safe_haven/types/enums.py b/data_safe_haven/types/enums.py index 7096e478ce..17d5dda8e3 100644 --- a/data_safe_haven/types/enums.py +++ b/data_safe_haven/types/enums.py @@ -37,6 +37,11 @@ class DatabaseSystem(str, Enum): POSTGRESQL = "postgresql" +@verify(UNIQUE) +class EntraApplicationId(str, Enum): + MICROSOFT_GRAPH = "00000003-0000-0000-c000-000000000000" + + @verify(UNIQUE) class EntraAppPermissionType(str, Enum): APPLICATION = "Role" From 78033544a9be300317d7aa6730acf94d1c83422d Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 09:57:12 +0000 Subject: [PATCH 052/113] Fix tests for teardown --- tests/commands/test_shm.py | 4 ++++ tests/commands/test_sre.py | 3 +++ 2 files changed, 7 insertions(+) diff --git a/tests/commands/test_shm.py b/tests/commands/test_shm.py index 8258d2a16a..3366ad7cb4 100644 --- a/tests/commands/test_shm.py +++ b/tests/commands/test_shm.py @@ -1,3 +1,5 @@ +from rich.prompt import Confirm + from data_safe_haven.commands.shm import shm_command_group @@ -41,10 +43,12 @@ def test_infrastructure_auth_failure( class TestTeardownSHM: def test_teardown( self, + mocker, runner, mock_imperative_shm_teardown_then_exit, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 ): + mocker.patch.object(Confirm, "ask", return_value="yes") result = runner.invoke(shm_command_group, ["teardown"]) assert result.exit_code == 1 assert "mock teardown" in result.stdout diff --git a/tests/commands/test_sre.py b/tests/commands/test_sre.py index 6c4a13f545..431f783d85 100644 --- a/tests/commands/test_sre.py +++ b/tests/commands/test_sre.py @@ -1,5 +1,6 @@ from pytest import CaptureFixture, LogCaptureFixture from pytest_mock import MockerFixture +from rich.prompt import Confirm from typer.testing import CliRunner from data_safe_haven.commands.sre import sre_command_group @@ -103,6 +104,7 @@ def test_no_shm( class TestTeardownSRE: def test_teardown( self, + mocker: MockerFixture, runner: CliRunner, mock_graph_api_token, # noqa: ARG002 mock_ip_1_2_3_4, # noqa: ARG002 @@ -111,6 +113,7 @@ def test_teardown( mock_sre_config_from_remote, # noqa: ARG002 mock_sre_project_manager_teardown_then_exit, # noqa: ARG002 ) -> None: + mocker.patch.object(Confirm, "ask", return_value="yes") result = runner.invoke(sre_command_group, ["teardown", "sandbox"]) assert result.exit_code == 1 assert "mock teardown" in result.stdout From fd340423cef5a1b5dde8f315096688599ec694ad Mon Sep 17 00:00:00 2001 From: James Robinson Date: Tue, 29 Oct 2024 11:13:47 +0000 Subject: [PATCH 053/113] :recycle: Refactor following suggestions from code review --- .../components/composite/entra_application.py | 47 ++++++++++--------- 1 file changed, 26 insertions(+), 21 deletions(-) diff --git a/data_safe_haven/infrastructure/components/composite/entra_application.py b/data_safe_haven/infrastructure/components/composite/entra_application.py index 4dd68da425..e5bcff949b 100644 --- a/data_safe_haven/infrastructure/components/composite/entra_application.py +++ b/data_safe_haven/infrastructure/components/composite/entra_application.py @@ -135,24 +135,29 @@ def __init__( ) # Grant admin approval for requested application permissions - for permission_type, permission in props.application_permissions: - if permission_type == EntraAppPermissionType.APPLICATION: - entra.AppRoleAssignment( - replace_separators( - f"{self._name}_application_role_grant_{permission_type.value}_{permission}", - "_", - ).lower(), - app_role_id=props.msgraph_permissions[permission_type][permission], - principal_object_id=self.application_service_principal.object_id, - resource_object_id=props.msgraph_object_id, - ) - if permission_type == EntraAppPermissionType.DELEGATED: - entra.ServicePrincipalDelegatedPermissionGrant( - replace_separators( - f"{self._name}_application_delegated_grant_{permission_type.value}_{permission}", - "_", - ).lower(), - claim_values=[permission], - resource_service_principal_object_id=props.msgraph_object_id, - service_principal_object_id=self.application_service_principal.object_id, - ) + [ + entra.AppRoleAssignment( + replace_separators( + f"{self._name}_application_role_grant_{permission_type.value}_{permission}", + "_", + ).lower(), + app_role_id=props.msgraph_permissions[permission_type][permission], + principal_object_id=self.application_service_principal.object_id, + resource_object_id=props.msgraph_object_id, + ) + for permission_type, permission in props.application_permissions + if permission_type == EntraAppPermissionType.APPLICATION + ] + [ + entra.ServicePrincipalDelegatedPermissionGrant( + replace_separators( + f"{self._name}_application_delegated_grant_{permission_type.value}_{permission}", + "_", + ).lower(), + claim_values=[permission], + resource_service_principal_object_id=props.msgraph_object_id, + service_principal_object_id=self.application_service_principal.object_id, + ) + for permission_type, permission in props.application_permissions + if permission_type == EntraAppPermissionType.DELEGATED + ] From becfeaef74fa9403e55c81ffcf43a707976df319 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 12:27:46 +0000 Subject: [PATCH 054/113] use runner input instead of patching confirm --- tests/commands/test_sre.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/commands/test_sre.py b/tests/commands/test_sre.py index 431f783d85..aa74ccd428 100644 --- a/tests/commands/test_sre.py +++ b/tests/commands/test_sre.py @@ -113,8 +113,7 @@ def test_teardown( mock_sre_config_from_remote, # noqa: ARG002 mock_sre_project_manager_teardown_then_exit, # noqa: ARG002 ) -> None: - mocker.patch.object(Confirm, "ask", return_value="yes") - result = runner.invoke(sre_command_group, ["teardown", "sandbox"]) + result = runner.invoke(sre_command_group, ["teardown", "sandbox"], input="y") assert result.exit_code == 1 assert "mock teardown" in result.stdout From 418b7914921967f26a15ec019f134037a8e74378 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 12:28:05 +0000 Subject: [PATCH 055/113] remove unnecessary import --- tests/commands/test_sre.py | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/commands/test_sre.py b/tests/commands/test_sre.py index aa74ccd428..6f373da351 100644 --- a/tests/commands/test_sre.py +++ b/tests/commands/test_sre.py @@ -1,6 +1,5 @@ from pytest import CaptureFixture, LogCaptureFixture from pytest_mock import MockerFixture -from rich.prompt import Confirm from typer.testing import CliRunner from data_safe_haven.commands.sre import sre_command_group From f124e9c41864b7cc2bb48c335157b642b15077c6 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 12:28:41 +0000 Subject: [PATCH 056/113] Use runner for input and add test for when SREs are deployed --- tests/commands/test_shm.py | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/tests/commands/test_shm.py b/tests/commands/test_shm.py index 3366ad7cb4..17ac1f178c 100644 --- a/tests/commands/test_shm.py +++ b/tests/commands/test_shm.py @@ -1,7 +1,5 @@ -from rich.prompt import Confirm - from data_safe_haven.commands.shm import shm_command_group - +from data_safe_haven.config import DSHPulumiConfig class TestDeploySHM: def test_infrastructure_deploy( @@ -48,8 +46,7 @@ def test_teardown( mock_imperative_shm_teardown_then_exit, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 ): - mocker.patch.object(Confirm, "ask", return_value="yes") - result = runner.invoke(shm_command_group, ["teardown"]) + result = runner.invoke(shm_command_group, ["teardown"], input="y") assert result.exit_code == 1 assert "mock teardown" in result.stdout @@ -73,3 +70,15 @@ def test_auth_failure( assert "mock get_credential\n" in result.stdout assert "mock get_credential error" in result.stdout assert "Could not teardown Safe Haven Management environment." in result.stdout + + def test_teardown_sres_exist( + self, + mocker, + runner, + mock_azuresdk_get_subscription_name, # noqa: ARG002 + mock_pulumi_config_from_remote, # noqa: ARG002 + mock_shm_config_from_remote, # noqa: ARG002 + ): + result = runner.invoke(shm_command_group, ["teardown"], input="y") + assert result.exit_code == 1 + assert "Found deployed SREs" in result.stdout \ No newline at end of file From 309f621bf88250e150e6bde1a9c94ab163a0451b Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 13:19:27 +0000 Subject: [PATCH 057/113] Add test of SRE teardown cancellation --- tests/commands/test_sre.py | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/tests/commands/test_sre.py b/tests/commands/test_sre.py index 18d6c6daa1..4cee2b1cce 100644 --- a/tests/commands/test_sre.py +++ b/tests/commands/test_sre.py @@ -100,11 +100,9 @@ def test_no_shm( class TestTeardownSRE: def test_teardown( self, - mocker: MockerFixture, runner: CliRunner, mock_ip_1_2_3_4, # noqa: ARG002 mock_pulumi_config_from_remote, # noqa: ARG002 - mock_shm_config_from_remote, # noqa: ARG002 mock_sre_config_from_remote, # noqa: ARG002 mock_sre_project_manager_teardown_then_exit, # noqa: ARG002 ) -> None: @@ -139,3 +137,15 @@ def test_auth_failure( assert result.exit_code == 1 assert "mock get_credential\n" in result.stdout assert "mock get_credential error" in result.stdout + + def test_teardown_cancelled( + self, + runner: CliRunner, + mock_ip_1_2_3_4, # noqa: ARG002 + mock_pulumi_config_from_remote, # noqa: ARG002 + mock_sre_config_from_remote, # noqa: ARG002 + mock_sre_project_manager_teardown_then_exit, # noqa: ARG002 + ) -> None: + result = runner.invoke(sre_command_group, ["teardown", "sandbox"], input="n") + assert result.exit_code == 0 + assert "cancelled by user" in result.stdout \ No newline at end of file From 58c48807e4094509beec7831ffde97bca6d89fd3 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 13:21:48 +0000 Subject: [PATCH 058/113] remove unnecessary import and mocker --- tests/commands/test_shm.py | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/tests/commands/test_shm.py b/tests/commands/test_shm.py index 36b62c7524..e78609146a 100644 --- a/tests/commands/test_shm.py +++ b/tests/commands/test_shm.py @@ -1,5 +1,5 @@ from data_safe_haven.commands.shm import shm_command_group -from data_safe_haven.config import DSHPulumiConfig + class TestDeploySHM: def test_infrastructure_deploy( @@ -72,12 +72,11 @@ def test_auth_failure( def test_teardown_sres_exist( self, - mocker, runner, - mock_azuresdk_get_subscription_name, # noqa: ARG002 + mock_azuresdk_get_subscription_name, # noqa: ARG002 mock_pulumi_config_from_remote, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 ): result = runner.invoke(shm_command_group, ["teardown"], input="y") assert result.exit_code == 1 - assert "Found deployed SREs" in result.stdout \ No newline at end of file + assert "Found deployed SREs" in result.stdout From 597fa1d40cb34463157643863185bd5eb5e2603a Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 13:21:58 +0000 Subject: [PATCH 059/113] Fix linting --- tests/commands/test_sre.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/commands/test_sre.py b/tests/commands/test_sre.py index 4cee2b1cce..a13518a878 100644 --- a/tests/commands/test_sre.py +++ b/tests/commands/test_sre.py @@ -148,4 +148,4 @@ def test_teardown_cancelled( ) -> None: result = runner.invoke(sre_command_group, ["teardown", "sandbox"], input="n") assert result.exit_code == 0 - assert "cancelled by user" in result.stdout \ No newline at end of file + assert "cancelled by user" in result.stdout From bb89362a5e6ce42435069c605176b3c086e36478 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 13:22:29 +0000 Subject: [PATCH 060/113] Remove unnecessary mocker --- tests/commands/test_shm.py | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/commands/test_shm.py b/tests/commands/test_shm.py index e78609146a..38f53cdae5 100644 --- a/tests/commands/test_shm.py +++ b/tests/commands/test_shm.py @@ -40,7 +40,6 @@ def test_infrastructure_auth_failure( class TestTeardownSHM: def test_teardown( self, - mocker, runner, mock_imperative_shm_teardown_then_exit, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 From 2e9746ad01c15e37da9982b6bc07c755bc0df52c Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 13:27:03 +0000 Subject: [PATCH 061/113] Print confirmation of cancellation to user rather than log --- data_safe_haven/commands/shm.py | 2 +- data_safe_haven/commands/sre.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/commands/shm.py b/data_safe_haven/commands/shm.py index ff73884db9..3dc4e07c12 100644 --- a/data_safe_haven/commands/shm.py +++ b/data_safe_haven/commands/shm.py @@ -148,7 +148,7 @@ def teardown() -> None: if not console.confirm( "Do you wish to continue tearing down the SHM?", default_to_yes=True ): - logger.info("SHM teardown cancelled by user.") + console.print("SHM teardown cancelled by user.") raise typer.Exit(0) shm_infra.teardown() except DataSafeHavenError as exc: diff --git a/data_safe_haven/commands/sre.py b/data_safe_haven/commands/sre.py index 8e6b3036fd..3d3d29db15 100644 --- a/data_safe_haven/commands/sre.py +++ b/data_safe_haven/commands/sre.py @@ -199,7 +199,7 @@ def teardown( if not console.confirm( "Do you wish to continue tearing down the SRE?", default_to_yes=True ): - logger.info("SRE teardown cancelled by user.") + console.print("SRE teardown cancelled by user.") raise typer.Exit(0) # Check whether current IP address is authorised to take administrator actions From 44ee3f7a409ecb4b102d0ee7b48b3a40dac847e8 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 13:28:10 +0000 Subject: [PATCH 062/113] Default to false and stopping teardown --- data_safe_haven/commands/shm.py | 2 +- data_safe_haven/commands/sre.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/commands/shm.py b/data_safe_haven/commands/shm.py index 3dc4e07c12..87ffe210c7 100644 --- a/data_safe_haven/commands/shm.py +++ b/data_safe_haven/commands/shm.py @@ -146,7 +146,7 @@ def teardown() -> None: "Tearing down the Safe Haven Management environment will permanently delete all associated resources, including remotely stored configurations." ) if not console.confirm( - "Do you wish to continue tearing down the SHM?", default_to_yes=True + "Do you wish to continue tearing down the SHM?", default_to_yes=False ): console.print("SHM teardown cancelled by user.") raise typer.Exit(0) diff --git a/data_safe_haven/commands/sre.py b/data_safe_haven/commands/sre.py index 3d3d29db15..8c3e0b5cdc 100644 --- a/data_safe_haven/commands/sre.py +++ b/data_safe_haven/commands/sre.py @@ -197,7 +197,7 @@ def teardown( "Ensure that any desired outputs have been extracted before continuing." ) if not console.confirm( - "Do you wish to continue tearing down the SRE?", default_to_yes=True + "Do you wish to continue tearing down the SRE?", default_to_yes=False ): console.print("SRE teardown cancelled by user.") raise typer.Exit(0) From f65b9bcaf46ae673b2e4a92e42b1d91e8861198f Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 13:29:21 +0000 Subject: [PATCH 063/113] Add test for user SHM teardown cancellation --- tests/commands/test_shm.py | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/tests/commands/test_shm.py b/tests/commands/test_shm.py index 38f53cdae5..0ec3d9a55a 100644 --- a/tests/commands/test_shm.py +++ b/tests/commands/test_shm.py @@ -79,3 +79,14 @@ def test_teardown_sres_exist( result = runner.invoke(shm_command_group, ["teardown"], input="y") assert result.exit_code == 1 assert "Found deployed SREs" in result.stdout + + def test_teardown_user_cancelled( + self, + runner, + mock_azuresdk_get_subscription_name, # noqa: ARG002 + mock_pulumi_config_from_remote, # noqa: ARG002 + mock_shm_config_from_remote, # noqa: ARG002 + ): + result = runner.invoke(shm_command_group, ["teardown"], input="n") + assert result.exit_code == 0 + assert "cancelled" in result.stdout From 05b6962dd40e525fce487d223f580f3335f469b0 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 15:10:14 +0000 Subject: [PATCH 064/113] add remote_fails pulumi config fixture --- tests/commands/conftest.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tests/commands/conftest.py b/tests/commands/conftest.py index dc6811ff9f..15f47ba987 100644 --- a/tests/commands/conftest.py +++ b/tests/commands/conftest.py @@ -95,6 +95,15 @@ def mock_pulumi_config_from_remote(mocker, pulumi_config): mocker.patch.object(DSHPulumiConfig, "from_remote", return_value=pulumi_config) +@fixture +def mock_pulumi_config_from_remote_fails(mocker): + mocker.patch.object( + DSHPulumiConfig, + "from_remote", + return_value=DataSafeHavenAzureError("mock from_remote failure"), + ) + + @fixture def mock_pulumi_config_from_remote_or_create(mocker, pulumi_config_empty): mocker.patch.object( From 22d7f3bce5e4bf78e8be9227e2d5dcb5a82aa1ce Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 15:10:27 +0000 Subject: [PATCH 065/113] add test for no pulumi config --- tests/commands/test_shm.py | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/tests/commands/test_shm.py b/tests/commands/test_shm.py index 0ec3d9a55a..49d24f917f 100644 --- a/tests/commands/test_shm.py +++ b/tests/commands/test_shm.py @@ -90,3 +90,15 @@ def test_teardown_user_cancelled( result = runner.invoke(shm_command_group, ["teardown"], input="n") assert result.exit_code == 0 assert "cancelled" in result.stdout + + def test_teardown_no_pulumi_config( + self, + runner, + mock_azuresdk_get_subscription_name, # noqa: ARG002 + mock_pulumi_config_from_remote_fails, # noqa: ARG002 + mock_shm_config_from_remote, # noqa: ARG002 + mock_imperative_shm_teardown_then_exit, # noqa: ARG002 + ): + result = runner.invoke(shm_command_group, ["teardown"], input="y") + assert result.exit_code == 1 + assert "mock teardown" in result.stdout From db0448aa41291e0e3438cabb2bf83d85ef4361e7 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 29 Oct 2024 15:10:46 +0000 Subject: [PATCH 066/113] Catch errors when no pulumi config present --- data_safe_haven/infrastructure/programs/imperative_shm.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/infrastructure/programs/imperative_shm.py b/data_safe_haven/infrastructure/programs/imperative_shm.py index c4c0f5e761..023e015789 100644 --- a/data_safe_haven/infrastructure/programs/imperative_shm.py +++ b/data_safe_haven/infrastructure/programs/imperative_shm.py @@ -178,8 +178,12 @@ def teardown(self) -> None: DataSafeHavenAzureError if any resources cannot be destroyed """ logger = get_logger() - pulumi_config = DSHPulumiConfig.from_remote(self.context) - deployed = pulumi_config.project_names + try: + pulumi_config = DSHPulumiConfig.from_remote(self.context) + deployed = pulumi_config.project_names + except DataSafeHavenAzureError: + deployed = None + pass if deployed: logger.info(f"Found deployed SREs: {deployed}.") msg = "Deployed SREs must be torn down before the SHM can be torn down." From daa2015440d74fc41d9bcf1c4ded80d72b82bccd Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Wed, 30 Oct 2024 10:46:23 +0000 Subject: [PATCH 067/113] Catch errors when no SHM is deployed --- data_safe_haven/commands/shm.py | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/data_safe_haven/commands/shm.py b/data_safe_haven/commands/shm.py index 87ffe210c7..f8701f1ca5 100644 --- a/data_safe_haven/commands/shm.py +++ b/data_safe_haven/commands/shm.py @@ -140,17 +140,23 @@ def teardown() -> None: # Teardown Data Safe Haven SHM infrastructure. try: - config = SHMConfig.from_remote(context) - shm_infra = ImperativeSHM(context, config) - console.print( - "Tearing down the Safe Haven Management environment will permanently delete all associated resources, including remotely stored configurations." - ) - if not console.confirm( - "Do you wish to continue tearing down the SHM?", default_to_yes=False - ): - console.print("SHM teardown cancelled by user.") - raise typer.Exit(0) - shm_infra.teardown() + if SHMConfig.remote_exists(context): + config = SHMConfig.from_remote(context) + shm_infra = ImperativeSHM(context, config) + console.print( + "Tearing down the Safe Haven Management environment will permanently delete all associated resources, including remotely stored configurations." + ) + if not console.confirm( + "Do you wish to continue tearing down the SHM?", default_to_yes=False + ): + console.print("SHM teardown cancelled by user.") + raise typer.Exit(0) + shm_infra.teardown() + else: + logger.critical( + f"No deployed SHM found for context [green]{context.name}." + ) + raise typer.Exit(1) except DataSafeHavenError as exc: logger.critical("Could not teardown Safe Haven Management environment.") raise typer.Exit(1) from exc From c4d50b0ac683b38b4899602004f53c9704f5f8c4 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Wed, 30 Oct 2024 10:47:20 +0000 Subject: [PATCH 068/113] fix linting --- data_safe_haven/commands/shm.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/data_safe_haven/commands/shm.py b/data_safe_haven/commands/shm.py index f8701f1ca5..c25012d46d 100644 --- a/data_safe_haven/commands/shm.py +++ b/data_safe_haven/commands/shm.py @@ -153,9 +153,7 @@ def teardown() -> None: raise typer.Exit(0) shm_infra.teardown() else: - logger.critical( - f"No deployed SHM found for context [green]{context.name}." - ) + logger.critical(f"No deployed SHM found for context [green]{context.name}.") raise typer.Exit(1) except DataSafeHavenError as exc: logger.critical("Could not teardown Safe Haven Management environment.") From 1096cd5498d7344f33d5694aa1455b6ad08f569a Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Wed, 30 Oct 2024 10:47:28 +0000 Subject: [PATCH 069/113] fix tests --- tests/commands/test_shm.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tests/commands/test_shm.py b/tests/commands/test_shm.py index 49d24f917f..b1b8ec7ab0 100644 --- a/tests/commands/test_shm.py +++ b/tests/commands/test_shm.py @@ -43,6 +43,7 @@ def test_teardown( runner, mock_imperative_shm_teardown_then_exit, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 + mock_shm_config_remote_exists, # noqa: ARG002 ): result = runner.invoke(shm_command_group, ["teardown"], input="y") assert result.exit_code == 1 @@ -62,6 +63,7 @@ def test_auth_failure( self, runner, mock_azuresdk_get_credential_failure, # noqa: ARG002 + mock_shm_config_remote_exists, # noqa: ARG002 ): result = runner.invoke(shm_command_group, ["teardown"]) assert result.exit_code == 1 @@ -75,6 +77,7 @@ def test_teardown_sres_exist( mock_azuresdk_get_subscription_name, # noqa: ARG002 mock_pulumi_config_from_remote, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 + mock_shm_config_remote_exists, # noqa: ARG002 ): result = runner.invoke(shm_command_group, ["teardown"], input="y") assert result.exit_code == 1 @@ -86,6 +89,7 @@ def test_teardown_user_cancelled( mock_azuresdk_get_subscription_name, # noqa: ARG002 mock_pulumi_config_from_remote, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 + mock_shm_config_remote_exists, # noqa: ARG002 ): result = runner.invoke(shm_command_group, ["teardown"], input="n") assert result.exit_code == 0 @@ -98,6 +102,7 @@ def test_teardown_no_pulumi_config( mock_pulumi_config_from_remote_fails, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 mock_imperative_shm_teardown_then_exit, # noqa: ARG002 + mock_shm_config_remote_exists, # noqa: ARG002 ): result = runner.invoke(shm_command_group, ["teardown"], input="y") assert result.exit_code == 1 From 6111140a57954fcaefd7542ffda2d508e805bb9d Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Wed, 30 Oct 2024 12:00:38 +0000 Subject: [PATCH 070/113] Check if pulumi config exists rather than catch errors --- .../infrastructure/programs/imperative_shm.py | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/data_safe_haven/infrastructure/programs/imperative_shm.py b/data_safe_haven/infrastructure/programs/imperative_shm.py index 023e015789..f233c7c7fe 100644 --- a/data_safe_haven/infrastructure/programs/imperative_shm.py +++ b/data_safe_haven/infrastructure/programs/imperative_shm.py @@ -178,16 +178,13 @@ def teardown(self) -> None: DataSafeHavenAzureError if any resources cannot be destroyed """ logger = get_logger() - try: + if DSHPulumiConfig.remote_exists(self.context): pulumi_config = DSHPulumiConfig.from_remote(self.context) deployed = pulumi_config.project_names - except DataSafeHavenAzureError: - deployed = None - pass - if deployed: - logger.info(f"Found deployed SREs: {deployed}.") - msg = "Deployed SREs must be torn down before the SHM can be torn down." - raise DataSafeHavenAzureError(msg) + if deployed: + logger.info(f"Found deployed SREs: {deployed}.") + msg = "Deployed SREs must be torn down before the SHM can be torn down." + raise DataSafeHavenAzureError(msg) try: logger.info( f"Removing [green]{self.context.description}[/] resource group {self.context.resource_group_name}." From 29fa7f5fea9b9fbc1fdc7a7fa56d56db319f5228 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Wed, 30 Oct 2024 13:38:49 +0000 Subject: [PATCH 071/113] add pulumi config remote_exists fixture --- tests/commands/conftest.py | 3 +++ tests/commands/test_shm.py | 1 + 2 files changed, 4 insertions(+) diff --git a/tests/commands/conftest.py b/tests/commands/conftest.py index 15f47ba987..8630cf193f 100644 --- a/tests/commands/conftest.py +++ b/tests/commands/conftest.py @@ -122,6 +122,9 @@ def mock_pulumi_config_no_key_from_remote(mocker, pulumi_config_no_key): def mock_pulumi_config_upload(mocker): mocker.patch.object(DSHPulumiConfig, "upload", return_value=None) +@fixture +def mock_pulumi_config_remote_exists(mocker): + mocker.patch.object(DSHPulumiConfig, "remote_exists", return_value=True) @fixture def mock_shm_config_from_remote(mocker, shm_config): diff --git a/tests/commands/test_shm.py b/tests/commands/test_shm.py index b1b8ec7ab0..e8f3919ed9 100644 --- a/tests/commands/test_shm.py +++ b/tests/commands/test_shm.py @@ -76,6 +76,7 @@ def test_teardown_sres_exist( runner, mock_azuresdk_get_subscription_name, # noqa: ARG002 mock_pulumi_config_from_remote, # noqa: ARG002 + mock_pulumi_config_remote_exists, # noqa: ARG002 mock_shm_config_from_remote, # noqa: ARG002 mock_shm_config_remote_exists, # noqa: ARG002 ): From 938bc78c2cfa2cb5c59ddbed690c87b1e70173a5 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Wed, 30 Oct 2024 13:42:44 +0000 Subject: [PATCH 072/113] fix linting --- tests/commands/conftest.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/commands/conftest.py b/tests/commands/conftest.py index 8630cf193f..d675398bfc 100644 --- a/tests/commands/conftest.py +++ b/tests/commands/conftest.py @@ -122,10 +122,12 @@ def mock_pulumi_config_no_key_from_remote(mocker, pulumi_config_no_key): def mock_pulumi_config_upload(mocker): mocker.patch.object(DSHPulumiConfig, "upload", return_value=None) + @fixture def mock_pulumi_config_remote_exists(mocker): mocker.patch.object(DSHPulumiConfig, "remote_exists", return_value=True) + @fixture def mock_shm_config_from_remote(mocker, shm_config): mocker.patch.object(SHMConfig, "from_remote", return_value=shm_config) From 8687863fa0748ab562460e9c97baf52fb0ee1149 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Wed, 30 Oct 2024 15:09:17 +0000 Subject: [PATCH 073/113] convert copy and paste table to html --- docs/source/deployment/deploy_sre.md | 67 +++++++++++++++++++++++++--- 1 file changed, 60 insertions(+), 7 deletions(-) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index 20a41c52e8..94ad3f711e 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -187,14 +187,67 @@ The ability to copy and paste text to or from SRE workspaces via the Guacamole c These options have no impact on the ability to use copy and paste within a workspace. The impact of setting each of these options is detailed in the below table. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Configuration of copy and paste#
Configuration settingResulting behaviour
allow_copyallow_pasteCopy/paste within workspaceCopy/paste between workspacesCopy to local machinePaste from local machine
truetrueyesyesyesyes
truefalseyesnoyesno
falsetrueyesnonoyes
falsefalseyesnonono
+ -:::{table} **Configuration of copy and paste** -| allow_copy | allow_paste | Copy/paste within workspace | Copy/paste between workspaces | Copy to local machine | Paste from local machine | -|-----|-----|-----|-----|-----|-----| -| true | true | yes | yes | yes | yes | -| true | false | yes | no | yes | no | -| false | true | yes | no | no | yes| -| false | false | yes | no | no | no | ::: ## Upload the configuration file From a19cc61ce6009afde616287f2f56fb4e75e9f674 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Thu, 31 Oct 2024 10:40:47 +0000 Subject: [PATCH 074/113] Allow public network access for Internet tag --- .../wrapped/nfsv3_storage_account.py | 33 ++++++++++++------- .../infrastructure/programs/sre/data.py | 28 ++++++++++------ .../programs/sre/desired_state.py | 1 + 3 files changed, 41 insertions(+), 21 deletions(-) diff --git a/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py b/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py index 181839e71d..873a44c949 100644 --- a/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py +++ b/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py @@ -4,6 +4,7 @@ from pulumi_azure_native import storage from data_safe_haven.external import AzureIPv4Range +from data_safe_haven.types import AzureServiceTag class WrappedNFSV3StorageAccount(storage.StorageAccount): @@ -24,17 +25,35 @@ def __init__( resource_name: str, *, account_name: Input[str], - allowed_ip_addresses: Input[Sequence[str]], + allowed_ip_addresses: Input[Sequence[str]] | None, + allowed_service_tag: Input[Sequence[str]] | None, location: Input[str], resource_group_name: Input[str], subnet_id: Input[str], opts: ResourceOptions, tags: Input[Mapping[str, Input[str]]], ): + if allowed_service_tag == AzureServiceTag.INTERNET: + public_network_access = storage.PublicNetworkAccess.ENABLED + ip_rules = None + else: + public_network_access = storage.PublicNetworkAccess.DISABLED + ip_rules = Output.from_input(allowed_ip_addresses).apply( + lambda ip_ranges: [ + storage.IPRuleArgs( + action=storage.Action.ALLOW, + i_p_address_or_range=str(ip_address), + ) + for ip_range in sorted(ip_ranges) + for ip_address in AzureIPv4Range.from_cidr(ip_range).all_ips() + ] + ) + self.resource_group_name_ = Output.from_input(resource_group_name) super().__init__( resource_name, account_name=account_name, + allow_blob_public_access=False, enable_https_traffic_only=True, enable_nfs_v3=True, encryption=self.encryption_args, @@ -45,22 +64,14 @@ def __init__( network_rule_set=storage.NetworkRuleSetArgs( bypass=storage.Bypass.AZURE_SERVICES, default_action=storage.DefaultAction.DENY, - ip_rules=Output.from_input(allowed_ip_addresses).apply( - lambda ip_ranges: [ - storage.IPRuleArgs( - action=storage.Action.ALLOW, - i_p_address_or_range=str(ip_address), - ) - for ip_range in sorted(ip_ranges) - for ip_address in AzureIPv4Range.from_cidr(ip_range).all_ips() - ] - ), + ip_rules=ip_rules, virtual_network_rules=[ storage.VirtualNetworkRuleArgs( virtual_network_resource_id=subnet_id, ) ], ), + public_network_access=public_network_access, resource_group_name=resource_group_name, sku=storage.SkuArgs(name=storage.SkuName.PREMIUM_ZRS), opts=opts, diff --git a/data_safe_haven/infrastructure/programs/sre/data.py b/data_safe_haven/infrastructure/programs/sre/data.py index 9e18666277..711b76139f 100644 --- a/data_safe_haven/infrastructure/programs/sre/data.py +++ b/data_safe_haven/infrastructure/programs/sre/data.py @@ -35,7 +35,7 @@ SSLCertificateProps, WrappedNFSV3StorageAccount, ) -from data_safe_haven.types import AzureDnsZoneNames +from data_safe_haven.types import AzureDnsZoneNames, AzureServiceTag class SREDataProps: @@ -46,7 +46,7 @@ def __init__( admin_email_address: Input[str], admin_group_id: Input[str], admin_ip_addresses: Input[Sequence[str]], - data_provider_ip_addresses: Input[Sequence[str]], + data_provider_ip_addresses: Input[list[str]] | AzureServiceTag, dns_private_zones: Input[dict[str, network.PrivateZone]], dns_record: Input[network.RecordSet], dns_server_admin_password: Input[pulumi_random.RandomPassword], @@ -64,13 +64,7 @@ def __init__( self.admin_email_address = admin_email_address self.admin_group_id = admin_group_id self.data_configuration_ip_addresses = admin_ip_addresses - self.data_private_sensitive_ip_addresses = Output.all( - admin_ip_addresses, data_provider_ip_addresses - ).apply( - lambda address_lists: { - ip for address_list in address_lists for ip in address_list - } - ) + self.data_provider_ip_addresses = data_provider_ip_addresses self.dns_private_zones = dns_private_zones self.dns_record = dns_record self.password_dns_server_admin = dns_server_admin_password @@ -112,6 +106,19 @@ def __init__( child_opts = ResourceOptions.merge(opts, ResourceOptions(parent=self)) child_tags = {"component": "data"} | (tags if tags else {}) + if isinstance(props.data_provider_ip_addresses, list): + data_private_sensitive_service_tag = None + data_private_sensitive_ip_addresses = Output.all( + props.data_configuration_ip_addresses, props.data_provider_ip_addresses + ).apply( + lambda address_lists: { + ip for address_list in address_lists for ip in address_list + } + ) + else: + data_private_sensitive_ip_addresses = None + data_private_sensitive_service_tag = props.data_provider_ip_addresses + # Define Key Vault reader identity_key_vault_reader = managedidentity.UserAssignedIdentity( f"{self._name}_id_key_vault_reader", @@ -466,7 +473,8 @@ def __init__( account_name=alphanumeric( f"{''.join(truncate_tokens(stack_name.split('-'), 11))}sensitivedata{sha256hash(self._name)}" )[:24], - allowed_ip_addresses=props.data_private_sensitive_ip_addresses, + allowed_ip_addresses=data_private_sensitive_ip_addresses, + allowed_service_tag=data_private_sensitive_service_tag, location=props.location, subnet_id=props.subnet_data_private_id, resource_group_name=props.resource_group_name, diff --git a/data_safe_haven/infrastructure/programs/sre/desired_state.py b/data_safe_haven/infrastructure/programs/sre/desired_state.py index 73466d6c5b..c4392f5210 100644 --- a/data_safe_haven/infrastructure/programs/sre/desired_state.py +++ b/data_safe_haven/infrastructure/programs/sre/desired_state.py @@ -108,6 +108,7 @@ def __init__( f"{''.join(truncate_tokens(stack_name.split('-'), 11))}desiredstate{sha256hash(self._name)}" )[:24], allowed_ip_addresses=props.admin_ip_addresses, + allowed_service_tag=None, location=props.location, resource_group_name=props.resource_group_name, subnet_id=props.subnet_desired_state_id, From 60158d594df00344a58f07b8376787acfffec164 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Thu, 31 Oct 2024 12:03:22 +0000 Subject: [PATCH 075/113] Use default action --- .../components/wrapped/nfsv3_storage_account.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py b/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py index 873a44c949..61b0db634c 100644 --- a/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py +++ b/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py @@ -34,10 +34,10 @@ def __init__( tags: Input[Mapping[str, Input[str]]], ): if allowed_service_tag == AzureServiceTag.INTERNET: - public_network_access = storage.PublicNetworkAccess.ENABLED + default_action = storage.DefaultAction.ALLOW, ip_rules = None else: - public_network_access = storage.PublicNetworkAccess.DISABLED + default_action = storage.DefaultAction.DENY, ip_rules = Output.from_input(allowed_ip_addresses).apply( lambda ip_ranges: [ storage.IPRuleArgs( @@ -63,7 +63,7 @@ def __init__( minimum_tls_version=storage.MinimumTlsVersion.TLS1_2, network_rule_set=storage.NetworkRuleSetArgs( bypass=storage.Bypass.AZURE_SERVICES, - default_action=storage.DefaultAction.DENY, + default_action=default_action, ip_rules=ip_rules, virtual_network_rules=[ storage.VirtualNetworkRuleArgs( @@ -71,7 +71,7 @@ def __init__( ) ], ), - public_network_access=public_network_access, + public_network_access=storage.PublicNetworkAccess.ENABLED, resource_group_name=resource_group_name, sku=storage.SkuArgs(name=storage.SkuName.PREMIUM_ZRS), opts=opts, From 7db5ea776902fd7e144daf9da3b57fb76e0e78d4 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Thu, 31 Oct 2024 14:51:21 +0000 Subject: [PATCH 076/113] Tidy table style --- docs/source/deployment/deploy_sre.md | 52 +++++++++++----------------- 1 file changed, 21 insertions(+), 31 deletions(-) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index 94ad3f711e..fff23112ea 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -187,50 +187,43 @@ The ability to copy and paste text to or from SRE workspaces via the Guacamole c These options have no impact on the ability to use copy and paste within a workspace. The impact of setting each of these options is detailed in the below table. - - - - + +
Configuration of copy and paste#
+ - - - + + + - - - - - - - - - + + + + + + + - + + - - + - + - + - + @@ -238,17 +231,14 @@ The impact of setting each of these options is detailed in the below table. - + -
Configuration of copy and paste#
Configuration settingResulting behaviour
Configuration settingResulting behaviour
allow_copyallow_pasteCopy/paste within workspaceCopy/paste between workspacesCopy to local machinePaste from local machine
allow_copyallow_pasteCopy/paste within workspaceCopy/paste between workspacesCopy to local machinePaste from local machine
true truetrue yes yes yes yes
truefalsefalse yes no yes no
falsetruetrue yes no no
falsefalsefalse yes no no no
- - -::: + ## Upload the configuration file From 0912f7a57d584c39e00a2117526c873ea8a9c3d9 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Thu, 31 Oct 2024 14:52:58 +0000 Subject: [PATCH 077/113] Change grammar --- docs/source/deployment/deploy_sre.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index fff23112ea..184eae255c 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -186,7 +186,7 @@ The ability to copy and paste text to or from SRE workspaces via the Guacamole c `allow_paste` allows users to paste text into an SRE workspace from the Guacamole clipboard. These options have no impact on the ability to use copy and paste within a workspace. -The impact of setting each of these options is detailed in the below table. +The impact of setting each of these options is detailed in the following table. From 0243ae588181fa669f68d64f89c0a082fa177516 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Thu, 31 Oct 2024 14:54:31 +0000 Subject: [PATCH 078/113] Add note for inter-workspace copy/paste --- docs/source/deployment/deploy_sre.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index 184eae255c..5157de2267 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -209,7 +209,7 @@ The impact of setting each of these options is detailed in the following table. - + From 025155a3f4a82b966e759a673a13684001373576 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Thu, 31 Oct 2024 15:25:53 +0000 Subject: [PATCH 079/113] Correct syntax --- .../components/wrapped/nfsv3_storage_account.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py b/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py index 61b0db634c..e259de4806 100644 --- a/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py +++ b/data_safe_haven/infrastructure/components/wrapped/nfsv3_storage_account.py @@ -26,7 +26,7 @@ def __init__( *, account_name: Input[str], allowed_ip_addresses: Input[Sequence[str]] | None, - allowed_service_tag: Input[Sequence[str]] | None, + allowed_service_tag: AzureServiceTag | None, location: Input[str], resource_group_name: Input[str], subnet_id: Input[str], @@ -34,10 +34,10 @@ def __init__( tags: Input[Mapping[str, Input[str]]], ): if allowed_service_tag == AzureServiceTag.INTERNET: - default_action = storage.DefaultAction.ALLOW, - ip_rules = None + default_action = storage.DefaultAction.ALLOW + ip_rules = [] else: - default_action = storage.DefaultAction.DENY, + default_action = storage.DefaultAction.DENY ip_rules = Output.from_input(allowed_ip_addresses).apply( lambda ip_ranges: [ storage.IPRuleArgs( From 3986da3c866ffbd551be59550f956b4a4f327080 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Tue, 5 Nov 2024 16:10:57 +0000 Subject: [PATCH 080/113] Give VM system assigned managed identity --- .../infrastructure/components/composite/virtual_machine.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/data_safe_haven/infrastructure/components/composite/virtual_machine.py b/data_safe_haven/infrastructure/components/composite/virtual_machine.py index 71fed08246..9471991a4a 100644 --- a/data_safe_haven/infrastructure/components/composite/virtual_machine.py +++ b/data_safe_haven/infrastructure/components/composite/virtual_machine.py @@ -199,6 +199,9 @@ def __init__( ), ), vm_name=props.vm_name, + identity=compute.VirtualMachineIdentityArgs( + type=compute.ResourceIdentityType.SYSTEM_ASSIGNED, + ), opts=ResourceOptions.merge( child_opts, ResourceOptions( From 24c2c9069f17a729ab1d98dcdda60b2a24123696 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Fri, 8 Nov 2024 15:28:26 +0000 Subject: [PATCH 081/113] validate SRE name input for config show --- data_safe_haven/commands/config.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/commands/config.py b/data_safe_haven/commands/config.py index a774868516..2ebc01e3b8 100644 --- a/data_safe_haven/commands/config.py +++ b/data_safe_haven/commands/config.py @@ -6,7 +6,7 @@ import typer -from data_safe_haven import console +from data_safe_haven import console, validators from data_safe_haven.config import ( ContextManager, DSHPulumiConfig, @@ -107,7 +107,8 @@ def available() -> None: @config_command_group.command() def show( - name: Annotated[str, typer.Argument(help="Name of SRE to show")], + name: Annotated[str, typer.Argument(help="Name of SRE to show", + callback=validators.typer_safe_string)], file: Annotated[ Optional[Path], # noqa: UP007 typer.Option(help="File path to write configuration template to."), @@ -200,6 +201,7 @@ def upload( logger.error("Check for missing or incorrect fields in the configuration.") raise typer.Exit(1) from exc + print(config.filename) # Present diff to user if (not force) and SREConfig.remote_exists(context, filename=config.filename): try: From 1d7527269d17b065f1a4eb2b9c5a12324a80a494 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Fri, 8 Nov 2024 15:28:57 +0000 Subject: [PATCH 082/113] Remove json_safe function and use SafeString name in filename --- data_safe_haven/config/sre_config.py | 3 +-- data_safe_haven/functions/__init__.py | 2 -- data_safe_haven/functions/strings.py | 5 ----- 3 files changed, 1 insertion(+), 9 deletions(-) diff --git a/data_safe_haven/config/sre_config.py b/data_safe_haven/config/sre_config.py index 9fba89e12f..951b1079a8 100644 --- a/data_safe_haven/config/sre_config.py +++ b/data_safe_haven/config/sre_config.py @@ -4,7 +4,6 @@ from typing import ClassVar, Self -from data_safe_haven.functions import json_safe from data_safe_haven.serialisers import AzureSerialisableModel, ContextBase from data_safe_haven.types import SafeString, SoftwarePackageCategory @@ -19,7 +18,7 @@ def sre_config_name(sre_name: str) -> str: """Construct a safe YAML filename given an input SRE name.""" - return f"sre-{json_safe(sre_name)}.yaml" + return f"sre-{sre_name}.yaml" class SREConfig(AzureSerialisableModel): diff --git a/data_safe_haven/functions/__init__.py b/data_safe_haven/functions/__init__.py index e11b326135..4a83a76463 100644 --- a/data_safe_haven/functions/__init__.py +++ b/data_safe_haven/functions/__init__.py @@ -3,7 +3,6 @@ alphanumeric, b64encode, get_key_vault_name, - json_safe, next_occurrence, password, replace_separators, @@ -18,7 +17,6 @@ "current_ip_address", "get_key_vault_name", "ip_address_in_list", - "json_safe", "next_occurrence", "password", "replace_separators", diff --git a/data_safe_haven/functions/strings.py b/data_safe_haven/functions/strings.py index 0d5b06b33e..bf229c4f5e 100644 --- a/data_safe_haven/functions/strings.py +++ b/data_safe_haven/functions/strings.py @@ -27,11 +27,6 @@ def get_key_vault_name(stack_name: str) -> str: return f"{''.join(truncate_tokens(stack_name.split('-'), 17))}secrets" -def json_safe(input_string: str) -> str: - """Construct a JSON-safe version of an input string""" - return alphanumeric(input_string).lower() - - def next_occurrence( hour: int, minute: int, timezone: str, *, time_format: str = "iso" ) -> str: From 8b9145dbfec279bf4a09c595de1df7199d9d01ae Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Fri, 8 Nov 2024 15:29:56 +0000 Subject: [PATCH 083/113] Remove print statement --- data_safe_haven/commands/config.py | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/data_safe_haven/commands/config.py b/data_safe_haven/commands/config.py index 2ebc01e3b8..9f264d36c3 100644 --- a/data_safe_haven/commands/config.py +++ b/data_safe_haven/commands/config.py @@ -107,8 +107,12 @@ def available() -> None: @config_command_group.command() def show( - name: Annotated[str, typer.Argument(help="Name of SRE to show", - callback=validators.typer_safe_string)], + name: Annotated[ + str, + typer.Argument( + help="Name of SRE to show", callback=validators.typer_safe_string + ), + ], file: Annotated[ Optional[Path], # noqa: UP007 typer.Option(help="File path to write configuration template to."), @@ -201,7 +205,6 @@ def upload( logger.error("Check for missing or incorrect fields in the configuration.") raise typer.Exit(1) from exc - print(config.filename) # Present diff to user if (not force) and SREConfig.remote_exists(context, filename=config.filename): try: From 4d2231315b5e8030bcbf44442e01dca4cbc21390 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Fri, 8 Nov 2024 15:32:21 +0000 Subject: [PATCH 084/113] Remove json_safe test --- tests/functions/test_strings.py | 9 --------- 1 file changed, 9 deletions(-) diff --git a/tests/functions/test_strings.py b/tests/functions/test_strings.py index 3e57965d98..7bd12d9490 100644 --- a/tests/functions/test_strings.py +++ b/tests/functions/test_strings.py @@ -4,7 +4,6 @@ from data_safe_haven.exceptions import DataSafeHavenValueError from data_safe_haven.functions import ( get_key_vault_name, - json_safe, next_occurrence, ) @@ -70,11 +69,3 @@ def test_invalid_timeformat(self): ) def test_get_key_vault_name(value, expected): assert get_key_vault_name(value) == expected - - -@pytest.mark.parametrize( - "value,expected", - [(r"Test SRE", "testsre"), (r"%*aBc", "abc"), (r"MY_SRE", "mysre")], -) -def test_json_safe(value, expected): - assert json_safe(value) == expected From b85681a892475618b32f94c7c0fd002dd5764b22 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 11 Nov 2024 03:33:26 +0000 Subject: [PATCH 085/113] Bump lycheeverse/lychee-action from 2.0.2 to 2.1.0 Bumps [lycheeverse/lychee-action](https://github.com/lycheeverse/lychee-action) from 2.0.2 to 2.1.0. - [Release notes](https://github.com/lycheeverse/lychee-action/releases) - [Commits](https://github.com/lycheeverse/lychee-action/compare/v2.0.2...v2.1.0) --- updated-dependencies: - dependency-name: lycheeverse/lychee-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] --- .github/workflows/build_documentation.yaml | 2 +- .github/workflows/test_code.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/build_documentation.yaml b/.github/workflows/build_documentation.yaml index 5350075540..4ed3b51860 100644 --- a/.github/workflows/build_documentation.yaml +++ b/.github/workflows/build_documentation.yaml @@ -46,7 +46,7 @@ jobs: run: hatch run docs:build - name: Link Checker - uses: lycheeverse/lychee-action@v2.0.2 + uses: lycheeverse/lychee-action@v2.1.0 with: args: --config='./.lychee.toml' --no-progress './docs/build/html/**/*.html' fail: true # fail on broken links diff --git a/.github/workflows/test_code.yaml b/.github/workflows/test_code.yaml index 0119816136..996c06ead4 100644 --- a/.github/workflows/test_code.yaml +++ b/.github/workflows/test_code.yaml @@ -55,7 +55,7 @@ jobs: shell: bash run: npm install -g markdown-link-check - name: Link Checker - uses: lycheeverse/lychee-action@v2.0.2 + uses: lycheeverse/lychee-action@v2.1.0 with: args: --config='./.lychee.toml' --no-progress --offline '**/*.md' --exclude-path './docs' fail: true # fail on broken links From 73523200db59a742dec5ba091856354ec791bb18 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 11 Nov 2024 03:45:12 +0000 Subject: [PATCH 086/113] Bump the production-dependencies group across 1 directory with 9 updates Bumps the production-dependencies group with 9 updates in the / directory: | Package | From | To | | --- | --- | --- | | [azure-core](https://github.com/Azure/azure-sdk-for-python) | `1.31.0` | `1.32.0` | | [pulumi-azure-native](https://github.com/pulumi/pulumi-azure-native) | `2.68.0` | `2.71.0` | | [pulumi](https://github.com/pulumi/pulumi) | `3.137.0` | `3.138.0` | | [rich](https://github.com/Textualize/rich) | `13.9.3` | `13.9.4` | | [simple-acme-dns](https://github.com/jaredhendrickson13/simple_acme_dns) | `3.1.0` | `3.2.0` | | [typer](https://github.com/fastapi/typer) | `0.12.5` | `0.13.0` | | [ansible-dev-tools](https://github.com/ansible/ansible-dev-tools) | `24.10.0` | `24.10.2` | | [ansible](https://github.com/ansible-community/ansible-build-data) | `10.5.0` | `10.6.0` | | [ruff](https://github.com/astral-sh/ruff) | `0.7.1` | `0.7.3` | Updates `azure-core` from 1.31.0 to 1.32.0 - [Release notes](https://github.com/Azure/azure-sdk-for-python/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md) - [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-core_1.31.0...azure-core_1.32.0) Updates `pulumi-azure-native` from 2.68.0 to 2.71.0 - [Release notes](https://github.com/pulumi/pulumi-azure-native/releases) - [Changelog](https://github.com/pulumi/pulumi-azure-native/blob/master/CHANGELOG_OLD.md) - [Commits](https://github.com/pulumi/pulumi-azure-native/compare/v2.68.0...v2.71.0) Updates `pulumi` from 3.137.0 to 3.138.0 - [Release notes](https://github.com/pulumi/pulumi/releases) - [Changelog](https://github.com/pulumi/pulumi/blob/master/CHANGELOG.md) - [Commits](https://github.com/pulumi/pulumi/compare/v3.137.0...v3.138.0) Updates `rich` from 13.9.3 to 13.9.4 - [Release notes](https://github.com/Textualize/rich/releases) - [Changelog](https://github.com/Textualize/rich/blob/master/CHANGELOG.md) - [Commits](https://github.com/Textualize/rich/compare/v13.9.3...v13.9.4) Updates `simple-acme-dns` from 3.1.0 to 3.2.0 - [Release notes](https://github.com/jaredhendrickson13/simple_acme_dns/releases) - [Commits](https://github.com/jaredhendrickson13/simple_acme_dns/compare/v3.1.0...v3.2.0) Updates `typer` from 0.12.5 to 0.13.0 - [Release notes](https://github.com/fastapi/typer/releases) - [Changelog](https://github.com/fastapi/typer/blob/master/docs/release-notes.md) - [Commits](https://github.com/fastapi/typer/compare/0.12.5...0.13.0) Updates `ansible-dev-tools` from 24.10.0 to 24.10.2 - [Release notes](https://github.com/ansible/ansible-dev-tools/releases) - [Commits](https://github.com/ansible/ansible-dev-tools/compare/v24.10.0...v24.10.2) Updates `ansible` from 10.5.0 to 10.6.0 - [Changelog](https://github.com/ansible-community/ansible-build-data/blob/main/docs/release-process.md) - [Commits](https://github.com/ansible-community/ansible-build-data/compare/10.5.0...10.6.0) Updates `ruff` from 0.7.1 to 0.7.3 - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](https://github.com/astral-sh/ruff/compare/0.7.1...0.7.3) --- updated-dependencies: - dependency-name: azure-core dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: pulumi-azure-native dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: pulumi dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: rich dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: simple-acme-dns dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: typer dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: ansible-dev-tools dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: ansible dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: ruff dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] --- pyproject.toml | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index 8bcd84a1ca..e3cb46525e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -25,7 +25,7 @@ classifiers = [ license = { text = "BSD-3-Clause" } dependencies = [ "appdirs==1.4.4", - "azure-core==1.31.0", + "azure-core==1.32.0", "azure-identity==1.19.0", "azure-keyvault-certificates==4.9.0", "azure-keyvault-keys==4.10.0", @@ -45,17 +45,17 @@ dependencies = [ "cryptography==43.0.3", "fqdn==1.5.1", "psycopg[binary]==3.1.19", # needed for installation on older MacOS versions - "pulumi-azure-native==2.68.0", + "pulumi-azure-native==2.71.0", "pulumi-azuread==6.0.1", "pulumi-random==4.16.7", - "pulumi==3.137.0", + "pulumi==3.138.0", "pydantic==2.9.2", "pyjwt[crypto]==2.9.0", "pytz==2024.2", "pyyaml==6.0.2", - "rich==13.9.3", - "simple-acme-dns==3.1.0", - "typer==0.12.5", + "rich==13.9.4", + "simple-acme-dns==3.2.0", + "typer==0.13.0", "websocket-client==1.8.0", ] @@ -73,13 +73,13 @@ docs = [ "sphinx==8.1.3", ] lint = [ - "ansible-dev-tools==24.10.0", - "ansible==10.5.0", + "ansible-dev-tools==24.10.2", + "ansible==10.6.0", "black==24.10.0", "mypy==1.13.0", "pandas-stubs==2.2.3.241009", "pydantic==2.9.2", - "ruff==0.7.1", + "ruff==0.7.3", "types-appdirs==1.4.3.5", "types-chevron==0.14.2.20240310", "types-pytz==2024.2.0.20241003", From d739b5c35541a7da55260fea8b2febd7ef99056b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" Date: Mon, 11 Nov 2024 03:52:06 +0000 Subject: [PATCH 087/113] [dependabot skip] :wrench: Update Python requirements files --- .hatch/requirements-docs.txt | 4 ++-- .hatch/requirements-lint.txt | 28 +++++++++++++------------- .hatch/requirements-test.txt | 39 ++++++++++++++++++------------------ .hatch/requirements.txt | 35 ++++++++++++++++---------------- 4 files changed, 52 insertions(+), 54 deletions(-) diff --git a/.hatch/requirements-docs.txt b/.hatch/requirements-docs.txt index d93aeb3bbf..95fcbdfd41 100644 --- a/.hatch/requirements-docs.txt +++ b/.hatch/requirements-docs.txt @@ -50,7 +50,7 @@ mdurl==0.1.2 # via markdown-it-py myst-parser==4.0.0 # via hatch.envs.docs -packaging==24.1 +packaging==24.2 # via sphinx pydata-sphinx-theme==0.16.0 # via hatch.envs.docs @@ -91,7 +91,7 @@ typing-extensions==4.12.2 # via pydata-sphinx-theme urllib3==2.2.3 # via requests -wheel==0.44.0 +wheel==0.45.0 # via sphinx-togglebutton # The following packages are considered to be unsafe in a requirements file: diff --git a/.hatch/requirements-lint.txt b/.hatch/requirements-lint.txt index 4bd0234a71..2ffa3e2ba4 100644 --- a/.hatch/requirements-lint.txt +++ b/.hatch/requirements-lint.txt @@ -1,13 +1,13 @@ # # This file is autogenerated by hatch-pip-compile with Python 3.12 # -# - ansible-dev-tools==24.10.0 -# - ansible==10.5.0 +# - ansible-dev-tools==24.10.2 +# - ansible==10.6.0 # - black==24.10.0 # - mypy==1.13.0 # - pandas-stubs==2.2.3.241009 # - pydantic==2.9.2 -# - ruff==0.7.1 +# - ruff==0.7.3 # - types-appdirs==1.4.3.5 # - types-chevron==0.14.2.20240310 # - types-pytz==2024.2.0.20241003 @@ -17,7 +17,7 @@ annotated-types==0.7.0 # via pydantic -ansible==10.5.0 +ansible==10.6.0 # via hatch.envs.lint ansible-builder==3.1.0 # via @@ -29,18 +29,18 @@ ansible-compat==24.9.1 # ansible-lint # molecule # pytest-ansible -ansible-core==2.17.5 +ansible-core==2.17.6 # via # ansible # ansible-compat # ansible-lint # molecule # pytest-ansible -ansible-creator==24.9.0 +ansible-creator==24.10.1 # via ansible-dev-tools ansible-dev-environment==24.9.0 # via ansible-dev-tools -ansible-dev-tools==24.10.0 +ansible-dev-tools==24.10.2 # via hatch.envs.lint ansible-lint==24.9.2 # via @@ -133,11 +133,11 @@ mypy-extensions==1.0.0 # via # black # mypy -numpy==2.1.2 +numpy==2.1.3 # via pandas-stubs onigurumacffi==1.3.0 # via ansible-navigator -packaging==24.1 +packaging==24.2 # via # ansible-builder # ansible-compat @@ -220,12 +220,12 @@ referencing==0.35.1 # jsonschema-specifications resolvelib==1.0.1 # via ansible-core -rich==13.9.3 +rich==13.9.4 # via # ansible-lint # enrich # molecule -rpds-py==0.20.0 +rpds-py==0.21.0 # via # jsonschema # referencing @@ -233,7 +233,7 @@ ruamel-yaml==0.18.6 # via ansible-lint ruamel-yaml-clib==0.2.12 # via ruamel-yaml -ruff==0.7.1 +ruff==0.7.3 # via hatch.envs.lint subprocess-tee==0.4.2 # via @@ -265,7 +265,7 @@ tzdata==2024.2 # via ansible-navigator urllib3==2.2.3 # via types-requests -virtualenv==20.27.0 +virtualenv==20.27.1 # via tox wcmatch==10.0 # via @@ -273,7 +273,7 @@ wcmatch==10.0 # molecule yamllint==1.35.1 # via ansible-lint -zipp==3.20.2 +zipp==3.21.0 # via importlib-metadata # The following packages are considered to be unsafe in a requirements file: diff --git a/.hatch/requirements-test.txt b/.hatch/requirements-test.txt index f224befaa0..3b6dd5421f 100644 --- a/.hatch/requirements-test.txt +++ b/.hatch/requirements-test.txt @@ -1,10 +1,10 @@ # # This file is autogenerated by hatch-pip-compile with Python 3.12 # -# [constraints] .hatch/requirements.txt (SHA256: 7ad151976b72bb564d491ec5c0ead3b24dcd4d259b99700df8e92cc8a0cd07ed) +# [constraints] .hatch/requirements.txt (SHA256: ac6baaee77c4015ca7749690da5f658092b9d5572c7e4083c08b8c4bf75c5c6c) # # - appdirs==1.4.4 -# - azure-core==1.31.0 +# - azure-core==1.32.0 # - azure-identity==1.19.0 # - azure-keyvault-certificates==4.9.0 # - azure-keyvault-keys==4.10.0 @@ -24,17 +24,17 @@ # - cryptography==43.0.3 # - fqdn==1.5.1 # - psycopg[binary]==3.1.19 -# - pulumi-azure-native==2.68.0 +# - pulumi-azure-native==2.71.0 # - pulumi-azuread==6.0.1 # - pulumi-random==4.16.7 -# - pulumi==3.137.0 +# - pulumi==3.138.0 # - pydantic==2.9.2 # - pyjwt[crypto]==2.9.0 # - pytz==2024.2 # - pyyaml==6.0.2 -# - rich==13.9.3 -# - simple-acme-dns==3.1.0 -# - typer==0.12.5 +# - rich==13.9.4 +# - simple-acme-dns==3.2.0 +# - typer==0.13.0 # - websocket-client==1.8.0 # - coverage==7.6.4 # - freezegun==1.5.1 @@ -74,7 +74,7 @@ azure-common==1.1.28 # azure-mgmt-rdbms # azure-mgmt-resource # azure-mgmt-storage -azure-core==1.31.0 +azure-core==1.32.0 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -111,7 +111,7 @@ azure-mgmt-containerinstance==10.1.0 # via # -c .hatch/requirements.txt # hatch.envs.test -azure-mgmt-core==1.4.0 +azure-mgmt-core==1.5.0 # via # -c .hatch/requirements.txt # azure-mgmt-compute @@ -195,7 +195,7 @@ cryptography==43.0.3 # msal # pyjwt # pyopenssl -debugpy==1.8.7 +debugpy==1.8.8 # via # -c .hatch/requirements.txt # pulumi @@ -203,7 +203,7 @@ dill==0.3.9 # via # -c .hatch/requirements.txt # pulumi -dnspython==2.6.1 +dnspython==2.7.0 # via # -c .hatch/requirements.txt # simple-acme-dns @@ -269,7 +269,7 @@ oauthlib==3.2.2 # via # -c .hatch/requirements.txt # requests-oauthlib -packaging==24.1 +packaging==24.2 # via pytest parver==0.5 # via @@ -295,14 +295,14 @@ psycopg-binary==3.1.19 # via # -c .hatch/requirements.txt # psycopg -pulumi==3.137.0 +pulumi==3.138.0 # via # -c .hatch/requirements.txt # hatch.envs.test # pulumi-azure-native # pulumi-azuread # pulumi-random -pulumi-azure-native==2.68.0 +pulumi-azure-native==2.71.0 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -340,7 +340,7 @@ pyopenssl==24.2.1 # -c .hatch/requirements.txt # acme # josepy -pyrfc3339==1.1 +pyrfc3339==2.0.1 # via # -c .hatch/requirements.txt # acme @@ -357,7 +357,6 @@ pytz==2024.2 # -c .hatch/requirements.txt # hatch.envs.test # acme - # pyrfc3339 pyyaml==6.0.2 # via # -c .hatch/requirements.txt @@ -378,7 +377,7 @@ requests-oauthlib==2.0.0 # via # -c .hatch/requirements.txt # msrest -rich==13.9.3 +rich==13.9.4 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -394,7 +393,7 @@ shellingham==1.5.4 # via # -c .hatch/requirements.txt # typer -simple-acme-dns==3.1.0 +simple-acme-dns==3.2.0 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -404,7 +403,7 @@ six==1.16.0 # azure-core # pulumi # python-dateutil -typer==0.12.5 +typer==0.13.0 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -431,7 +430,7 @@ urllib3==2.2.3 # via # -c .hatch/requirements.txt # requests -validators==0.28.3 +validators==0.34.0 # via # -c .hatch/requirements.txt # simple-acme-dns diff --git a/.hatch/requirements.txt b/.hatch/requirements.txt index e9e357bfc0..5451c40b45 100644 --- a/.hatch/requirements.txt +++ b/.hatch/requirements.txt @@ -2,7 +2,7 @@ # This file is autogenerated by hatch-pip-compile with Python 3.12 # # - appdirs==1.4.4 -# - azure-core==1.31.0 +# - azure-core==1.32.0 # - azure-identity==1.19.0 # - azure-keyvault-certificates==4.9.0 # - azure-keyvault-keys==4.10.0 @@ -22,17 +22,17 @@ # - cryptography==43.0.3 # - fqdn==1.5.1 # - psycopg[binary]==3.1.19 -# - pulumi-azure-native==2.68.0 +# - pulumi-azure-native==2.71.0 # - pulumi-azuread==6.0.1 # - pulumi-random==4.16.7 -# - pulumi==3.137.0 +# - pulumi==3.138.0 # - pydantic==2.9.2 # - pyjwt[crypto]==2.9.0 # - pytz==2024.2 # - pyyaml==6.0.2 -# - rich==13.9.3 -# - simple-acme-dns==3.1.0 -# - typer==0.12.5 +# - rich==13.9.4 +# - simple-acme-dns==3.2.0 +# - typer==0.13.0 # - websocket-client==1.8.0 # @@ -56,7 +56,7 @@ azure-common==1.1.28 # azure-mgmt-rdbms # azure-mgmt-resource # azure-mgmt-storage -azure-core==1.31.0 +azure-core==1.32.0 # via # hatch.envs.default # azure-identity @@ -80,7 +80,7 @@ azure-mgmt-compute==33.0.0 # via hatch.envs.default azure-mgmt-containerinstance==10.1.0 # via hatch.envs.default -azure-mgmt-core==1.4.0 +azure-mgmt-core==1.5.0 # via # azure-mgmt-compute # azure-mgmt-containerinstance @@ -134,11 +134,11 @@ cryptography==43.0.3 # msal # pyjwt # pyopenssl -debugpy==1.8.7 +debugpy==1.8.8 # via pulumi dill==0.3.9 # via pulumi -dnspython==2.6.1 +dnspython==2.7.0 # via simple-acme-dns fqdn==1.5.1 # via hatch.envs.default @@ -192,13 +192,13 @@ psycopg==3.1.19 # via hatch.envs.default psycopg-binary==3.1.19 # via psycopg -pulumi==3.137.0 +pulumi==3.138.0 # via # hatch.envs.default # pulumi-azure-native # pulumi-azuread # pulumi-random -pulumi-azure-native==2.68.0 +pulumi-azure-native==2.71.0 # via hatch.envs.default pulumi-azuread==6.0.1 # via hatch.envs.default @@ -220,13 +220,12 @@ pyopenssl==24.2.1 # via # acme # josepy -pyrfc3339==1.1 +pyrfc3339==2.0.1 # via acme pytz==2024.2 # via # hatch.envs.default # acme - # pyrfc3339 pyyaml==6.0.2 # via # hatch.envs.default @@ -240,7 +239,7 @@ requests==2.32.3 # requests-oauthlib requests-oauthlib==2.0.0 # via msrest -rich==13.9.3 +rich==13.9.4 # via # hatch.envs.default # typer @@ -252,13 +251,13 @@ semver==2.13.0 # pulumi-random shellingham==1.5.4 # via typer -simple-acme-dns==3.1.0 +simple-acme-dns==3.2.0 # via hatch.envs.default six==1.16.0 # via # azure-core # pulumi -typer==0.12.5 +typer==0.13.0 # via hatch.envs.default typing-extensions==4.12.2 # via @@ -280,7 +279,7 @@ typing-extensions==4.12.2 # typer urllib3==2.2.3 # via requests -validators==0.28.3 +validators==0.34.0 # via simple-acme-dns websocket-client==1.8.0 # via hatch.envs.default From 99fa613ee94cd453bacb3d5f7e868ebabffa5a48 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 11 Nov 2024 13:31:00 +0000 Subject: [PATCH 088/113] modify test for creation of sre yaml filename --- tests/config/test_sre_config.py | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/tests/config/test_sre_config.py b/tests/config/test_sre_config.py index 66bd50a40d..7ac6d61981 100644 --- a/tests/config/test_sre_config.py +++ b/tests/config/test_sre_config.py @@ -7,7 +7,6 @@ ConfigSectionDockerHub, ConfigSectionSRE, ) -from data_safe_haven.config.sre_config import sre_config_name from data_safe_haven.exceptions import ( DataSafeHavenTypeError, ) @@ -126,14 +125,5 @@ def test_upload(self, mocker, context, sre_config) -> None: context.storage_container_name, ) - -@pytest.mark.parametrize( - "value,expected", - [ - (r"Test SRE", "sre-testsre.yaml"), - (r"*a^b$c", "sre-abc.yaml"), - (r";'@-", "sre-.yaml"), - ], -) -def test_sre_config_name(value, expected): - assert sre_config_name(value) == expected + def test_sre_config_yaml_name(self, sre_config: SREConfig) -> None: + assert sre_config.filename == "sre-sandbox.yaml" From ff1cd334a93f46997284a21d6d20c9b52ce2efe0 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 11 Nov 2024 13:31:29 +0000 Subject: [PATCH 089/113] Use lowercase config filename --- data_safe_haven/config/sre_config.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/config/sre_config.py b/data_safe_haven/config/sre_config.py index 951b1079a8..cc17501f12 100644 --- a/data_safe_haven/config/sre_config.py +++ b/data_safe_haven/config/sre_config.py @@ -17,8 +17,8 @@ def sre_config_name(sre_name: str) -> str: - """Construct a safe YAML filename given an input SRE name.""" - return f"sre-{sre_name}.yaml" + """Construct a YAML filename given an input SRE name.""" + return f"sre-{sre_name.lower()}.yaml" class SREConfig(AzureSerialisableModel): From 4367c5ae36f832972fada7a541c0cec01737b7a9 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 11 Nov 2024 15:46:08 +0000 Subject: [PATCH 090/113] Add safe_sre_name validator --- data_safe_haven/validators/validators.py | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/data_safe_haven/validators/validators.py b/data_safe_haven/validators/validators.py index 27507d26b4..01936058a6 100644 --- a/data_safe_haven/validators/validators.py +++ b/data_safe_haven/validators/validators.py @@ -135,6 +135,13 @@ def safe_string(safe_string: str) -> str: return safe_string +def safe_sre_name(safe_sre_name: str) -> str: + if not re.match(r"^[a-zA-Z0-9_-]*$", safe_sre_name) or not safe_sre_name: + msg = "Expected valid string containing only lowercase letters, numbers, hyphens and underscores." + raise ValueError(msg) + return safe_sre_name + + def timezone(timezone: str) -> str: if timezone not in pytz.all_timezones: msg = "Expected valid timezone, for example 'Europe/London'." From 69d6f4521e977e46b7bbb95875093000712aafc8 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 11 Nov 2024 15:46:41 +0000 Subject: [PATCH 091/113] add typer_validator for safe_sre_name --- data_safe_haven/validators/__init__.py | 4 ++++ data_safe_haven/validators/typer.py | 1 + 2 files changed, 5 insertions(+) diff --git a/data_safe_haven/validators/__init__.py b/data_safe_haven/validators/__init__.py index 849b199857..5e08eed24d 100644 --- a/data_safe_haven/validators/__init__.py +++ b/data_safe_haven/validators/__init__.py @@ -1,4 +1,5 @@ from .typer import ( + type_safe_sre_name, typer_aad_guid, typer_azure_subscription_name, typer_azure_vm_sku, @@ -6,6 +7,7 @@ typer_entra_group_name, typer_fqdn, typer_ip_address, + typer_safe_sre_name, typer_safe_string, typer_timezone, ) @@ -18,6 +20,7 @@ entra_group_name, fqdn, ip_address, + safe_sre_name, safe_string, timezone, unique_list, @@ -32,6 +35,7 @@ "entra_group_name", "fqdn", "ip_address", + "safe_sre_name", "safe_string", "timezone", "typer_aad_guid", diff --git a/data_safe_haven/validators/typer.py b/data_safe_haven/validators/typer.py index f1c8239ecc..fd50774290 100644 --- a/data_safe_haven/validators/typer.py +++ b/data_safe_haven/validators/typer.py @@ -33,5 +33,6 @@ def typer_validator(x: Any) -> Any: typer_entra_group_name = typer_validator_factory(validators.entra_group_name) typer_fqdn = typer_validator_factory(validators.fqdn) typer_ip_address = typer_validator_factory(validators.ip_address) +typer_safe_sre_name = typer_validator_factory(validators.safe_sre_name) typer_safe_string = typer_validator_factory(validators.safe_string) typer_timezone = typer_validator_factory(validators.timezone) From 3e44018cf638ad481265bb9c0226d8affcb873cb Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 11 Nov 2024 15:47:24 +0000 Subject: [PATCH 092/113] Add SafeSreName type --- data_safe_haven/types/__init__.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/data_safe_haven/types/__init__.py b/data_safe_haven/types/__init__.py index e12c76bdc6..728df06c19 100644 --- a/data_safe_haven/types/__init__.py +++ b/data_safe_haven/types/__init__.py @@ -8,6 +8,7 @@ Fqdn, Guid, IpAddress, + SafeSreName, SafeString, TimeZone, UniqueList, @@ -52,6 +53,7 @@ "PathType", "PermittedDomains", "Ports", + "SafeSreName", "SafeString", "SoftwarePackageCategory", "TimeZone", From 30b76ada9b0b7ea5c727ba6a20d17e8edcc3db11 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 11 Nov 2024 16:04:06 +0000 Subject: [PATCH 093/113] Use new validator for SRE name --- data_safe_haven/config/sre_config.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/data_safe_haven/config/sre_config.py b/data_safe_haven/config/sre_config.py index cc17501f12..53adb673e0 100644 --- a/data_safe_haven/config/sre_config.py +++ b/data_safe_haven/config/sre_config.py @@ -5,7 +5,7 @@ from typing import ClassVar, Self from data_safe_haven.serialisers import AzureSerialisableModel, ContextBase -from data_safe_haven.types import SafeString, SoftwarePackageCategory +from data_safe_haven.types import SafeSreName, SoftwarePackageCategory from .config_sections import ( ConfigSectionAzure, @@ -18,7 +18,7 @@ def sre_config_name(sre_name: str) -> str: """Construct a YAML filename given an input SRE name.""" - return f"sre-{sre_name.lower()}.yaml" + return f"sre-{sre_name}.yaml" class SREConfig(AzureSerialisableModel): @@ -30,7 +30,7 @@ class SREConfig(AzureSerialisableModel): azure: ConfigSectionAzure description: str dockerhub: ConfigSectionDockerHub - name: SafeString + name: SafeSreName sre: ConfigSectionSRE @property From 42c9f4c181948181e12abb1e8cc3f3d05c973930 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 11 Nov 2024 16:04:32 +0000 Subject: [PATCH 094/113] Catch uppercase letters in SRE name --- data_safe_haven/validators/__init__.py | 2 +- data_safe_haven/validators/validators.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/validators/__init__.py b/data_safe_haven/validators/__init__.py index 5e08eed24d..30316e0834 100644 --- a/data_safe_haven/validators/__init__.py +++ b/data_safe_haven/validators/__init__.py @@ -1,5 +1,4 @@ from .typer import ( - type_safe_sre_name, typer_aad_guid, typer_azure_subscription_name, typer_azure_vm_sku, @@ -45,6 +44,7 @@ "typer_entra_group_name", "typer_fqdn", "typer_ip_address", + "typer_safe_sre_name", "typer_safe_string", "typer_timezone", "unique_list", diff --git a/data_safe_haven/validators/validators.py b/data_safe_haven/validators/validators.py index 01936058a6..9ab7fef587 100644 --- a/data_safe_haven/validators/validators.py +++ b/data_safe_haven/validators/validators.py @@ -136,7 +136,7 @@ def safe_string(safe_string: str) -> str: def safe_sre_name(safe_sre_name: str) -> str: - if not re.match(r"^[a-zA-Z0-9_-]*$", safe_sre_name) or not safe_sre_name: + if not re.match(r"^[a-z0-9_-]*$", safe_sre_name) or not safe_sre_name: msg = "Expected valid string containing only lowercase letters, numbers, hyphens and underscores." raise ValueError(msg) return safe_sre_name From e34ac85487fb14d29693d85ffa282aa22485f198 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 11 Nov 2024 16:04:46 +0000 Subject: [PATCH 095/113] Add new annotated type for SRE name --- data_safe_haven/types/annotated_types.py | 1 + 1 file changed, 1 insertion(+) diff --git a/data_safe_haven/types/annotated_types.py b/data_safe_haven/types/annotated_types.py index 639bf03129..d6258b0e7a 100644 --- a/data_safe_haven/types/annotated_types.py +++ b/data_safe_haven/types/annotated_types.py @@ -21,6 +21,7 @@ Fqdn = Annotated[str, AfterValidator(validators.fqdn)] Guid = Annotated[str, AfterValidator(validators.aad_guid)] IpAddress = Annotated[str, AfterValidator(validators.ip_address)] +SafeSreName = Annotated[str, AfterValidator(validators.safe_sre_name)] SafeString = Annotated[str, AfterValidator(validators.safe_string)] TimeZone = Annotated[str, AfterValidator(validators.timezone)] TH = TypeVar("TH", bound=Hashable) From 000e4f9e640abe4357fa161a205aeaef9065dde5 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 11 Nov 2024 16:05:10 +0000 Subject: [PATCH 096/113] Remove validator for SRE config show --- data_safe_haven/commands/config.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data_safe_haven/commands/config.py b/data_safe_haven/commands/config.py index 9f264d36c3..471209f0c9 100644 --- a/data_safe_haven/commands/config.py +++ b/data_safe_haven/commands/config.py @@ -110,7 +110,7 @@ def show( name: Annotated[ str, typer.Argument( - help="Name of SRE to show", callback=validators.typer_safe_string + help="Name of SRE to show" ), ], file: Annotated[ From 7c099d50580c33035cabd9203de960afd70d1148 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 11 Nov 2024 16:14:24 +0000 Subject: [PATCH 097/113] update tests to handle new SRE name style --- tests/commands/test_config_sre.py | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/commands/test_config_sre.py b/tests/commands/test_config_sre.py index 7460a908eb..c4391e1c70 100644 --- a/tests/commands/test_config_sre.py +++ b/tests/commands/test_config_sre.py @@ -167,7 +167,7 @@ class TestUploadSRE: def test_upload_new( self, mocker, context, runner, sre_config_yaml, sre_config_file ): - sre_name = "SandBox" + sre_name = "sandbox" sre_filename = sre_config_name(sre_name) mock_exists = mocker.patch.object( SREConfig, "remote_exists", return_value=False @@ -191,7 +191,7 @@ def test_upload_new( def test_upload_no_changes( self, mocker, context, runner, sre_config, sre_config_file ): - sre_name = "SandBox" + sre_name = "sandbox" sre_filename = sre_config_name(sre_name) mock_exists = mocker.patch.object(SREConfig, "remote_exists", return_value=True) mock_from_remote = mocker.patch.object( @@ -247,9 +247,9 @@ def test_upload_changes( assert "+++ local" in result.stdout def test_upload_changes_n( - self, mocker, context, runner, sre_config_alternate, sre_config_file + self, mocker, context, runner, sre_config, sre_config_alternate, sre_config_file ): - sre_name = "SandBox" + sre_name = "sandbox" sre_filename = sre_config_name(sre_name) mock_exists = mocker.patch.object(SREConfig, "remote_exists", return_value=True) mock_from_remote = mocker.patch.object( @@ -285,9 +285,9 @@ def test_upload_file_does_not_exist(self, mocker, runner): assert "Configuration file 'fake_config.yaml' not found." in result.stdout def test_upload_invalid_config( - self, mocker, runner, context, sre_config_file, sre_config_yaml + self, mocker, runner, context, sre_config, sre_config_file, sre_config_yaml ): - sre_name = "SandBox" + sre_name = "sandbox" sre_filename = sre_config_name(sre_name) mock_exists = mocker.patch.object(SREConfig, "remote_exists", return_value=True) @@ -310,7 +310,7 @@ def test_upload_invalid_config( def test_upload_invalid_config_force( self, mocker, runner, context, sre_config_file, sre_config_yaml ): - sre_name = "SandBox" + sre_name = "sandbox" sre_filename = sre_config_name(sre_name) mocker.patch.object( From 506ebc3c0a800ba10781e01ab54ff149a0a6190e Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Mon, 11 Nov 2024 16:17:13 +0000 Subject: [PATCH 098/113] fix linting --- data_safe_haven/commands/config.py | 6 ++---- tests/commands/test_config_sre.py | 4 ++-- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/data_safe_haven/commands/config.py b/data_safe_haven/commands/config.py index 471209f0c9..4d096f45ac 100644 --- a/data_safe_haven/commands/config.py +++ b/data_safe_haven/commands/config.py @@ -6,7 +6,7 @@ import typer -from data_safe_haven import console, validators +from data_safe_haven import console from data_safe_haven.config import ( ContextManager, DSHPulumiConfig, @@ -109,9 +109,7 @@ def available() -> None: def show( name: Annotated[ str, - typer.Argument( - help="Name of SRE to show" - ), + typer.Argument(help="Name of SRE to show"), ], file: Annotated[ Optional[Path], # noqa: UP007 diff --git a/tests/commands/test_config_sre.py b/tests/commands/test_config_sre.py index c4391e1c70..263def9236 100644 --- a/tests/commands/test_config_sre.py +++ b/tests/commands/test_config_sre.py @@ -247,7 +247,7 @@ def test_upload_changes( assert "+++ local" in result.stdout def test_upload_changes_n( - self, mocker, context, runner, sre_config, sre_config_alternate, sre_config_file + self, mocker, context, runner, sre_config_alternate, sre_config_file ): sre_name = "sandbox" sre_filename = sre_config_name(sre_name) @@ -285,7 +285,7 @@ def test_upload_file_does_not_exist(self, mocker, runner): assert "Configuration file 'fake_config.yaml' not found." in result.stdout def test_upload_invalid_config( - self, mocker, runner, context, sre_config, sre_config_file, sre_config_yaml + self, mocker, runner, context, sre_config_file, sre_config_yaml ): sre_name = "sandbox" sre_filename = sre_config_name(sre_name) From 35455af73e54f512c8324ab09b28fca6eea5a867 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Tue, 12 Nov 2024 10:45:52 +0000 Subject: [PATCH 099/113] Switch regexes from any number to one or more --- data_safe_haven/validators/validators.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/data_safe_haven/validators/validators.py b/data_safe_haven/validators/validators.py index 9ab7fef587..a7befd5ece 100644 --- a/data_safe_haven/validators/validators.py +++ b/data_safe_haven/validators/validators.py @@ -129,14 +129,14 @@ def ip_address(ip_address: str) -> str: def safe_string(safe_string: str) -> str: - if not re.match(r"^[a-zA-Z0-9_-]*$", safe_string) or not safe_string: + if not re.match(r"^[a-zA-Z0-9_-]+$", safe_string) or not safe_string: msg = "Expected valid string containing only letters, numbers, hyphens and underscores." raise ValueError(msg) return safe_string def safe_sre_name(safe_sre_name: str) -> str: - if not re.match(r"^[a-z0-9_-]*$", safe_sre_name) or not safe_sre_name: + if not re.match(r"^[a-z0-9_-]+$", safe_sre_name) or not safe_sre_name: msg = "Expected valid string containing only lowercase letters, numbers, hyphens and underscores." raise ValueError(msg) return safe_sre_name From d74c0d0b82edc8881df79caa169dd65d8239e1bd Mon Sep 17 00:00:00 2001 From: Carlos Gavidia-Calderon Date: Wed, 13 Nov 2024 08:28:34 +0000 Subject: [PATCH 100/113] Changing suggested SKU to Standard_D8s_v5 --- docs/source/deployment/deploy_sre.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/deployment/deploy_sre.md b/docs/source/deployment/deploy_sre.md index 5157de2267..5a5a5b4166 100644 --- a/docs/source/deployment/deploy_sre.md +++ b/docs/source/deployment/deploy_sre.md @@ -167,7 +167,7 @@ As some general recommendations, - For general purpose use, the D family gives decent performance and a good balance of CPU and memory. The [Dsv6 series](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/general-purpose/dsv6-series#sizes-in-series) is a good starting point and can be scaled from 2 CPUs and 8 GB RAM to 128 CPUs and 512 GB RAM. - - `Standard_D8s_v6` should give reasonable performance for a single concurrent user. + - `Standard_D8s_v5` should give reasonable performance for a single concurrent user. - For GPU accelerated work, the NC family provides Nvidia GPUs and a good balance of CPU and memory. In order of increasing throughput, the `NCv3` series features Nvidia V100 GPUs, the `NC_A100_v4` series features Nvidia A100 GPUs, and the `NCads_H100_v5` series features Nvidia H100 GPUs. - `Stanard_NC6s_v3` should give reasonable performance for a single concurrent user with AI/ML workloads. From 5deaee7ee1edd8c51543230c5ffdf9ac4fcf491a Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Wed, 13 Nov 2024 15:36:00 +0000 Subject: [PATCH 101/113] Add section on update SRE configurations --- docs/source/management/index.md | 44 +++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/docs/source/management/index.md b/docs/source/management/index.md index e9f49a5733..8663adc7d1 100644 --- a/docs/source/management/index.md +++ b/docs/source/management/index.md @@ -141,6 +141,50 @@ Tearing down the SHM also renders the SREs inaccessible to users and prevents th All SREs associated with the SHM should be torn down before the SHM is torn down. :::: +### Updating SRE configurations + +Changes to SRE configurations are made by first editing the relevant configuration file for the SRE, uploading the new configuration, and then redeploying the SRE. + +- The existing configuration for the SRE can be shown using the following: + +```{code} shell +$ dsh config show YOUR_SRE_NAME +``` + +- If you do not have a local copy of the configuration for the SRE, it can be downloaded by adding the `file` argument: + +```{code} shell +$ dsh config show YOUR_SRE_NAME --file YOUR_SRE_NAME.yaml +``` + +- Edit the configuration file locally, and upload the new version: + +```{code} shell +$ dsh config upload YOUR_SRE_NAME.yaml +``` + +- You will be shown the differences between the existing configuration and the new configuration and asked to confirm that they are correct. + +- Finally, redeploy your SRE for the infrastructure to reflect your new changes + +```{code} shell +$ dsh sre deploy YOUR_SRE_NAME +``` + +::::{admonition} Changing allowed administrator IP addresses +:class: warning +If you are changing the IP addresses from which administrators are allowed to make changes to the infrastructure, you **must** redeploy the SRE from the **original** IP address. +You will not be able to complete redeployment from the new IP address, because you will not be able to modify the IP addresses that are allowed to access the storage accounts. + +- Add the new IP address to the configuration, but do not delete the original IP address. +- Upload the new configuration and redeploy the SRE from the original IP address, as above + +If you then wish to remove the original IP address from the list of allowed IP addresses, then + +- Remove the old IP address from the configuration +- Upload the new configuration and redeploy the SRE from the new IP address +:::: + ## Managing data ingress and egress ### Data Ingress From a30ca29fd0580d6731db6194b64197b175fc3205 Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Wed, 13 Nov 2024 15:41:07 +0000 Subject: [PATCH 102/113] Add blank line after list --- docs/source/management/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/source/management/index.md b/docs/source/management/index.md index 8663adc7d1..1c83d79f58 100644 --- a/docs/source/management/index.md +++ b/docs/source/management/index.md @@ -183,6 +183,7 @@ If you then wish to remove the original IP address from the list of allowed IP a - Remove the old IP address from the configuration - Upload the new configuration and redeploy the SRE from the new IP address + :::: ## Managing data ingress and egress From d471c44b06c8e209c8e1f5cb8d6850883079bb14 Mon Sep 17 00:00:00 2001 From: Jim Madge Date: Fri, 15 Nov 2024 09:17:01 +0000 Subject: [PATCH 103/113] docs: update @cptanalatriste as a contributor --- .all-contributorsrc | 3 ++- README.md | 2 +- docs/source/contributing/index.md | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/.all-contributorsrc b/.all-contributorsrc index a36e18670c..576f5b41c8 100644 --- a/.all-contributorsrc +++ b/.all-contributorsrc @@ -580,7 +580,8 @@ "profile": "https://carlos.gavidia.me/", "contributions": [ "bug", - "ideas" + "ideas", + "doc" ] }, { diff --git a/README.md b/README.md index 79659201f0..95bdbac05e 100644 --- a/README.md +++ b/README.md @@ -50,7 +50,7 @@ See our [Code of Conduct](CODE_OF_CONDUCT.md) and our [Contributor Guide](CONTRI - + diff --git a/docs/source/contributing/index.md b/docs/source/contributing/index.md index e5f99fd3a0..20c14073bf 100644 --- a/docs/source/contributing/index.md +++ b/docs/source/contributing/index.md @@ -13,7 +13,7 @@ - + From 8f27d7b2a8f065dacf12af8a4bce329640c3ce52 Mon Sep 17 00:00:00 2001 From: Matt Craddock Date: Fri, 15 Nov 2024 16:20:05 +0000 Subject: [PATCH 104/113] Update docs/source/management/index.md Co-authored-by: Jim Madge --- docs/source/management/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/management/index.md b/docs/source/management/index.md index 1c83d79f58..5260880923 100644 --- a/docs/source/management/index.md +++ b/docs/source/management/index.md @@ -141,7 +141,7 @@ Tearing down the SHM also renders the SREs inaccessible to users and prevents th All SREs associated with the SHM should be torn down before the SHM is torn down. :::: -### Updating SRE configurations +### Updating SREs Changes to SRE configurations are made by first editing the relevant configuration file for the SRE, uploading the new configuration, and then redeploying the SRE. From e45accedd4a5502333573f3299d53f87b4d4e6bb Mon Sep 17 00:00:00 2001 From: Matt Craddock Date: Fri, 15 Nov 2024 16:20:45 +0000 Subject: [PATCH 105/113] Update docs/source/management/index.md Co-authored-by: Jim Madge --- docs/source/management/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/management/index.md b/docs/source/management/index.md index 5260880923..ebeb6d019d 100644 --- a/docs/source/management/index.md +++ b/docs/source/management/index.md @@ -151,7 +151,7 @@ Changes to SRE configurations are made by first editing the relevant configurati $ dsh config show YOUR_SRE_NAME ``` -- If you do not have a local copy of the configuration for the SRE, it can be downloaded by adding the `file` argument: +- If you do not have a local copy, you can write one with the `--file` option: ```{code} shell $ dsh config show YOUR_SRE_NAME --file YOUR_SRE_NAME.yaml From 5cb2cf3fd8fc41edfd7a7d0a116ac0d5c9d9de8c Mon Sep 17 00:00:00 2001 From: Matt Craddock Date: Fri, 15 Nov 2024 16:20:52 +0000 Subject: [PATCH 106/113] Update docs/source/management/index.md Co-authored-by: Jim Madge --- docs/source/management/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/management/index.md b/docs/source/management/index.md index ebeb6d019d..b25dab1276 100644 --- a/docs/source/management/index.md +++ b/docs/source/management/index.md @@ -171,7 +171,7 @@ $ dsh config upload YOUR_SRE_NAME.yaml $ dsh sre deploy YOUR_SRE_NAME ``` -::::{admonition} Changing allowed administrator IP addresses +::::{admonition} Changing administrator IP addresses :class: warning If you are changing the IP addresses from which administrators are allowed to make changes to the infrastructure, you **must** redeploy the SRE from the **original** IP address. You will not be able to complete redeployment from the new IP address, because you will not be able to modify the IP addresses that are allowed to access the storage accounts. From fac6675c375c4c1fee0359d2f14c596830cae3c3 Mon Sep 17 00:00:00 2001 From: Matt Craddock Date: Fri, 15 Nov 2024 16:21:51 +0000 Subject: [PATCH 107/113] Update docs/source/management/index.md Co-authored-by: Jim Madge --- docs/source/management/index.md | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/docs/source/management/index.md b/docs/source/management/index.md index b25dab1276..5f271190c2 100644 --- a/docs/source/management/index.md +++ b/docs/source/management/index.md @@ -173,17 +173,10 @@ $ dsh sre deploy YOUR_SRE_NAME ::::{admonition} Changing administrator IP addresses :class: warning -If you are changing the IP addresses from which administrators are allowed to make changes to the infrastructure, you **must** redeploy the SRE from the **original** IP address. -You will not be able to complete redeployment from the new IP address, because you will not be able to modify the IP addresses that are allowed to access the storage accounts. - -- Add the new IP address to the configuration, but do not delete the original IP address. -- Upload the new configuration and redeploy the SRE from the original IP address, as above - -If you then wish to remove the original IP address from the list of allowed IP addresses, then - -- Remove the old IP address from the configuration -- Upload the new configuration and redeploy the SRE from the new IP address +The administrator IP addresses declared in the SRE configuration are used to create access rules for SRE infrastructure. +Therefore, after an SRE has been deployed, some changes can only be made from IP addresses on that list. +As a consequence, if you want to update the list of administrator IP addresses, for example to add a new administrator, you must do so from an IP address that is already allowed. :::: ## Managing data ingress and egress From 8cf11cb845205296c9a3202f2608fd259a0b7931 Mon Sep 17 00:00:00 2001 From: Matt Craddock Date: Fri, 15 Nov 2024 16:22:03 +0000 Subject: [PATCH 108/113] Update docs/source/management/index.md Co-authored-by: Jim Madge --- docs/source/management/index.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/source/management/index.md b/docs/source/management/index.md index 5f271190c2..2b9f65be7f 100644 --- a/docs/source/management/index.md +++ b/docs/source/management/index.md @@ -164,8 +164,7 @@ $ dsh config upload YOUR_SRE_NAME.yaml ``` - You will be shown the differences between the existing configuration and the new configuration and asked to confirm that they are correct. - -- Finally, redeploy your SRE for the infrastructure to reflect your new changes +- Finally, deploy your SRE to apply any changes: ```{code} shell $ dsh sre deploy YOUR_SRE_NAME From 3142c1ec4b86a07f7d099ee43cb64f8b4ee85dbd Mon Sep 17 00:00:00 2001 From: Matt Craddock Date: Fri, 15 Nov 2024 16:23:30 +0000 Subject: [PATCH 109/113] Update docs/source/management/index.md Co-authored-by: Jim Madge --- docs/source/management/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/management/index.md b/docs/source/management/index.md index 2b9f65be7f..234fe8136a 100644 --- a/docs/source/management/index.md +++ b/docs/source/management/index.md @@ -143,7 +143,7 @@ All SREs associated with the SHM should be torn down before the SHM is torn down ### Updating SREs -Changes to SRE configurations are made by first editing the relevant configuration file for the SRE, uploading the new configuration, and then redeploying the SRE. +SREs are modified by updating the configuration then running the deploy command. - The existing configuration for the SRE can be shown using the following: From 629c6baec092adba11d25169bb4684cbd26ba85e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 18 Nov 2024 03:55:22 +0000 Subject: [PATCH 110/113] Bump the production-dependencies group with 8 updates Bumps the production-dependencies group with 8 updates: | Package | From | To | | --- | --- | --- | | [azure-storage-blob](https://github.com/Azure/azure-sdk-for-python) | `12.23.1` | `12.24.0` | | [azure-storage-file-datalake](https://github.com/Azure/azure-sdk-for-python) | `12.17.0` | `12.18.0` | | [azure-storage-file-share](https://github.com/Azure/azure-sdk-for-python) | `12.19.0` | `12.20.0` | | [pulumi-azure-native](https://github.com/pulumi/pulumi-azure-native) | `2.71.0` | `2.72.0` | | [pulumi](https://github.com/pulumi/pulumi) | `3.138.0` | `3.139.0` | | [pyjwt[crypto]](https://github.com/jpadilla/pyjwt) | `2.9.0` | `2.10.0` | | [ruff](https://github.com/astral-sh/ruff) | `0.7.3` | `0.7.4` | | [coverage](https://github.com/nedbat/coveragepy) | `7.6.4` | `7.6.7` | Updates `azure-storage-blob` from 12.23.1 to 12.24.0 - [Release notes](https://github.com/Azure/azure-sdk-for-python/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md) - [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-storage-blob_12.23.1...azure-storage-blob_12.24.0) Updates `azure-storage-file-datalake` from 12.17.0 to 12.18.0 - [Release notes](https://github.com/Azure/azure-sdk-for-python/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md) - [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-storage-file-datalake_12.17.0...azure-storage-file-datalake_12.18.0) Updates `azure-storage-file-share` from 12.19.0 to 12.20.0 - [Release notes](https://github.com/Azure/azure-sdk-for-python/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-python/blob/main/doc/esrp_release.md) - [Commits](https://github.com/Azure/azure-sdk-for-python/compare/azure-storage-file-share_12.19.0...azure-storage-file-share_12.20.0) Updates `pulumi-azure-native` from 2.71.0 to 2.72.0 - [Release notes](https://github.com/pulumi/pulumi-azure-native/releases) - [Changelog](https://github.com/pulumi/pulumi-azure-native/blob/master/CHANGELOG_OLD.md) - [Commits](https://github.com/pulumi/pulumi-azure-native/compare/v2.71.0...v2.72.0) Updates `pulumi` from 3.138.0 to 3.139.0 - [Release notes](https://github.com/pulumi/pulumi/releases) - [Changelog](https://github.com/pulumi/pulumi/blob/master/CHANGELOG.md) - [Commits](https://github.com/pulumi/pulumi/compare/v3.138.0...v3.139.0) Updates `pyjwt[crypto]` from 2.9.0 to 2.10.0 - [Release notes](https://github.com/jpadilla/pyjwt/releases) - [Changelog](https://github.com/jpadilla/pyjwt/blob/master/CHANGELOG.rst) - [Commits](https://github.com/jpadilla/pyjwt/compare/2.9.0...2.10.0) Updates `ruff` from 0.7.3 to 0.7.4 - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](https://github.com/astral-sh/ruff/compare/0.7.3...0.7.4) Updates `coverage` from 7.6.4 to 7.6.7 - [Release notes](https://github.com/nedbat/coveragepy/releases) - [Changelog](https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst) - [Commits](https://github.com/nedbat/coveragepy/compare/7.6.4...7.6.7) --- updated-dependencies: - dependency-name: azure-storage-blob dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: azure-storage-file-datalake dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: azure-storage-file-share dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: pulumi-azure-native dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: pulumi dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: pyjwt[crypto] dependency-type: direct:production update-type: version-update:semver-minor dependency-group: production-dependencies - dependency-name: ruff dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies - dependency-name: coverage dependency-type: direct:production update-type: version-update:semver-patch dependency-group: production-dependencies ... Signed-off-by: dependabot[bot] --- pyproject.toml | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index e3cb46525e..9304f1bd32 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -38,19 +38,19 @@ dependencies = [ "azure-mgmt-rdbms==10.1.0", "azure-mgmt-resource==23.2.0", "azure-mgmt-storage==21.2.1", - "azure-storage-blob==12.23.1", - "azure-storage-file-datalake==12.17.0", - "azure-storage-file-share==12.19.0", + "azure-storage-blob==12.24.0", + "azure-storage-file-datalake==12.18.0", + "azure-storage-file-share==12.20.0", "chevron==0.14.0", "cryptography==43.0.3", "fqdn==1.5.1", "psycopg[binary]==3.1.19", # needed for installation on older MacOS versions - "pulumi-azure-native==2.71.0", + "pulumi-azure-native==2.72.0", "pulumi-azuread==6.0.1", "pulumi-random==4.16.7", - "pulumi==3.138.0", + "pulumi==3.139.0", "pydantic==2.9.2", - "pyjwt[crypto]==2.9.0", + "pyjwt[crypto]==2.10.0", "pytz==2024.2", "pyyaml==6.0.2", "rich==13.9.4", @@ -79,7 +79,7 @@ lint = [ "mypy==1.13.0", "pandas-stubs==2.2.3.241009", "pydantic==2.9.2", - "ruff==0.7.3", + "ruff==0.7.4", "types-appdirs==1.4.3.5", "types-chevron==0.14.2.20240310", "types-pytz==2024.2.0.20241003", @@ -87,7 +87,7 @@ lint = [ "types-requests==2.32.0.20241016", ] test = [ - "coverage==7.6.4", + "coverage==7.6.7", "freezegun==1.5.1", "pytest-mock==3.14.0", "pytest==8.3.3", From 246ff427191c4475b9c898929b0a1d4657fae1f4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" Date: Mon, 18 Nov 2024 04:02:13 +0000 Subject: [PATCH 111/113] [dependabot skip] :wrench: Update Python requirements files --- .hatch/requirements-lint.txt | 14 +++++++------- .hatch/requirements-test.txt | 30 +++++++++++++++--------------- .hatch/requirements.txt | 24 ++++++++++++------------ 3 files changed, 34 insertions(+), 34 deletions(-) diff --git a/.hatch/requirements-lint.txt b/.hatch/requirements-lint.txt index 2ffa3e2ba4..295c694a65 100644 --- a/.hatch/requirements-lint.txt +++ b/.hatch/requirements-lint.txt @@ -7,7 +7,7 @@ # - mypy==1.13.0 # - pandas-stubs==2.2.3.241009 # - pydantic==2.9.2 -# - ruff==0.7.3 +# - ruff==0.7.4 # - types-appdirs==1.4.3.5 # - types-chevron==0.14.2.20240310 # - types-pytz==2024.2.0.20241003 @@ -24,7 +24,7 @@ ansible-builder==3.1.0 # ansible-dev-environment # ansible-dev-tools # ansible-navigator -ansible-compat==24.9.1 +ansible-compat==24.10.0 # via # ansible-lint # molecule @@ -36,17 +36,17 @@ ansible-core==2.17.6 # ansible-lint # molecule # pytest-ansible -ansible-creator==24.10.1 +ansible-creator==24.11.0 # via ansible-dev-tools ansible-dev-environment==24.9.0 # via ansible-dev-tools ansible-dev-tools==24.10.2 # via hatch.envs.lint -ansible-lint==24.9.2 +ansible-lint==24.10.0 # via # ansible-dev-tools # ansible-navigator -ansible-navigator==24.9.0 +ansible-navigator==24.10.0 # via ansible-dev-tools ansible-runner==2.4.0 # via ansible-navigator @@ -233,7 +233,7 @@ ruamel-yaml==0.18.6 # via ansible-lint ruamel-yaml-clib==0.2.12 # via ruamel-yaml -ruff==0.7.3 +ruff==0.7.4 # via hatch.envs.lint subprocess-tee==0.4.2 # via @@ -242,7 +242,7 @@ subprocess-tee==0.4.2 # ansible-lint tox==4.23.2 # via tox-ansible -tox-ansible==24.9.0 +tox-ansible==24.10.0 # via ansible-dev-tools types-appdirs==1.4.3.5 # via hatch.envs.lint diff --git a/.hatch/requirements-test.txt b/.hatch/requirements-test.txt index 3b6dd5421f..643331837b 100644 --- a/.hatch/requirements-test.txt +++ b/.hatch/requirements-test.txt @@ -1,7 +1,7 @@ # # This file is autogenerated by hatch-pip-compile with Python 3.12 # -# [constraints] .hatch/requirements.txt (SHA256: ac6baaee77c4015ca7749690da5f658092b9d5572c7e4083c08b8c4bf75c5c6c) +# [constraints] .hatch/requirements.txt (SHA256: ca6dfe8295dd8d2e6e4ade0fce58d158854ce5df89be8d092b36c34fe2679f3f) # # - appdirs==1.4.4 # - azure-core==1.32.0 @@ -17,26 +17,26 @@ # - azure-mgmt-rdbms==10.1.0 # - azure-mgmt-resource==23.2.0 # - azure-mgmt-storage==21.2.1 -# - azure-storage-blob==12.23.1 -# - azure-storage-file-datalake==12.17.0 -# - azure-storage-file-share==12.19.0 +# - azure-storage-blob==12.24.0 +# - azure-storage-file-datalake==12.18.0 +# - azure-storage-file-share==12.20.0 # - chevron==0.14.0 # - cryptography==43.0.3 # - fqdn==1.5.1 # - psycopg[binary]==3.1.19 -# - pulumi-azure-native==2.71.0 +# - pulumi-azure-native==2.72.0 # - pulumi-azuread==6.0.1 # - pulumi-random==4.16.7 -# - pulumi==3.138.0 +# - pulumi==3.139.0 # - pydantic==2.9.2 -# - pyjwt[crypto]==2.9.0 +# - pyjwt[crypto]==2.10.0 # - pytz==2024.2 # - pyyaml==6.0.2 # - rich==13.9.4 # - simple-acme-dns==3.2.0 # - typer==0.13.0 # - websocket-client==1.8.0 -# - coverage==7.6.4 +# - coverage==7.6.7 # - freezegun==1.5.1 # - pytest-mock==3.14.0 # - pytest==8.3.3 @@ -146,16 +146,16 @@ azure-mgmt-storage==21.2.1 # via # -c .hatch/requirements.txt # hatch.envs.test -azure-storage-blob==12.23.1 +azure-storage-blob==12.24.0 # via # -c .hatch/requirements.txt # hatch.envs.test # azure-storage-file-datalake -azure-storage-file-datalake==12.17.0 +azure-storage-file-datalake==12.18.0 # via # -c .hatch/requirements.txt # hatch.envs.test -azure-storage-file-share==12.19.0 +azure-storage-file-share==12.20.0 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -180,7 +180,7 @@ click==8.1.7 # via # -c .hatch/requirements.txt # typer -coverage==7.6.4 +coverage==7.6.7 # via hatch.envs.test cryptography==43.0.3 # via @@ -295,14 +295,14 @@ psycopg-binary==3.1.19 # via # -c .hatch/requirements.txt # psycopg -pulumi==3.138.0 +pulumi==3.139.0 # via # -c .hatch/requirements.txt # hatch.envs.test # pulumi-azure-native # pulumi-azuread # pulumi-random -pulumi-azure-native==2.71.0 +pulumi-azure-native==2.72.0 # via # -c .hatch/requirements.txt # hatch.envs.test @@ -330,7 +330,7 @@ pygments==2.18.0 # via # -c .hatch/requirements.txt # rich -pyjwt==2.9.0 +pyjwt==2.10.0 # via # -c .hatch/requirements.txt # hatch.envs.test diff --git a/.hatch/requirements.txt b/.hatch/requirements.txt index 5451c40b45..b0f7aff926 100644 --- a/.hatch/requirements.txt +++ b/.hatch/requirements.txt @@ -15,19 +15,19 @@ # - azure-mgmt-rdbms==10.1.0 # - azure-mgmt-resource==23.2.0 # - azure-mgmt-storage==21.2.1 -# - azure-storage-blob==12.23.1 -# - azure-storage-file-datalake==12.17.0 -# - azure-storage-file-share==12.19.0 +# - azure-storage-blob==12.24.0 +# - azure-storage-file-datalake==12.18.0 +# - azure-storage-file-share==12.20.0 # - chevron==0.14.0 # - cryptography==43.0.3 # - fqdn==1.5.1 # - psycopg[binary]==3.1.19 -# - pulumi-azure-native==2.71.0 +# - pulumi-azure-native==2.72.0 # - pulumi-azuread==6.0.1 # - pulumi-random==4.16.7 -# - pulumi==3.138.0 +# - pulumi==3.139.0 # - pydantic==2.9.2 -# - pyjwt[crypto]==2.9.0 +# - pyjwt[crypto]==2.10.0 # - pytz==2024.2 # - pyyaml==6.0.2 # - rich==13.9.4 @@ -102,13 +102,13 @@ azure-mgmt-resource==23.2.0 # via hatch.envs.default azure-mgmt-storage==21.2.1 # via hatch.envs.default -azure-storage-blob==12.23.1 +azure-storage-blob==12.24.0 # via # hatch.envs.default # azure-storage-file-datalake -azure-storage-file-datalake==12.17.0 +azure-storage-file-datalake==12.18.0 # via hatch.envs.default -azure-storage-file-share==12.19.0 +azure-storage-file-share==12.20.0 # via hatch.envs.default certifi==2024.8.30 # via @@ -192,13 +192,13 @@ psycopg==3.1.19 # via hatch.envs.default psycopg-binary==3.1.19 # via psycopg -pulumi==3.138.0 +pulumi==3.139.0 # via # hatch.envs.default # pulumi-azure-native # pulumi-azuread # pulumi-random -pulumi-azure-native==2.71.0 +pulumi-azure-native==2.72.0 # via hatch.envs.default pulumi-azuread==6.0.1 # via hatch.envs.default @@ -212,7 +212,7 @@ pydantic-core==2.23.4 # via pydantic pygments==2.18.0 # via rich -pyjwt==2.9.0 +pyjwt==2.10.0 # via # hatch.envs.default # msal From e31cbfd21ef794062010e9529e0a55155eac713e Mon Sep 17 00:00:00 2001 From: Matt Craddock Date: Tue, 19 Nov 2024 16:05:32 +0000 Subject: [PATCH 112/113] Update SECURITY.md --- SECURITY.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/SECURITY.md b/SECURITY.md index db056c976b..c81368a94e 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -7,8 +7,8 @@ All organisations using an earlier version in production should update to the la | Version | Supported | | --------------------------------------------------------------------------------------- | ------------------ | -| [5.0.1](https://github.com/alan-turing-institute/data-safe-haven/releases/tag/v5.0.1) | :white_check_mark: | -| < 5.0.1 | :x: | +| [5.1.0](https://github.com/alan-turing-institute/data-safe-haven/releases/tag/v5.1.0) | :white_check_mark: | +| < 5.1.0 | :x: | ## Reporting a Vulnerability From f9be9145a240a1efb33221fdcb7be550aa365edb Mon Sep 17 00:00:00 2001 From: Matt Craddock <5796417+craddm@users.noreply.github.com> Date: Thu, 21 Nov 2024 15:07:33 +0000 Subject: [PATCH 113/113] Bump to version 5.1.0 --- data_safe_haven/version.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data_safe_haven/version.py b/data_safe_haven/version.py index 0513a64c8f..6a7d91a4eb 100644 --- a/data_safe_haven/version.py +++ b/data_safe_haven/version.py @@ -1,2 +1,2 @@ -__version__ = "5.0.1" +__version__ = "5.1.0" __version_info__ = tuple(__version__.split("."))
Configuration of copy and paste#
true true yesyesyes (via local machine) yes yes
Benjamin Walden
Benjamin Walden
📖 🤔 🐛 📆 📓		 Brett Todd
Brett Todd
💻 🤔		 Callum Mole
Callum Mole
🐛 💻		Carlos Gavidia-Calderon
Carlos Gavidia-Calderon
🐛 🤔		Carlos Gavidia-Calderon
Carlos Gavidia-Calderon
🐛 🤔 📖		 Catalina Vallejos
Catalina Vallejos
🖋
Benjamin Walden
Benjamin Walden
📖 🤔 🐛 📆 📓		 Brett Todd
Brett Todd
💻 🤔		 Callum Mole
Callum Mole
🐛 💻		Carlos Gavidia-Calderon
Carlos Gavidia-Calderon
🐛 🤔		Carlos Gavidia-Calderon
Carlos Gavidia-Calderon
🐛 🤔 📖		 Catalina Vallejos
Catalina Vallejos
🖋