Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Management documentation updates #2254

Merged
merged 24 commits into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
209c706
reorder data_ingress sections
craddm Oct 23, 2024
7ddc304
update management docs
craddm Oct 23, 2024
3f3511d
Fix linting errors
craddm Oct 23, 2024
f4915cb
delete blank line
craddm Oct 23, 2024
9fb7541
Update docs/source/management/index.md
craddm Oct 23, 2024
55795da
Update docs/source/management/index.md
craddm Oct 23, 2024
a1a13f4
Apply many review suggestions
craddm Oct 23, 2024
89961da
Update docs/source/roles/data_provider_representative/data_ingress.md
craddm Oct 23, 2024
3d2ad85
Remove reference to approval/classification of output
craddm Oct 23, 2024
bd92224
Fix typo
craddm Oct 23, 2024
4862382
Switch to more consistent style of highlighting and labelling menu op…
craddm Oct 23, 2024
d5181b1
fix style of some remaining terms
craddm Oct 23, 2024
829a5f2
Apply review suggestions
craddm Oct 24, 2024
a60d55a
More detailed information about tearing down SHM/SRE, and more detail…
craddm Oct 24, 2024
6fb893e
Fix admonition titles
craddm Oct 24, 2024
e3b1b7b
Reword outputs section
craddm Oct 24, 2024
672e141
Merge remote-tracking branch 'upstream/release-v5.0.1rc1' into doc-up…
craddm Oct 24, 2024
5dee1ed
Update docs/source/management/index.md
craddm Oct 24, 2024
04b8d3a
Update docs/source/management/index.md
craddm Oct 24, 2024
440e30d
Update docs/source/management/index.md
craddm Oct 24, 2024
df08f6d
Update docs/source/management/index.md
craddm Oct 24, 2024
bd02f84
Update docs/source/management/index.md
craddm Oct 24, 2024
4c71226
Update docs/source/management/index.md
craddm Oct 24, 2024
f12de89
Update docs/source/management/index.md
craddm Oct 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/deployment/configure_entra_id.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Use the following settings:
- **Other fields:** leave them with their default values
- **Properties** tab:
- **Usage location:** set to the country being used for this deployment
- **Assigments** tab:
- **Assignments** tab:
- Click the **{guilabel}`+ Add role`** button
- Search for **Global Administrator**, and check the box
- Click the **{guilabel}`Select`** button
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
100 changes: 94 additions & 6 deletions docs/source/management/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Management

## Add users to the Data Safe Haven
## Managing users

### Add users to the Data Safe Haven

:::{important}
You will need a full name, phone number, email address and country for each user.
Expand All @@ -27,7 +29,7 @@ Grace;Hopper;+18005550100;[email protected];US
$ dsh users add PATH_TO_MY_CSV_FILE
```

## Listing available users
### Listing available users
craddm marked this conversation as resolved.
Show resolved Hide resolved

- You can do this from the [Microsoft Entra admin centre](https://entra.microsoft.com/)

Expand All @@ -54,7 +56,7 @@ $ dsh users add PATH_TO_MY_CSV_FILE
└──────────────────────────────┴──────────┴───────────────────┘
```

## Assign existing users to an SRE
### Assign existing users to an SRE

1. You can do this directly in your Entra tenant by adding them to the **Data Safe Haven SRE _YOUR\_SRE\_NAME_ Users** group, following the instructions [here](https://learn.microsoft.com/en-us/entra/fundamentals/groups-view-azure-portal#add-a-group-member).

Expand All @@ -70,7 +72,7 @@ $ dsh users add PATH_TO_MY_CSV_FILE
Do not include the Entra ID domain part of the username, just the part before the @.
:::

## Manually register users for self-service password reset
### Manually register users for self-service password reset

:::{tip}
Users created via the `dsh users` command line tool will be automatically registered for SSPR.
Expand All @@ -87,7 +89,9 @@ If you have manually created a user and want to enable SSPR, do the following
- **Email:** enter the user's email address here
- Click the **{guilabel}`Save`** icon in the top panel

## Listing available SRE configurations and deployment status
## Managing SREs

### Listing available SRE configurations and deployment status
craddm marked this conversation as resolved.
Show resolved Hide resolved

- Run the following if you want to check what SRE configurations are available in the current context, and whether those SREs are deployed

Expand All @@ -108,16 +112,100 @@ Available SRE configurations for context 'green':
└──────────────┴──────────┘
```

## Removing a deployed Data Safe Haven
### Removing a deployed Data Safe Haven
craddm marked this conversation as resolved.
Show resolved Hide resolved

- Run the following if you want to teardown a deployed SRE:

```{code} shell
$ dsh sre teardown YOUR_SRE_NAME
```

::::{danger} Tearing down an SRE is destructive and irreversible
Running `dsh sre teardown` will destroy **all** resources deployed within the SRE.
Ensure that any desired outputs have been extracted before deleting the SRE.
**All** data remaining on the SRE will be deleted.
The user groups for the SRE on Microsoft Entra ID will also be deleted.
::::

- Run the following if you want to teardown the deployed SHM:

```{code} shell
$ dsh shm teardown
```

::::{warning} Tearing down an SHM
All SREs associated with the SHM must be torn down before the SHM can be torn down.
craddm marked this conversation as resolved.
Show resolved Hide resolved
Tearing down the SHM permanently deletes **all** remotely stored configuration and state data.
::::

## Managing data ingress and egress

### Data Ingress

It is the data provider representative's responsibility to upload the data required by the safe haven.
craddm marked this conversation as resolved.
Show resolved Hide resolved

The following steps show how to generate a temporary write-only upload token that can be securely sent to the data provider, enabling them to upload the data:
craddm marked this conversation as resolved.
Show resolved Hide resolved

- In the Azure portal select **Subscriptions** then navigate to the subscription containing the relevant SHM
- Search for the resource group: **`shm-<YOUR_SHM_NAME>-sre-<YOUR_SRE_NAME>-rg`**, then click through to the storage account called: **`sh<first three letters of SHM name>sre<first three letters of SRE name>sensitivedata`**
craddm marked this conversation as resolved.
Show resolved Hide resolved
- Browse to **{menuselection}`Settings --> Networking`** and ensure that the data provider's IP address is one of those allowed under the **Firewall** header
- If it is not listed, modify and reupload the SRE configuration and redeploy the SRE using the `dsh` CLI, as per {ref}`deploy_sre`
- Browse to **{menuselection}`Data storage --> Containers`** from the menu on the left hand side
- Click **ingress**
- Browse to **{menuselection}`Settings --> Shared access tokens`** and do the following:
- Under **Signing method**, select **User delegation key**
- Under **Permissions**, check these boxes:
- **Write**
- **List**
- Set a 24 hour time window in the **Start and expiry date/time** (or an appropriate length of time)
- Leave everything else as default and click **{guilabel}`Generate SAS token and URL`**
- Copy the **Blob SAS URL**

```{image} ingress_token_write_only.png
:alt: write-only SAS token
:align: center
```

- Send the **Blob SAS URL** to the data provider through a secure channel
- The data provider should now be able to upload data
- Validate successful data ingress
- Browse to **{menuselection}`Data storage --> Containers`** (in the middle of the page)
- Select the **ingress** container and ensure that the uploaded files are present

### Data egress

```{important}
Assessment of output must be completed **before** an egress link is created.
craddm marked this conversation as resolved.
Show resolved Hide resolved
```

The {ref}`role_system_manager` creates a time-limited and IP restricted link to remove data from the environment.

- In the Azure portal select **Subscriptions** then navigate to the subscription containing the relevant SHM
- Search for the resource group: **`shm-<YOUR_SHM_NAME>-sre-<YOUR_SRE_NAME>-rg`**, then click through to the storage account called: **`sh<first three letters of SHM name>sre<first three letters of SRE name>sensitivedata`**
craddm marked this conversation as resolved.
Show resolved Hide resolved
- Browse to **{menuselection}`Settings --> Networking`** and check the list of pre-approved IP addresses allowed under the **Firewall** header
- Ensure that the IP address of the person to receive the outputs is listed
- If it is not listed, modify and reupload the SRE configuration and redeploy the SRE using the `dsh` CLI, as per {ref}`deploy_sre`
- Browse to **{menuselection}`Data storage --> Containers`**
- Select the **egress** container
- Browse to **{menuselection}`Settings --> Shared access tokens`** and do the following:
- Under **Signing method**, select **User delegation key**
- Under **Permissions**, check these boxes:
- **Read**
- **List**
- Set a time window in the **Start and expiry date/time** that gives enough time for the person who will perform the secure egress download to do so
- Leave everything else as default and press **{guilabel}`Generate SAS token and URL`**
- Copy the **Blob SAS URL**

```{image} egress_token_read_only.png
:alt: Read-only SAS token
:align: center
```

- Send the **Blob SAS URL** to the relevant person through a secure channel
- The appropriate person should now be able to download data

### The output volume

Once you have set up the egress connection in `Azure Storage Explorer`, you should be able to view data from the **output volume**, a read-write area intended for the extraction of results, such as figures for publication.
craddm marked this conversation as resolved.
Show resolved Hide resolved
On the workspaces, this volume is `/mnt/output` and is shared between all workspaces in an SRE.
For more information on shared SRE storage volumes, consult the {ref}`Safe Haven User Guide <role_researcher_shared_storage>`.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
30 changes: 15 additions & 15 deletions docs/source/roles/data_provider_representative/data_ingress.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,22 @@
The **Dataset Provider Representative** plays an important role in data ingress.
As well as being involved in agreeing an appropriate security tier for a project, they may also prepare the data to be uploaded.

## Preparing data
## Bringing data into the environment

Talk to your {ref}`role_system_manager` to discuss possible methods of bringing data into the environments.
It may be convenient to use [Azure Storage Explorer](https://azure.microsoft.com/en-us/products/storage/storage-explorer/).
In this case you will not need log-in credentials, as your {ref}`role_system_manager` can provide a short-lived secure access token which will let you upload data.

```{tip}
You may want to keep the following considerations in mind when transferring data in order to reduce the chance of a data breach.

- Use of short-lived access tokens limits the time within which an attacker can operate.
- Letting your {ref}`role_system_manager` know a fixed IP address you will be connecting from (_e.g._ a corporate VPN) limits the places an attacker can operate from.
- Communicating with your {ref}`role_system_manager` through a secure out-of-band channel (_e.g._ encrypted email) reduces the chances that an attacker can intercept or alter your messages in transit.

```

This section has some recommendations for preparing input data for the Data Safe Haven.
## Preparing input data for the Data Safe Haven

### Avoid archives

Expand Down Expand Up @@ -86,16 +99,3 @@ md5sum -c hashes.txt | grep FAILED
```

To use the `sha256` algorithm, replace `md5sum` with `sha256` in the above commands.

## Bringing data into the environment

Talk to your {ref}`role_system_manager` to discuss possible methods of bringing data into the environments.
It may be convenient to use [Azure Storage Explorer](https://azure.microsoft.com/en-us/products/storage/storage-explorer/).
In this case you will not need log-in credentials, as your {ref}`role_system_manager` can provide a short-lived secure access token which will let you upload data.

```{tip}
You may want to keep the following considerations in mind when transferring data in order to reduce the chance of a data breach
- use of short-lived access tokens limits the time within which an attacker can operate
- letting your {ref}`role_system_manager` know a fixed IP address you will be connecting from (eg. a corporate VPN) limits the places an attacker can operate from
- communicating with your {ref}`role_system_manager` through a secure out-of-band channel (eg. encrypted email) reduces the chances that an attacker can intercept or alter your messages in transit
```
Loading