Move pulumi context settings and persistent data

[JimMadge]

✅ Checklist

• You have given your pull request a meaningful title (e.g. Enable foobar integration rather than 515 foobar).
• You are targeting the appropriate branch. If you're not certain which one this is, it should be develop.
• Your branch is up-to-date with the target branch (it probably was when you started, but it may have changed since then).
• You have marked this pull request as a draft and added '[WIP]' to the title if needed (if you're not yet ready to merge).

⤴️ Summary

• Add ConfigClass base class for configuration synced to Azure Storage in the Context
  • Add from_remote_or_create method to fetch config if it exists in Azure Storage, otherwise create a new instance
• Add DSHPulumiProject and DSHPulumiConfig classes for storing persistent Pulumi project information
• Use ConfigClass for both Config and DSHPulumiConfig
• Remove the need for Context to construct Config. Removes a lot of potential circular imports.
• Use information from DSHPulumiProject instances to create Pulumi workspaces and stacks, avoiding the need to write Pulumi.<stack>.yaml and Pulumi.yaml files in a non-temp work directory
• Propagated above changes through infrastructure and commands modules
• Many tests
  • Adding initial tests for StackManager classes
  • Add mocking frameworks for testing some Azure/Pulumi functions

🌂 Related issues

Closes #1689 #1828

🔬 Tests

New deployment, repeating commands successfully fetches Pulumi persistent data (stack settings, stack state)

[JimMadge]

Working on this, the names are becoming a mess.

Terms here.

We have two Pulumi Programs, one for SHM, one for SRE.
Each 'thing' we deploy is a new Pulumi Project, with a Local Workspace containing project configuration Pulumi.yaml and a single stack with configuration Pulumi.<stack>.yaml. (We might not need the Local Workspace at all #1828)

We could use stacks a bit more like they are intended, with dev and prod deployments. I don't see us doing that now though. So, the state is, each 'thing' is the only stack of a Pulumi Project.

I think that has lead to conflating the terms program/stack and not using the term project.

I propose

• DSHPulumiConfig - Class to hold persistent Pulumi information that is written to the config share (multiple instances of DSHPulumiProject. I.e. not stacks which are in the Pulumi backend, not the Pulumi backend address/encryption key which are in the context
• DSHPulumiProject - Class to hold persistent information about Pulumi projects, which right now is just a name and a single stack configuration (like Pulumi.<stack>.yaml)
• We should be more strict and consistent with naming and be careful not to call projects stacks

@craddm @jemrobinson thoughts?

[JimMadge]

Also, I must be tired because I implemented a class where you can look up items using strings, which must be unique, and didn't use a dict 🤦.

[jemrobinson]

Can you give a quick sketch of what would be in each of these config files? Could be e.g. a YAML representation of the pieces of data in each of them.

[JimMadge]

Can you give a quick sketch of what would be in each of these config files?

The Pulumi files, or the classes we are creating?

For the later, I would write the DSHPulumiConfig class to the storage account in the context in a similar way to Config, taking advantage of the Pydantic serializer.

[jemrobinson]

The Pulumi files, or the classes we are creating?

The data structure of the classes we're creating (I'm not worried about the serialisation part). I'm thinking something like:

DSHPulumiConfig
- azure
  - subscription_id
  - tenant_id
- something_else

DSHPulumiProject
- something
  - A
  - B
- something_else
  - C
  - D  

[JimMadge]

This is working now and I think I've addressed all the comments.