Azure Cloud Starter Project

Create and manage the basic resources needed for a new Azure project
using Terraform and Github Actions.

Getting Started

You will need to install the Azure CLI and log-in. I would also reccommend you install a secrets or password manager to hold the credentials we will create. I'll be using a free Bitwarden account along with their bitwarden-cli to manage my secrets for this demonstration.

Install Azure CLI
```
brew update && \
brew install azure-cli
```
Install Bitwarden CLI
```
brew install bitwarden-cli
```
Login
```
# Authorize the Azure CLI
az login
```

IAM Setup

You need to create a service account to represent your digital self and use that to run terraform locally. In production, a unique service account should be created for running and applying the terraform jobs,and it should create smaller accounts at instantiation to run the infra it provisions.

'Owner' level access is required because we need to create role assignments. This may potentially be scoped down to 'User Access Administrator' + 'Contributor'

Create your service principle and then add the resulting data to KeePassXC / Bitwarden for now. You will need it again multiple times.

SUBSCRIPTION=$(az account show --query id --output tsv)
SP_NAME="myserviceaccount"

az ad sp create-for-rbac --sdk-auth \
  --display-name="${SP_NAME}" \
  --role="Owner" \
  --scopes="/subscriptions/$SUBSCRIPTION"

Setup the Azure Active Directory Permissions for your Service Principle. This is required in order to set AD roles in terraform.
- Login to https://portal.azure.com/
- Navigate to Azure Active Directory
- Select Roles and administrators from the left-side menu
- Click Application administrator
- Click Add Assignments
- Search for your service accounts name
- Repeat for Application Developer Role.

Creating the State Bucket

Before we start you should login to Azure again, but now as the service principle we created. We will use this account to create the terraform state bucket. That way, only the service-principle will have access to it.

az login --service-principal \
   --username $(bw get item admin-robot |jq -r '.fields[] |select(.name=="clientId") |.value') \
   --password $(bw get item admin-robot |jq -r '.fields[] |select(.name=="clientSecret") |.value') \
   --tenant $(bw get item admin-robot |jq -r '.fields[] |select(.name=="tenantId") |.value')

Now Create the Terraform state bucket

export SUBSCRIPTION=$(az account show --query id --output tsv)    
export KIND="StorageV2"
export LOCATION="westeurope"
export RG_NAME="example-tf-state"
export STORAGE_NAME="examplertfstatebucket"
export STORAGE_SKU="Standard_RAGRS"
export CONTAINER_NAME="exampletfstate"

az group create \
  -l="${LOCATION}" \
  -n="${RG_NAME}"

az storage account create \
  --name="${STORAGE_NAME}" \
  --resource-group="${RG_NAME}" \
  --location="${LOCATION}" \
  --sku="${STORAGE_SKU}" \
  --kind="${KIND}"

az storage account encryption-scope create \
  --account-name="${STORAGE_NAME}"  \
  --key-source Microsoft.Storage \
  --name="tfencryption"\
  --resource-group="${RG_NAME}" \
  --subscription="${SUBSCRIPTION}"

az storage container create \
    --name="${CONTAINER_NAME}" \
    --account-name="${STORAGE_NAME}" \
    --resource-group="${RG_NAME}" \
    --default-encryption-scope="tfencryption" \
    --prevent-encryption-scope-override="true" \
    --auth-mode="login" \
    --fail-on-exist \
    --public-access="off"

Initializing your Terraform Project

Add your state-bucket details to the providers.tf file if you made any customisations

backend "azurerm" {
   resource_group_name  = "example-tf-state"
   storage_account_name = "examplertfstatebucket"
   container_name       = "exampletfstate"
   key                  = "example.terraform.tfstate"
}

Add your ip-address and personal azure account's clientID to the environment-base.tf file under the Firewall header.
```
# Firewall
allowed_ips = ["192.168.50.1"]
admin_users = ["clientId-goes-here"]
```

Initialize the terraform project

docker run --platform linux/amd64 -it \
   -e ARM_CLIENT_ID=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="clientId") |.value') \
   -e ARM_CLIENT_SECRET=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="clientSecret") |.value') \
   -e ARM_SUBSCRIPTION_ID=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="subscriptionId") |.value') \
   -e ARM_TENANT_ID=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="tenantId") |.value') \
   -v $(pwd):/terraform -w /terraform \
   hashicorp/terraform init

Plan / Apply resources

docker run --platform linux/amd64 -it \
   -e ARM_CLIENT_ID=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="clientId") |.value') \
   -e ARM_CLIENT_SECRET=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="clientSecret") |.value') \
   -e ARM_SUBSCRIPTION_ID=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="subscriptionId") |.value') \
   -e ARM_TENANT_ID=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="tenantId") |.value') \
   -v $(pwd):/terraform -w /terraform \
   hashicorp/terraform apply

Destroy Resources

docker run --platform linux/amd64 -it \
   -e ARM_CLIENT_ID=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="clientId") |.value') \
   -e ARM_CLIENT_SECRET=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="clientSecret") |.value') \
   -e ARM_SUBSCRIPTION_ID=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="subscriptionId") |.value') \
   -e ARM_TENANT_ID=$(bw get item admin-robot |jq -r '.fields[] |select(.name=="tenantId") |.value') \
   -v $(pwd):/terraform -w /terraform \
   hashicorp/terraform destroy

Cleanup
```
# ToDo
```

Instance Types and Limits

This project is tested using NVadsA10v5, NCasT4_v3, and NVv3 instances which utilize Nvidia A10, T4 and M60 GPUs. If you do not need a GPU, you are advised to consider Hetzner or Equinix who have better prices on CPU-only instances. Buildstar Online the following quota limits in west-europe:

72 vCores for NVadsA10v5
64 vCores for NCasT4_v3
48 vCores for NVv3

All prices are for Spot/hourly in the closest datacenter to Amsterdam, NL

NVadsA10v5

NVadsA10v5 series virtual machines are powered by NVIDIA A10 GPUs and AMD EPYC 74F3V(Milan) CPUs with a base frequency of 3.2 GHz, peak frequency of all cores of 4.0 GHz.

Instance Size	GPUs	GPU RAM	vCPUs	RAM (GiB)	Disk Size (GB)	Network (Gbps)	Spot Price
Standard_NV6ads_A10_v5	1/6	4	6	55	180	5	€0.1362
Standard_NV12ads_A10_v5	1/3	8	12	110	360	10	€0.2724
Standard_NV18ads_A10_v5	1/2	12	18	220	720	20	€0.4802
Standard_NV36ads_A10_v5	1	24	36	440	1440	40	€0.9604
Standard_NV72ads_A10_v5	2	48	72	880	2880	80	€1.9568

NCasT4_v3

NCasT4_v3 series virtual machines are powered by Nvidia Tesla T4 GPUs and AMD EPYC 7V12(Rome) CPUs.

Instance Size	GPUs	GPU RAM	vCPUs	RAM (GiB)	Disk Size (GB)	Network (Gbps)	Spot Price
Standard_NC4as_T4_v3	1	16	4	28	180	8	€0.1275
Standard_NC8as_T4_v3	1	16	8	56	360	8	€0.1821
Standard_NC16as_T4_v3	1	16	16	110	360	8	€0.2916
Standard_NC64as_T4_v3	4	64	64	440	2880	32	€1.0539

NVv3

The NVv3 series virtual machines are powered by NVIDIA Tesla M60 GPUs and NVIDIA GRID technology with Intel E5-2690 v4 (Broadwell) CPUs and Intel Hyper-Threading Technology.

Instance Size	GPUs	GPU RAM	vCPUs	RAM (GiB)	Disk Size (GB)	Network (Gbps)	Spot Price
Standard_NV12s_v3	1	8	12	112	320	6	€0.1317
Standard_NV24s_v3	2	16	24	224	640	12	€0.2632
Standard_NV48s_v3	4	32	48	448	1280	24	€0.5264

Requirements

Name	Version
azuread	~>2.36.0
azurerm	~>3.47.0
random	~>3.4.3
time	~>0.9.1
tls	~>4.0.4

Providers

Name	Version
azurerm	~>3.47.0

Modules

Name	Source	Version
environment-base	github.com/cloudymax/modules-azure-tf-base	n/a
scale-set	github.com/cloudymax/modules-azure-tf-scale-set	n/a

Resources

Name	Type
azurerm_client_config.current	data source

Inputs

Name	Description	Type	Default	Required
account_replication_type	n/a	`string`	`"LRS"`	no
account_tier	n/a	`string`	`"Standard"`	no
admin_identity	n/a	`string`	`"bradley"`	no
allowed_ips	n/a	`list(string)`	n/a	yes
cr_sku	n/a	`string`	`"Basic"`	no
environment	n/a	`string`	`"dmeo"`	no
eviction_policy	n/a	`string`	`"Deallocate"`	no
github_username	n/a	`string`	n/a	yes
hostname	n/a	`string`	`"azurespot"`	no
kv_sku_name	n/a	`string`	`"standard"`	no
location	n/a	`string`	`"westeurope"`	no
log_storage_tier	n/a	`string`	`"Hot"`	no
max_bid_price	n/a	`string`	n/a	yes
overprovision	n/a	`bool`	`false`	no
priority	n/a	`string`	`"Spot"`	no
resource_group	n/a	`string`	`"demo-rg"`	no
scale_in_force_deletion_enabled	n/a	`bool`	`true`	no
scale_in_rule	n/a	`string`	`"NewestVM"`	no
scale_set_name	n/a	`string`	`"scale-set"`	no
spot_restore_enabled	n/a	`bool`	`true`	no
spot_restore_timeout	n/a	`string`	`"PT1H30M"`	no
ultra_ssd_enabled	n/a	`bool`	`false`	no
user_data_path	n/a	`string`	`"./NVadsA10v5.yaml"`	no
username	n/a	`string`	n/a	yes
vm_instances	n/a	`number`	`1`	no
vm_network_interface	n/a	`string`	`"vm-nic"`	no
vm_os_disk_size_gb	n/a	`number`	`64`	no
vm_sku	n/a	`string`	n/a	yes

Outputs

No outputs.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

Azure Cloud Starter Project

Getting Started

IAM Setup

Creating the State Bucket

Initializing your Terraform Project

Instance Types and Limits

NVadsA10v5

NCasT4_v3

NVv3

Requirements

Providers

Modules

Resources

Inputs

Outputs

Files

README.md

Latest commit

History

README.md

File metadata and controls

Azure Cloud Starter Project

Getting Started

IAM Setup

Creating the State Bucket

Initializing your Terraform Project

Instance Types and Limits

NVadsA10v5

NCasT4_v3

NVv3

Requirements

Providers

Modules

Resources

Inputs

Outputs