Skip to content

Commit

Permalink
Support Azure Workload Identity
Browse files Browse the repository at this point in the history
With an update to github.com/fluxcd/pkg/oci v0.22.0.

This includes a pin of `github.com/docker/docker` to `v20.10.x`, to
prevent Oras from complaining.

Co-authored-by: Hidde Beydals <[email protected]>
Signed-off-by: Somtochi Onyekwere <[email protected]>
  • Loading branch information
somtochiama and hiddeco committed Mar 30, 2023
1 parent 6b235f0 commit 4995cff
Show file tree
Hide file tree
Showing 7 changed files with 283 additions and 78 deletions.
95 changes: 89 additions & 6 deletions docs/spec/v1beta2/buckets.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,7 @@ Without a [Secret reference](#secret-reference), authentication using a chain
with:

- [Environment credentials](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#EnvironmentCredential)
- [Workload Identity](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/[email protected]#WorkloadIdentityCredential)
- [Managed Identity](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity#ManagedIdentityCredential)
with the `AZURE_CLIENT_ID`
- Managed Identity with a system-assigned identity
Expand Down Expand Up @@ -436,16 +437,97 @@ data:
accountKey: <BASE64>
```

#### Managed Identity with AAD Pod Identity
##### Workload Identity

If you are using [aad pod identity](https://azure.github.io/aad-pod-identity/docs), you can create an identity that has access to Azure Storage.
If you have [Workload Identity mutating webhook](https://azure.github.io/azure-workload-identity/docs/installation/managed-clusters.html)
installed on your cluster. You would need to create an Azure Identity and
give it access to Azure Blob Storage.

```shell
export IDENTITY_NAME="blob-access"
az role assignment create --role "Storage Blob Data Reader" \
--assignee-object-id "$(az identity show -n $IDENTITY_NAME -o tsv --query principalId -g $RESOURCE_GROUP)" \
--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
```

Establish federated identity between the identity and the source-controller
ServiceAccount.

```shell
export SERVICE_ACCOUNT_ISSUER="$(az aks show --resource-group <RESOURCE_GROUP> --name <CLUSTER-NAME> --query "oidcIssuerProfile.issuerUrl" -otsv)"
az identity federated-credential create \
--name "kubernetes-federated-credential" \
--identity-name "${IDENTITY_NAME}" \
--resource-group "${RESOURCE_GROUP}" \
--issuer "${SERVICE_ACCOUNT_ISSUER}" \
--subject "system:serviceaccount:flux-system:source-controller"
```

Add a patch to label and annotate the source-controller Pods and ServiceAccount
correctly so that it can match an identity binding:

```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gotk-components.yaml
- gotk-sync.yaml
patches:
- patch: |-
apiVersion: v1
kind: ServiceAccount
metadata:
name: source-controller
namespace: flux-system
annotations:
azure.workload.identity/client-id: <AZURE_CLIENT_ID>
labels:
azure.workload.identity/use: "true"
- patch: |-
apiVersion: apps/v1
kind: Deployment
metadata:
name: source-controller
namespace: flux-system
labels:
azure.workload.identity/use: "true"
spec:
template:
metadata:
labels:
azure.workload.identity/use: "true"
```

If you have set Workload Identity up correctly and labeled the source-controller
Pod and ServiceAccount, then you don't need to reference a Secret. For more information,
please see [documentation](https://azure.github.io/azure-workload-identity/docs/quick-start.html).

```yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: Bucket
metadata:
name: azure-bucket
namespace: flux-system
spec:
interval: 5m0s
provider: azure
bucketName: testsas
endpoint: https://testfluxsas.blob.core.windows.net
```

##### Managed Identity with AAD Pod Identity

If you are using [aad pod identity](https://azure.github.io/aad-pod-identity/docs),
you can create an identity that has access to Azure Storage.

```sh
export IDENTITY_NAME="blob-access"
az role assignment create --role "Storage Blob Data Contributor" \
--assignee-object-id "$(az identity show -n blob-access -o tsv --query principalId -g $RESOURCE_GROUP)" \
--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/aks-somto/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
az role assignment create --role "Storage Blob Data Reader" \
--assignee-object-id "$(az identity show -n $IDENTITY_NAME -o tsv --query principalId -g $RESOURCE_GROUP)" \
--scope "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.Storage/storageAccounts/<account-name>/blobServices/default/containers/<container-name>"
export IDENTITY_CLIENT_ID="$(az identity show -n ${IDENTITY_NAME} -g ${RESOURCE_GROUP} -otsv --query clientId)"
export IDENTITY_RESOURCE_ID="$(az identity show -n ${IDENTITY_NAME} -otsv --query id)"
Expand Down Expand Up @@ -493,7 +575,8 @@ spec:
aadpodidbinding: ${IDENTITY_NAME} # match the AzureIdentity name
```

If you have set aad-pod-identity up correctly and labeled the source-controller pod, then you don't need to reference a secret.
If you have set aad-pod-identity up correctly and labeled the source-controller
pod, then you don't need to reference a secret.

```yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
Expand Down
55 changes: 55 additions & 0 deletions docs/spec/v1beta2/helmrepositories.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,9 +190,13 @@ The `aws` provider can be used to authenticate automatically using the EKS worke
node IAM role or IAM Role for Service Accounts (IRSA), and by extension gain access
to ECR.

##### EKS Worker Node IAM Role

When the worker node IAM role has access to ECR, source-controller running on it
will also have access to ECR.

##### IAM Role for Service Accounts (IRSA)

When using IRSA to enable access to ECR, add the following patch to your bootstrap
repository, in the `flux-system/kustomization.yaml` file:

Expand Down Expand Up @@ -224,9 +228,56 @@ The `azure` provider can be used to authenticate automatically using kubelet man
identity or Azure Active Directory pod-managed identity (aad-pod-identity), and
by extension gain access to ACR.

##### Kubelet Managed Identity

When the kubelet managed identity has access to ACR, source-controller running on
it will also have access to ACR.

##### Workload Identity

When using Workload Identity to enable access to ACR, add the following patch to
your bootstrap repository, in the `flux-system/kustomization.yaml` file:

```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gotk-components.yaml
- gotk-sync.yaml
patches:
- patch: |-
apiVersion: v1
kind: ServiceAccount
metadata:
name: source-controller
namespace: flux-system
annotations:
azure.workload.identity/client-id: <AZURE_CLIENT_ID>
labels:
azure.workload.identity/use: "true"
- patch: |-
apiVersion: apps/v1
kind: Deployment
metadata:
name: source-controller
namespace: flux-system
labels:
azure.workload.identity/use: "true"
spec:
template:
metadata:
labels:
azure.workload.identity/use: "true"
```

To use Workload Identity, you have to install the Workload Identity
mutating webhook and create an identity that has access to ACR. Next, establish
a federated identity between the source-controller ServiceAccount and the
identity. Patch the source-controller Pod and ServiceAccount as shown in the patch
above. Please take a look at this [guide](https://azure.github.io/azure-workload-identity/docs/quick-start.html#6-establish-federated-identity-credential-between-the-identity-and-the-service-account-issuer--subject).

##### AAD Pod Identity

When using aad-pod-identity to enable access to ACR, add the following patch to
your bootstrap repository, in the `flux-system/kustomization.yaml` file:

Expand Down Expand Up @@ -261,9 +312,13 @@ if you want to use AKS pod-managed identities add-on that is in preview.
The `gcp` provider can be used to authenticate automatically using OAuth scopes or
Workload Identity, and by extension gain access to GCR or Artifact Registry.

#### Access Scopes

When the GKE nodes have the appropriate OAuth scope for accessing GCR and Artifact Registry,
source-controller running on it will also have access to them.

#### Workload Identity

When using Workload Identity to enable access to GCR or Artifact Registry, add the
following patch to your bootstrap repository, in the `flux-system/kustomization.yaml`
file:
Expand Down
46 changes: 46 additions & 0 deletions docs/spec/v1beta2/ocirepositories.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,9 +161,55 @@ The `azure` provider can be used to authenticate automatically using kubelet
managed identity or Azure Active Directory pod-managed identity (aad-pod-identity),
and by extension gain access to ACR.

##### Kubelet Managed Identity

When the kubelet managed identity has access to ACR, source-controller running
on it will also have access to ACR.

##### Workload Identity

When using Workload Identity to enable access to ACR, add the following patch to
your bootstrap repository, in the `flux-system/kustomization.yaml` file:

```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- gotk-components.yaml
- gotk-sync.yaml
patches:
- patch: |-
apiVersion: v1
kind: ServiceAccount
metadata:
name: source-controller
namespace: flux-system
annotations:
azure.workload.identity/client-id: <AZURE_CLIENT_ID>
labels:
azure.workload.identity/use: "true"
- patch: |-
apiVersion: apps/v1
kind: Deployment
metadata:
name: source-controller
namespace: flux-system
labels:
azure.workload.identity/use: "true"
spec:
template:
metadata:
labels:
azure.workload.identity/use: "true"
```

To use Workload Identity, you have to install the Workload Identity
mutating webhook and create an identity that has access to ACR. Next, establish
a federated identity between the source-controller ServiceAccount and the
identity. Patch the source-controller Pod and ServiceAccount as shown in the patch
above. Please take a look at this [guide](https://azure.github.io/azure-workload-identity/docs/quick-start.html#6-establish-federated-identity-credential-between-the-identity-and-the-service-account-issuer--subject).

##### AAD Pod Identity
When using aad-pod-identity to enable access to ACR, add the following patch to
your bootstrap repository, in the `flux-system/kustomization.yaml` file:

Expand Down
49 changes: 28 additions & 21 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,18 @@ replace github.com/fluxcd/source-controller/api => ./api
// xref: https://github.com/opencontainers/go-digest/pull/66
replace github.com/opencontainers/go-digest => github.com/opencontainers/go-digest v1.0.1-0.20220411205349-bde1400a84be

// Required to keep oras.land/oras-go happy, as it will otherwise fail with
// "assignment mismatch: 3 variables but registry.PingV2Registry returns 2 values"
//
// Check again when oras.land/oras-go is updated to >=v2.0.0, which is a
// dependency of Helm.
replace github.com/docker/docker => github.com/docker/docker v20.10.23+incompatible

require (
cloud.google.com/go/storage v1.30.1
github.com/AdaLogics/go-fuzz-headers v0.0.0-20230106234847-43070de90fa1
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.4.0
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.2.1
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.5.0-beta.1
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.3.0-beta.4
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.0.0
github.com/Masterminds/semver/v3 v3.2.0
github.com/cyphar/filepath-securejoin v0.2.3
Expand All @@ -28,7 +35,7 @@ require (
github.com/fluxcd/pkg/helmtestserver v0.12.0
github.com/fluxcd/pkg/lockedfile v0.1.0
github.com/fluxcd/pkg/masktoken v0.2.0
github.com/fluxcd/pkg/oci v0.21.1
github.com/fluxcd/pkg/oci v0.22.0
github.com/fluxcd/pkg/runtime v0.35.0
github.com/fluxcd/pkg/sourceignore v0.3.3
github.com/fluxcd/pkg/ssh v0.7.3
Expand All @@ -38,7 +45,7 @@ require (
github.com/fluxcd/source-controller/api v0.36.1
github.com/go-git/go-billy/v5 v5.4.1
github.com/go-logr/logr v1.2.3
github.com/google/go-containerregistry v0.13.0
github.com/google/go-containerregistry v0.14.0
github.com/google/go-containerregistry/pkg/authn/k8schain v0.0.0-20230307034325-57f010d26af8
github.com/google/uuid v1.3.0
github.com/minio/minio-go/v7 v7.0.50
Expand Down Expand Up @@ -85,7 +92,7 @@ require (
github.com/Azure/go-autorest/autorest/date v0.3.0 // indirect
github.com/Azure/go-autorest/logger v0.2.1 // indirect
github.com/Azure/go-autorest/tracing v0.6.0 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v0.8.1 // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v0.9.0 // indirect
github.com/BurntSushi/toml v1.2.1 // indirect
github.com/MakeNowJust/heredoc v1.0.0 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
Expand All @@ -109,19 +116,19 @@ require (
github.com/alibabacloud-go/tea-xml v1.1.2 // indirect
github.com/aliyun/credentials-go v1.2.3 // indirect
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
github.com/aws/aws-sdk-go-v2 v1.17.5 // indirect
github.com/aws/aws-sdk-go-v2/config v1.18.15 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.13.15 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.12.23 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.29 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.23 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.30 // indirect
github.com/aws/aws-sdk-go-v2/service/ecr v1.18.5 // indirect
github.com/aws/aws-sdk-go-v2 v1.17.7 // indirect
github.com/aws/aws-sdk-go-v2/config v1.18.19 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.13.18 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.13.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.1.31 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.4.25 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.3.32 // indirect
github.com/aws/aws-sdk-go-v2/service/ecr v1.18.7 // indirect
github.com/aws/aws-sdk-go-v2/service/ecrpublic v1.13.17 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.23 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.12.4 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.4 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.18.5 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.9.25 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.12.6 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.14.6 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.18.7 // indirect
github.com/aws/smithy-go v1.13.5 // indirect
github.com/awslabs/amazon-ecr-credential-helper/ecr-login v0.0.0-20221004211355-a250ad2ca1e3 // indirect
github.com/benbjohnson/clock v1.1.0 // indirect
Expand All @@ -144,7 +151,7 @@ require (
github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be // indirect
github.com/containerd/containerd v1.6.18 // indirect
github.com/containerd/continuity v0.3.0 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.12.1 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
github.com/coreos/go-oidc/v3 v3.5.0 // indirect
github.com/coreos/go-semver v0.3.0 // indirect
github.com/coreos/go-systemd/v22 v22.3.2 // indirect
Expand All @@ -153,7 +160,7 @@ require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dimchansky/utfbom v1.1.1 // indirect
github.com/docker/distribution v2.8.1+incompatible // indirect
github.com/docker/docker v20.10.21+incompatible // indirect
github.com/docker/docker v23.0.1+incompatible // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
Expand Down Expand Up @@ -198,7 +205,7 @@ require (
github.com/gofrs/uuid v4.2.0+incompatible // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt v3.2.2+incompatible // indirect
github.com/golang-jwt/jwt/v4 v4.4.2 // indirect
github.com/golang-jwt/jwt/v4 v4.4.3 // indirect
github.com/golang/glog v1.0.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/mock v1.6.0 // indirect
Expand Down Expand Up @@ -233,7 +240,7 @@ require (
github.com/huandu/xstrings v1.4.0 // indirect
github.com/imdario/mergo v0.3.13 // indirect
github.com/in-toto/in-toto-golang v0.3.4-0.20220709202702-fa494aaa0add // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/jedisct1/go-minisign v0.0.0-20211028175153-1c139d1cc84b // indirect
github.com/jhump/protoreflect v1.14.0 // indirect
Expand Down
Loading

0 comments on commit 4995cff

Please sign in to comment.