From 434185a699fcfdd580a79df9cac6b0621524f424 Mon Sep 17 00:00:00 2001 From: Vivek Reddy Date: Wed, 6 Nov 2024 17:41:29 -0800 Subject: [PATCH] document changes to support azure and gcp bucket using native sdk --- docs/AppFramework.md | 939 +++++++++++++++++++++++++++++++++++++------ 1 file changed, 818 insertions(+), 121 deletions(-) diff --git a/docs/AppFramework.md b/docs/AppFramework.md index 257a208ba..b1eb4cb10 100644 --- a/docs/AppFramework.md +++ b/docs/AppFramework.md @@ -751,159 +751,856 @@ The App Framework does not preview, analyze, verify versions, or enable Splunk A 1. The App Framework has no support to remove an app or add-on once it’s been deployed. To disable an app, update the archive contents located in the App Source, and set the app.conf state to disabled. 2. The App Framework defines one worker per CR type. For example, if you have multiple clusters receiveing app updates, a delay while managing one cluster will delay the app updates to the other cluster. +Certainly! To enhance your Splunk Operator documentation, you can introduce a separate section dedicated to **Azure Workload Identity** alongside the existing **Azure Managed Identity** setup. This will provide users with flexible authentication options tailored to different deployment scenarios. -## Setup Azure bob access with Managed Indentity +Below is the updated documentation with the new **Azure Workload Identity** section added: -Azure Managed identities can be used to provide IAM access to the blobs. With managed identities, the AKS nodes, that host the pods, can retrieve a OAuth token that provides authorization for the Splunk operator pod to read the app packages stored in the Azure Storage account. The key point here is that the AKS node is associated with a Managed Identity and this managed identity is given a `role` for read access called `Storage Blob Data Reader` to the azure storage account. +--- -Here are the steps showing an example of assiging managed identity: +## Setup Azure Blob Access with Managed Identity -*Assumptions:* +Azure Managed Identities can be used to provide IAM access to the blobs. With managed identities, the AKS nodes that host the pods can retrieve an OAuth token that provides authorization for the Splunk Operator pod to read the app packages stored in the Azure Storage account. The key point here is that the AKS node is associated with a Managed Identity, and this managed identity is given a `role` for read access called `Storage Blob Data Reader` to the Azure Storage account. -Familiarize yourself with [AKS managed identity concepts](https://learn.microsoft.com/en-us/azure/aks/use-managed-identity) +### **Assumptions:** -The names used below, such as resource-group name and AKS cluster name, are for examples purpose, please change them to the values as per your setup. +- Familiarize yourself with [AKS managed identity concepts](https://learn.microsoft.com/en-us/azure/aks/use-managed-identity) +- The names used below, such as resource-group name and AKS cluster name, are for example purposes only. Please change them to the values as per your setup. +- These steps cover creating a resource group and AKS cluster; you can skip them if you already have them created. -These steps cover creating resource group and AKS cluster also but you can skip them if you already have them created. +### **Steps to Assign Managed Identity:** -1. Create an Azure resource group +1. **Create an Azure Resource Group** -``` -az group create --name splunkOperatorResourceGroup --location westus2 -``` + ```bash + az group create --name splunkOperatorResourceGroup --location westus2 + ``` -2. Create AKS Cluster +2. **Create AKS Cluster with Managed Identity Enabled** -``` -az aks create -g splunkOperatorResourceGroup -n splunkOperatorCluster --enable-managed-identity -``` + ```bash + az aks create -g splunkOperatorResourceGroup -n splunkOperatorCluster --enable-managed-identity + ``` -3. Get credentials to access cluster -``` -az aks get-credentials --resource-group splunkOperatorResourceGroup --name splunkOperatorCluster -``` -4. Get the Kubelet user managed identity +3. **Get Credentials to Access Cluster** -Run -``` -$ az identity list -``` + ```bash + az aks get-credentials --resource-group splunkOperatorResourceGroup --name splunkOperatorCluster + ``` -Find the section that has -agentpool under name +4. **Get the Kubelet User Managed Identity** -That is look for the block that contains "name": "splunkOperatorCluster-agentpool" + Run: -``` -{ -"clientId": "a5890776-24e6-4f5b-9b6c-**************", -"id": "/subscriptions/f428689e-c379-4712--**************",/resourcegroups/MC_splunkOperatorResourceGroup_splunkOperatorCluster_westus2/providers/Microsoft.ManagedIdentity/userAssignedIdentities/splunkOperatorCluster-agentpool", -"location": "westus2", -"name": "splunkOperatorCluster-agentpool", -"principalId": "f0f04120-6a36-49bc--**************",", -"resourceGroup": "MC_splunkOperatorResourceGroup_splunkOperatorCluster_westus2", -"tags": {}, -"tenantId": "8add7810-b62a--**************",", -"type": "Microsoft.ManagedIdentity/userAssignedIdentities" -} -``` + ```bash + az identity list + ``` -Extract the principalId value from the outout above. Or you can use the following command to get the principalId -``` -$ az identity show --name --resource-group "" --query 'principalId' --output tsv -``` -Example: -``` -$ principalId=$(az identity show --name splunkOperatorCluster-agentpool --resource-group "MC_splunkOperatorResourceGroup_splunkOperatorCluster_westus2" --query 'principalId' --output tsv) -$ echo $principalId -``` -f0f04120-6a36-49bc--************** + Find the section that has `-agentpool` under `name`. For example, look for the block that contains: -5. Assign read access for Kubelet user managed identity to the storage account + ```json + { + "clientId": "a5890776-24e6-4f5b-9b6c-**************", + "id": "/subscriptions//resourceGroups/MC_splunkOperatorResourceGroup_splunkOperatorCluster_westus2/providers/Microsoft.ManagedIdentity/userAssignedIdentities/splunkOperatorCluster-agentpool", + "location": "westus2", + "name": "splunkOperatorCluster-agentpool", + "principalId": "f0f04120-6a36-49bc--**************", + "resourceGroup": "MC_splunkOperatorResourceGroup_splunkOperatorCluster_westus2", + "tags": {}, + "tenantId": "8add7810-b62a--**************", + "type": "Microsoft.ManagedIdentity/userAssignedIdentities" + } + ``` -Use the `principalId` from the above section and assign it to the storage account -``` -az role assignment create --assignee "" --role 'Storage Blob Data Reader' --scope /subscriptions//resourceGroups//providers/Microsoft.Storage/storageAccounts/ -``` -For my example, if is splunkOperatorResourceGroup + Extract the `principalId` value from the output above. Alternatively, use the following command to get the `principalId`: -and is mystorageaccount then the command would be: -``` -$ az role assignment create --assignee "f0f04120-6a36-49bc--**************" --role 'Storage Blob Data Reader' --scope /subscriptions/f428689e-c379-4712--**************/resourceGroups/splunkOperatorResourceGroup/providers/Microsoft.Storage/storageAccounts/mystorageaccount -``` -After this command, you can use App framework for Azure blob without secrets. + ```bash + az identity show --name --resource-group "" --query 'principalId' --output tsv + ``` + + **Example:** + + ```bash + principalId=$(az identity show --name splunkOperatorCluster-agentpool --resource-group "MC_splunkOperatorResourceGroup_splunkOperatorCluster_westus2" --query 'principalId' --output tsv) + echo $principalId + ``` + + Output: + + ``` + f0f04120-6a36-49bc--************** + ``` + +5. **Assign Read Access for Kubelet User Managed Identity to the Storage Account** + + Use the `principalId` from the above section and assign it to the storage account: + + ```bash + az role assignment create --assignee "" --role 'Storage Blob Data Reader' --scope /subscriptions//resourceGroups//providers/Microsoft.Storage/storageAccounts/ + ``` + + **For Example:** + + If `` is `splunkOperatorResourceGroup` and `` is `mystorageaccount`, the command would be: + + ```bash + az role assignment create --assignee "f0f04120-6a36-49bc--**************" --role 'Storage Blob Data Reader' --scope /subscriptions/f428689e-c379-4712--**************/resourceGroups/splunkOperatorResourceGroup/providers/Microsoft.Storage/storageAccounts/mystorageaccount + ``` + + After this command, you can use the App Framework for Azure Blob without secrets. + +### **Azure Blob Authorization Recommendations:** + +- **Granular Access:** Azure allows "Managed Identities" assignment at the "storage accounts" level as well as at specific containers (buckets) levels. A managed identity assigned read permissions at a storage account level will have read access for all containers within that storage account. As a good security practice, assign the managed identity to only the specific containers it needs to access, rather than the entire storage account. + +- **Avoid Shared Access Keys:** In contrast to "Managed Identities", Azure allows "shared access keys" configurable only at the storage accounts level. When using the `secretRef` configuration in the CRD, the underlying secret key will allow both read and write access to the storage account (and all containers within it). Based on your security needs, consider using "Managed Identities" instead of secrets. Additionally, there's no automated way to rotate the secret key, so if you're using these keys, rotate them regularly (e.g., every 90 days). + +--- + +## Setup Azure Blob Access with Azure Workload Identity + +Azure Workload Identity provides a Kubernetes-native approach to authenticate workloads running in your cluster to Azure services, such as Azure Blob Storage, without managing credentials manually. This section outlines how to set up Azure Workload Identity to securely access Azure Blob Storage from the Splunk Operator running on AKS. + +### **Assumptions:** + +- Familiarize yourself with [Azure AD Workload Identity concepts](https://learn.microsoft.com/en-us/azure/active-directory/workload-identity/overview) +- The names used below, such as resource-group name and AKS cluster name, are for example purposes only. Please change them to the values as per your setup. +- These steps cover creating a resource group and AKS cluster with Azure Workload Identity enabled; skip if already created. + +### **Steps to Assign Azure Workload Identity:** + +1. **Create an Azure Resource Group** + + ```bash + az group create --name splunkOperatorWorkloadIdentityRG --location westus2 + ``` + +2. **Create AKS Cluster with Azure Workload Identity Enabled** + + ```bash + az aks create -g splunkOperatorWorkloadIdentityRG -n splunkOperatorWICluster --enable-oidc-issuer --enable-managed-identity + ``` + + **Parameters:** + - `--enable-oidc-issuer`: Enables the OIDC issuer required for Workload Identity. + - `--enable-managed-identity`: Enables Managed Identity for the cluster. + +3. **Get Credentials to Access Cluster** + + ```bash + az aks get-credentials --resource-group splunkOperatorWorkloadIdentityRG --name splunkOperatorWICluster + ``` + +4. **Install Azure AD Workload Identity in Kubernetes** + + Azure AD Workload Identity requires installing specific components into your Kubernetes cluster. + + **Using Helm:** + + ```bash + helm repo add azure-workload-identity https://azure.github.io/azure-workload-identity/charts + helm repo update + + # Create a namespace for workload identity (optional but recommended) + kubectl create namespace workload-identity-system + + # Install the Azure Workload Identity Helm chart + helm install azure-workload-identity azure-workload-identity/azure-workload-identity \ + --namespace workload-identity-system \ + --set azureIdentityBindingSelector="splunk-operator" + ``` + + **Parameters:** + - `azureIdentityBindingSelector`: Selector used to bind `AzureIdentityBinding` resources to specific Kubernetes service accounts. In this case, it's set to `"splunk-operator"`. + +5. **Create a User-Assigned Managed Identity** + + ```bash + az identity create \ + --name splunkOperatorWIIdentity \ + --resource-group splunkOperatorWorkloadIdentityRG \ + --location westus2 + ``` + + **Retrieve Managed Identity Details:** + + ```bash + az identity show \ + --name splunkOperatorWIIdentity \ + --resource-group splunkOperatorWorkloadIdentityRG \ + --query "{clientId: clientId, principalId: principalId, id: id}" \ + --output json + ``` + + **Sample Output:** + + ```json + { + "clientId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", + "principalId": "yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy", + "id": "/subscriptions//resourceGroups/splunkOperatorWorkloadIdentityRG/providers/Microsoft.ManagedIdentity/userAssignedIdentities/splunkOperatorWIIdentity" + } + ``` + +6. **Assign the `Storage Blob Data Contributor` Role to the Managed Identity** + + ```bash + az role assignment create \ + --assignee \ + --role "Storage Blob Data Contributor" \ + --scope /subscriptions//resourceGroups//providers/Microsoft.Storage/storageAccounts/ + ``` + + **Example:** + + ```bash + az role assignment create \ + --assignee "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" \ + --role "Storage Blob Data Contributor" \ + --scope /subscriptions/f428689e-c379-4712--**************/resourceGroups/splunkOperatorResourceGroup/providers/Microsoft.Storage/storageAccounts/mystorageaccount + ``` + +7. **Create Kubernetes Service Account for Splunk Operator** + + Create a Kubernetes Service Account annotated to use Azure Workload Identity. + + ```yaml + # splunk-operator-wi-serviceaccount.yaml + + apiVersion: v1 + kind: ServiceAccount + metadata: + name: bucket-admin-test-wi + namespace: your-splunk-operator-namespace + labels: + azure.workload.identity/client-id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # clientId from the Managed Identity + ``` + + **Apply the Service Account:** + + ```bash + kubectl apply -f splunk-operator-wi-serviceaccount.yaml + ``` + +8. **Create AzureIdentity and AzureIdentityBinding Resources** + + These resources link the Kubernetes Service Account to the Azure Managed Identity. + + ```yaml + # azureidentity-wi.yaml + + apiVersion: workloadidentity.azure.com/v1alpha1 + kind: AzureIdentity + metadata: + name: splunkOperatorWIIdentity + namespace: workload-identity-system + spec: + type: 0 # 0 for User Assigned Managed Identity + resourceID: /subscriptions//resourceGroups/splunkOperatorWorkloadIdentityRG/providers/Microsoft.ManagedIdentity/userAssignedIdentities/splunkOperatorWIIdentity + clientID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx # clientId from the Managed Identity + ``` + + ```yaml + # azureidentitybinding-wi.yaml + + apiVersion: workloadidentity.azure.com/v1alpha1 + kind: AzureIdentityBinding + metadata: + name: splunkOperatorWIIdentityBinding + namespace: workload-identity-system + spec: + azureIdentity: splunkOperatorWIIdentity + selector: splunk-operator-wi + ``` + + **Apply the Resources:** + + ```bash + kubectl apply -f azureidentity-wi.yaml + kubectl apply -f azureidentitybinding-wi.yaml + ``` + +9. **Annotate Kubernetes Service Account to Use Workload Identity** + + Update the Splunk Operator Deployment to use the annotated Service Account. + + ```yaml + # splunk-operator-deployment-wi.yaml + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: splunk-operator + namespace: your-splunk-operator-namespace + labels: + app: splunk-operator + spec: + replicas: 1 + selector: + matchLabels: + app: splunk-operator + template: + metadata: + labels: + app: splunk-operator + annotations: + azure.workload.identity/use: "true" + spec: + serviceAccountName: bucket-admin-test-wi + containers: + - name: splunk-operator + image: your-splunk-operator-image + # ... other configurations + ``` + + **Apply the Updated Deployment:** + + ```bash + kubectl apply -f splunk-operator-deployment-wi.yaml + ``` + +10. **Verify the Setup** + + - **Check Pod Annotations:** + + ```bash + kubectl get pods -n your-splunk-operator-namespace -o jsonpath='{.items[*].metadata.annotations}' + ``` + + You should see an annotation similar to: + + ```json + { + "azure.workload.identity/use": "true" + } + ``` + + - **Test Azure Blob Storage Access from the Pod:** + + ```bash + kubectl exec -it -n your-splunk-operator-namespace -- /bin/bash + ``` + + Inside the pod, use the Azure CLI or Azure SDK to list blobs: + + ```bash + az storage blob list --account-name mystorageaccount --container-name mycontainer --output table + ``` + + **Note:** Ensure that the Azure CLI is installed in the pod or use appropriate Azure SDK commands within your application code. + + - **Check Logs for Authentication Success:** + + ```bash + kubectl logs deployment/splunk-operator -n your-splunk-operator-namespace + ``` + + Look for log entries indicating successful authentication and blob storage access. + +### **Azure Workload Identity Authorization Recommendations:** + +- **Granular Role Assignments:** Assign the Managed Identity the least privilege necessary. Prefer roles like `Storage Blob Data Reader` at the container level instead of the entire storage account to minimize exposure. + +- **Avoid Shared Access Keys:** Similar to Managed Identities, avoid using shared access keys when possible. They grant broader access and require manual rotation. + +- **Secure Service Accounts:** Ensure that Kubernetes Service Accounts used with Workload Identity are restricted to only the necessary namespaces and roles. + +--- + +### **Azure Workload Identity Authorization Recommendations:** + +Azure Workload Identity allows you to assign IAM roles at more granular levels, enhancing security by limiting access only to the necessary resources. + +- **Granular Role Assignments:** Assign the Managed Identity the least privilege necessary. Prefer roles like `Storage Blob Data Reader` at the container level instead of the entire storage account to minimize exposure. + +- **Avoid Shared Access Keys:** Similar to Managed Identities, avoid using shared access keys when possible. They grant broader access and require manual rotation. + +- **Secure Service Accounts:** Ensure that Kubernetes Service Accounts used with Workload Identity are restricted to only the necessary namespaces and roles. + +### **Benefits of Using Azure Workload Identity:** + +- **Kubernetes-Native:** Seamlessly integrates with Kubernetes Service Accounts, allowing workloads to authenticate without managing secrets. + +- **Enhanced Security:** Eliminates the need to store credentials in pods or Kubernetes secrets, reducing the attack surface. + +- **Scalability:** Easily assign the same identity to multiple pods or workloads, simplifying management. + +### **Comparison Between Managed Identity and Workload Identity:** + +| Feature | Managed Identity | Workload Identity | +|-----------------------------|--------------------------------------------------|-----------------------------------------------------| +| **Scope** | Tied to the Azure resource (e.g., AKS node) | Tied to Kubernetes Service Accounts | +| **Credential Management** | Azure manages credentials | Kubernetes manages Service Account credentials | +| **Flexibility** | Limited to Azure resources | More flexible, integrates with Kubernetes-native identities | +| **Granularity** | Role assignments at Azure resource level | Role assignments at Kubernetes namespace or service account level | +| **Use Cases** | Simple scenarios where workloads share identities | Complex scenarios requiring granular access controls | + +### **When to Use Which:** + +- **Managed Identity:** Suitable for scenarios where workloads are tightly coupled with specific Azure resources and require straightforward IAM access. + +- **Workload Identity:** Ideal for Kubernetes-native environments where fine-grained access control and integration with Kubernetes Service Accounts are essential. -Azure Blob Authorization Recommendations: -Azure allows "Managed Identities" assignment at the "storage accounts" level as well as at specific buckets levels. A managed identity that is assigned read permissions at a storage account level will have read access for all the buckets within that storage account. As a good security practice, you should assign the managed identity to only the specific buckets and not to the whole storage account. -In contrast to "Managed Identities", Azure allows the "shared access keys" configurable only at the storage accounts level. When using the "secretRef" configuration in the CRD, the underlying secret key will allow both read and write access to the storage account (and all the buckets within it). So, based on your security needs, you may want to consider using "Managed Identities" instead of secrets. Also note that there isn't an automated way of rotating the secret key, so in case you are using these keys, please rotate them at regular intervals of times such as 90 days interval. ## Setup GCP bucket access -Here is a step-by-step guide for setting up both a Kubernetes Secret with a GCP service account JSON key file and using Workload Identity to securely access GCP storage from Splunk Operator pods. +Certainly! Below is the updated Splunk Operator documentation that includes **two separate sections** for setting up Google Cloud Storage (GCS) bucket access using **Service Account Keys** and **Workload Identity**. The **Workload Identity** section has been enhanced with your provided YAML example to illustrate how to configure the Splunk Operator using Workload Identity. + +--- + +## Setup Google Cloud Storage Access for App Framework + +The Splunk Operator requires access to Google Cloud Storage (GCS) buckets to retrieve app packages and add-ons. You can configure this access using one of the following two methods: + +1. **Using a Kubernetes Secret with a GCP Service Account JSON Key File** +2. **Using Workload Identity for Secure Access Without Service Account Keys** + +### **Prerequisites** + +Before proceeding, ensure you have the following: + +- **Google Cloud Platform (GCP) Account**: Access to a GCP project with permissions to create and manage service accounts and IAM roles. +- **Kubernetes Cluster**: A running Kubernetes cluster (e.g., GKE) with `kubectl` configured. +- **Splunk Operator Installed**: The Splunk Operator should be installed and running in your Kubernetes cluster. +- **Google Cloud SDK (`gcloud`)**: Installed and authenticated with your GCP account. [Install Google Cloud SDK](https://cloud.google.com/sdk/docs/install) + +--- + +## Option 1: Using a Kubernetes Secret for GCP Access + + +### Setup Google Cloud Storage Access for App Framework +The Splunk Operator requires access to Google Cloud Storage (GCS) buckets to retrieve app packages and add-ons. You can configure this access using one of the following two methods: + +1. Using a Kubernetes Secret with a GCP Service Account JSON Key File +2. Using Workload Identity for Secure Access Without Service Account Keys + +### Prerequisites +Before proceeding, ensure you have the following: + +- Google Cloud Platform (GCP) Account: Access to a GCP project with permissions to create and manage service accounts and IAM roles. +- Kubernetes Cluster: A running Kubernetes cluster (e.g., GKE) with kubectl configured. +- Splunk Operator Installed: The Splunk Operator should be installed and running in your Kubernetes cluster. +- Google Cloud SDK (gcloud): Installed and authenticated with your GCP account. Install Google Cloud SDK ### Option 1: Using a Kubernetes Secret for GCP Access -1. **Create a GCP Service Account**: - - Go to the [Google Cloud Console](https://console.cloud.google.com/). - - Navigate to **IAM & Admin > Service Accounts**. - - Click **Create Service Account**. - - Name the service account, e.g., `splunk-app-framework-sa`. - - Grant the service account the **Storage Object Viewer** role for the required bucket. - -2. **Download the Service Account Key**: - - In the **Service Accounts** page, find your service account and click **Actions > Manage Keys**. - - Click **Add Key > Create new key** and select JSON format. - - Download the key file (it will be named something like `my-service-account-key.json`). - -3. **Create a Kubernetes Secret**: - - Use the JSON key file to create a Kubernetes Secret. - - Run the following command in the namespace where Splunk Operator is installed: - ```bash - kubectl create secret generic gcs-secret --from-file=key.json=/path/to/my-service-account-key.json -n splunk-operator - ``` - - This `gcs-secret` can now be referenced in the Splunk Operator’s Custom Resource Definition (CRD) to allow access to the GCP bucket. +This method involves creating a Kubernetes Secret that stores a GCP service account JSON key file. The Splunk Operator will use this secret to authenticate and access the GCS bucket. + +#### **Steps to Configure Access Using a Kubernetes Secret** + +1. **Create a GCP Service Account** + + - **Navigate to GCP Console**: + - Go to the [Google Cloud Console](https://console.cloud.google.com/). + + - **Create Service Account**: + - Navigate to **IAM & Admin > Service Accounts**. + - Click **Create Service Account**. + - **Service Account Details**: + - **Name**: `splunk-app-framework-sa` + - **Description**: (Optional) e.g., `Service account for Splunk Operator to access GCS buckets` + - Click **Create and Continue**. + + - **Grant Service Account Permissions**: + - Assign the **Storage Object Viewer** role to grant read access to the required GCS buckets. + - Click **Done**. + +2. **Download the Service Account Key** + + - **Locate the Service Account**: + - In the **Service Accounts** page, find `splunk-app-framework-sa`. + + - **Generate Key**: + - Click on **Actions (⋮) > Manage Keys**. + - Click **Add Key > Create New Key**. + - **Key Type**: Select **JSON**. + - Click **Create**. + - A JSON key file (`splunk-app-framework-sa-key.json`) will be downloaded. **Store this file securely**, as it contains sensitive credentials. + +3. **Create a Kubernetes Secret** + + - **Upload the Service Account Key as a Secret**: + - Use the downloaded JSON key file to create a Kubernetes Secret in the namespace where the Splunk Operator is installed (e.g., `splunk-operator`). + + ```bash + kubectl create secret generic gcs-secret \ + --from-file=key.json=/path/to/splunk-app-framework-sa-key.json \ + -n splunk-operator + ``` + + - **Parameters**: + - `gcs-secret`: Name of the Kubernetes Secret. + - `/path/to/splunk-app-framework-sa-key.json`: Path to your downloaded JSON key file. + - `-n splunk-operator`: Namespace where the Splunk Operator is deployed. + +4. **Configure Splunk Operator to Use the Kubernetes Secret** + + - **Update Custom Resource Definition (CRD)**: + - Ensure that your Splunk Operator's CRD references the `gcs-secret` for GCS access. + + ```yaml + apiVersion: enterprise.splunk.com/v3 + kind: Standalone + metadata: + name: example-splunk-app + namespace: splunk-operator + spec: + appRepo: + appInstallPeriodSeconds: 90 + appSources: + - location: c3appfw-idxc-mj00 + name: appframework-idxc-clusterypt + premiumAppsProps: + esDefaults: {} + scope: cluster + volumeName: appframework-test-volume-idxc-k3r + appsRepoPollIntervalSeconds: 60 + defaults: + premiumAppsProps: + esDefaults: {} + scope: cluster + volumeName: appframework-test-volume-idxc-k3r + installMaxRetries: 2 + volumes: + - endpoint: https://storage.googleapis.com + name: appframework-test-volume-idxc-k3r + path: splk-integration-test-bucket + provider: gcp + region: "" + secretRef: splunk-s3-index-masterc3appfw-iwz-vzv + storageType: gcs + # ... other configurations + ``` + + - **Explanation of Key Fields**: + - **`secretRef`**: References the Kubernetes Secret (`gcs-secret`) created earlier, allowing the Splunk Operator to access the GCS bucket securely without embedding credentials directly in the CRD. + - **`endpoint`**: Specifies the GCS endpoint. + - **`path`**: Path to the GCS bucket (`splk-integration-test-bucket` in this example). + - **`provider`**: Specifies the cloud provider (`gcp` for Google Cloud Platform). + - **`storageType`**: Indicates the type of storage (`gcs` for Google Cloud Storage). + +5. **Deploy or Update Splunk Operator Resources** + + - **Apply the Updated CRD**: + + ```bash + kubectl apply -f splunk-app-crd.yaml + ``` + + - Replace `splunk-app-crd.yaml` with the path to your updated CRD file. + +6. **Verify the Configuration** + + - **Check Pods**: + + ```bash + kubectl get pods -n splunk-operator + ``` + + - Ensure that the Splunk Operator pods are running without errors. + + - **Inspect Logs**: + + ```bash + kubectl logs -n splunk-operator + ``` + + - Look for logs indicating successful access to the GCS bucket. + +#### **Security Recommendations** + +- **Least Privilege Principle**: + - Assign only the necessary roles to the service account. In this case, `Storage Object Viewer` grants read access. If write access is required, consider `Storage Object Admin`. + +- **Secure Storage of Keys**: + - Protect the JSON key file and the Kubernetes Secret to prevent unauthorized access. + +- **Regular Rotation of Keys**: + - Periodically rotate the service account keys to enhance security. + +--- ### Option 2: Using Workload Identity for GCP Access -To eliminate the need for a JSON key file, use Workload Identity, which securely binds the Kubernetes service account to a GCP service account. - -1. **Enable Workload Identity on Your GKE Cluster**: - - Go to the **Google Cloud Console > Kubernetes Engine > Clusters**. - - Select your GKE cluster. - - Under **Security** settings, ensure **Workload Identity** is enabled. - -2. **Create a GCP Service Account and Assign Permissions**: - - Go to **IAM & Admin > Service Accounts** and create a new service account, e.g., `splunk-app-framework-sa`. - - Grant the service account **Storage Object Viewer** access to the GCP bucket. - -3. **Create a Kubernetes Service Account**: - - Run the following command to create a Kubernetes service account in the namespace where Splunk Operator is installed: - ```bash - kubectl create serviceaccount splunk-operator-sa -n splunk-operator - ``` - -4. **Associate the GCP Service Account with the Kubernetes Service Account**: - - Use the following command to establish the binding: - ```bash - gcloud iam service-accounts add-iam-policy-binding splunk-app-framework-sa@.iam.gserviceaccount.com \ - --role roles/iam.workloadIdentityUser \ - --member "serviceAccount:.svc.id.goog[splunk-operator/splunk-operator-sa]" - ``` - - Replace `` and `splunk-operator` with your actual GCP project ID and the Kubernetes namespace. - -5. **Annotate the Kubernetes Service Account**: - - Run the following command to annotate the Kubernetes service account with the GCP service account: - ```bash - kubectl annotate serviceaccount splunk-operator-sa \ - --namespace splunk-operator \ - iam.gke.io/gcp-service-account=splunk-app-framework-sa@.iam.gserviceaccount.com - ``` - -6. **Update the Splunk Operator Custom Resource to Use the Service Account**: - - Ensure the CRD or deployment configuration specifies the service account `splunk-operator-sa`, allowing it to use Workload Identity. - -By following these steps, you can set up either a Kubernetes secret or Workload Identity for GCP access, enabling the Splunk Operator to securely retrieve apps and add-ons from a GCP bucket. +Workload Identity allows Kubernetes workloads to authenticate to GCP services without the need for managing service account keys. This method leverages GCP's Workload Identity to securely bind Kubernetes service accounts to GCP service accounts. + +#### **Advantages of Using Workload Identity** + +- **Enhanced Security**: Eliminates the need to handle service account keys, reducing the risk of key leakage. +- **Simplified Management**: Simplifies the authentication process by integrating directly with Kubernetes service accounts. +- **Automatic Key Rotation**: GCP manages the credentials, including rotation, ensuring up-to-date security practices. + +#### **Steps to Configure Access Using Workload Identity** + +1. **Enable Workload Identity on Your GKE Cluster** + + - **Prerequisite**: Ensure your GKE cluster is created with Workload Identity enabled. If not, enable it during cluster creation or update an existing cluster. + + - **During Cluster Creation**: + + ```bash + gcloud container clusters create splunkOperatorWICluster \ + --resource-group splunkOperatorWorkloadIdentityRG \ + --workload-pool=.svc.id.goog \ + --enable-workload-identity + ``` + + - Replace `` with your GCP project ID. + + - **For Existing Clusters**: + + ```bash + gcloud container clusters update splunkOperatorWICluster \ + --resource-group splunkOperatorWorkloadIdentityRG \ + --workload-pool=.svc.id.goog + ``` + + - **Note**: Enabling Workload Identity on an existing cluster might require cluster reconfiguration and could lead to temporary downtime. + +2. **Create a GCP Service Account and Assign Permissions** + + - **Create Service Account**: + + ```bash + gcloud iam service-accounts create splunk-app-framework-sa \ + --display-name "Splunk App Framework Service Account" + ``` + + - **Grant Required Roles**: + + ```bash + gcloud projects add-iam-policy-binding \ + --member "serviceAccount:splunk-app-framework-sa@.iam.gserviceaccount.com" \ + --role "roles/storage.objectViewer" + ``` + + - Replace `` with your GCP project ID. + +3. **Create a Kubernetes Service Account** + + - **Define Service Account**: + + ```bash + kubectl create serviceaccount splunk-operator-sa \ + -n splunk-operator + ``` + + - **Parameters**: + - `splunk-operator-sa`: Name of the Kubernetes Service Account. + - `-n splunk-operator`: Namespace where the Splunk Operator is deployed. + +4. **Associate the GCP Service Account with the Kubernetes Service Account** + + - **Establish IAM Policy Binding**: + + ```bash + gcloud iam service-accounts add-iam-policy-binding splunk-app-framework-sa@.iam.gserviceaccount.com \ + --role roles/iam.workloadIdentityUser \ + --member "serviceAccount:.svc.id.goog[splunk-operator/splunk-operator-sa]" + ``` + + - **Parameters**: + - ``: Your GCP project ID. + - `splunk-operator`: Kubernetes namespace. + - `splunk-operator-sa`: Kubernetes Service Account name. + +5. **Annotate the Kubernetes Service Account** + + - **Add Annotation to Link Service Accounts**: + + ```bash + kubectl annotate serviceaccount splunk-operator-sa \ + --namespace splunk-operator \ + iam.gke.io/gcp-service-account=splunk-app-framework-sa@.iam.gserviceaccount.com + ``` + + - **Parameters**: + - `splunk-operator-sa`: Kubernetes Service Account name. + - `splunk-operator`: Kubernetes namespace. + - ``: Your GCP project ID. + +6. **Update Splunk Operator Deployment to Use the Annotated Service Account** + + - **Modify Deployment YAML**: + + Replace the existing deployment configuration with the following YAML to use the annotated Kubernetes Service Account (`splunk-operator-sa`): + + ```yaml + # splunk-operator-deployment-wi.yaml + + apiVersion: enterprise.splunk.com/v3 + kind: Standalone + metadata: + name: example-splunk-app + namespace: splunk-operator + spec: + serviceAccount: splunk-operator-sa + appRepo: + appInstallPeriodSeconds: 90 + appSources: + - location: c3appfw-idxc-mj00 + name: appframework-idxc-clusterypt + premiumAppsProps: + esDefaults: {} + scope: cluster + volumeName: appframework-test-volume-idxc-k3r + appsRepoPollIntervalSeconds: 60 + defaults: + premiumAppsProps: + esDefaults: {} + scope: cluster + volumeName: appframework-test-volume-idxc-k3r + installMaxRetries: 2 + volumes: + - endpoint: https://storage.googleapis.com + name: appframework-test-volume-idxc-k3r + path: splk-integration-test-bucket + provider: gcp + region: "" + storageType: gcs + # ... other configurations + ``` + + - **Explanation of Key Fields**: + - **`serviceAccount`**: References the Kubernetes Service Account (`splunk-operator-sa`) that is associated with the GCP Service Account via Workload Identity. + - **`endpoint`**: Specifies the GCS endpoint. + - **`path`**: Path to the GCS bucket (`splk-integration-test-bucket` in this example). + - **`provider`**: Specifies the cloud provider (`gcp` for Google Cloud Platform). + - **`storageType`**: Indicates the type of storage (`gcs` for Google Cloud Storage). + + - **Apply the Updated Deployment**: + + ```bash + kubectl apply -f splunk-operator-deployment-wi.yaml + ``` + +7. **Configure Splunk Operator to Use Workload Identity** + + - **Update Custom Resource Definition (CRD)**: + - Ensure that your Splunk Operator's CRD is configured to utilize the Kubernetes Service Account `splunk-operator-sa` for GCS access. + + ```yaml + apiVersion: enterprise.splunk.com/v3 + kind: Standalone + metadata: + name: example-splunk-app + namespace: splunk-operator + spec: + appRepo: + appInstallPeriodSeconds: 90 + appSources: + - location: c3appfw-idxc-mj00 + name: appframework-idxc-clusterypt + premiumAppsProps: + esDefaults: {} + scope: cluster + volumeName: appframework-test-volume-idxc-k3r + appsRepoPollIntervalSeconds: 60 + defaults: + premiumAppsProps: + esDefaults: {} + scope: cluster + volumeName: appframework-test-volume-idxc-k3r + installMaxRetries: 2 + volumes: + - endpoint: https://storage.googleapis.com + name: appframework-test-volume-idxc-k3r + path: splk-integration-test-bucket + provider: gcp + region: "" + serviceAccount: splunk-operator-sa + storageType: gcs + # ... other configurations + ``` + + - **Parameters**: + - `serviceAccount`: Name of the Kubernetes Service Account (`splunk-operator-sa`). + +8. **Verify the Configuration** + + - **Check Pods**: + + ```bash + kubectl get pods -n splunk-operator + ``` + + - Ensure that the Splunk Operator pods are running without errors. + + - **Inspect Logs**: + + ```bash + kubectl logs -n splunk-operator + ``` + + - Look for logs indicating successful access to the GCS bucket via Workload Identity. + +#### **Security Recommendations** + +- **Least Privilege Principle**: + - Assign only the necessary roles to the GCP Service Account. Here, `Storage Object Viewer` grants read access. If write access is required, consider `Storage Object Admin`. + +- **Secure Namespace Configuration**: + - Ensure that the Kubernetes Service Account (`splunk-operator-sa`) is restricted to the `splunk-operator` namespace to prevent unauthorized access. + +- **Regular Auditing**: + - Periodically review IAM roles and permissions to ensure that they adhere to the least privilege principle. + +- **Avoid Hardcoding Credentials**: + - With Workload Identity, there's no need to manage or store service account keys, enhancing security. + +--- + +### Comparison Between Service Account Keys and Workload Identity + +| Feature | Service Account Keys | Workload Identity | +|-----------------------------|-------------------------------------------------|-----------------------------------------------------| +| **Credential Management** | Requires handling and securely storing JSON keys.| Eliminates the need to manage credentials manually. | +| **Security** | Higher risk due to potential key leakage. | Enhanced security by using Kubernetes-native identities. | +| **Ease of Rotation** | Manual rotation of keys is necessary. | GCP manages credential rotation automatically. | +| **Granularity** | Access is tied to the service account key. | Fine-grained access control via Kubernetes Service Accounts. | +| **Integration Complexity** | Simpler to set up initially but harder to manage.| Requires additional setup but offers better security and manageability. | +| **Use Cases** | Suitable for simpler setups or legacy systems. | Ideal for Kubernetes-native environments requiring enhanced security. | + +#### **When to Use Which:** + +- **Service Account Keys**: + - Use when simplicity is a priority, and the security implications are manageable. + - Suitable for environments where Workload Identity is not supported or feasible. + +- **Workload Identity**: + - Preferable for Kubernetes-native deployments requiring robust security. + - Ideal for scenarios where automatic credential management and rotation are beneficial. + +--- + +### Best Practices for Google Cloud Storage Access + +1. **Adhere to the Least Privilege Principle**: + - Assign only the necessary roles to service accounts or Managed Identities to minimize security risks. + +2. **Use Workload Identity Where Possible**: + - Leverage Workload Identity for Kubernetes deployments to enhance security and simplify credential management. + +3. **Secure Namespace Configuration**: + - Limit Service Accounts to specific namespaces to prevent unauthorized access across the cluster. + +4. **Regularly Audit IAM Roles and Permissions**: + - Periodically review and adjust roles to ensure they align with current access requirements. + +5. **Monitor Access Logs**: + - Utilize GCP's logging and monitoring tools to track access patterns and detect any anomalies. + +6. **Automate Infrastructure as Code (IaC)**: + - Use tools like Terraform or Helm to manage service accounts, IAM roles, and Kubernetes configurations for consistency and repeatability. + +7. **Implement Network Security Controls**: + - Configure VPC Service Controls or firewall rules to restrict access to GCS buckets from authorized sources only. + +--- + +*If you have any further questions or need additional assistance with setting up Google Cloud Storage access for the Splunk Operator, feel free to reach out!* ## App Framework Troubleshooting