Skip to content

Commit

Permalink
chore: Update helm chart to 7.0.3 (#113)
Browse files Browse the repository at this point in the history 
* chore: update readme v1

* new helm chart version updates

* Update Taskfile.yml

* values changes

* patching

* patching for filestore url

* Update readme (#117)

* fix: matrix env typo (#110)

* chore: add awscli to utils container (#111)

* correct values for preprod

* move to config map as source of truth

* create config map with correct values in it and update size limit

---------

Co-authored-by: Andrew Moore <[email protected]>

* Update migrate-s3.yaml

* Update migrate-s3.yaml

* Update migrate-s3.yaml

* Update migrate-s3.yaml

* Update migrate-s3.yaml

* add labels

* chore: tidy the task file

* chore: add dev

* update all envs for new chart

* attempt poc

---------

Co-authored-by: Andrew Moore <[email protected]>
  • Loading branch information
@georgepstaylor @andrewmooreio
georgepstaylor and andrewmooreio authored Jan 8, 2025
1 parent 95926b7 commit 74bf71e
Show file tree
Hide file tree
Showing 63 changed files with 8,751 additions and 1,386 deletions.
15 changes: 7 additions & 8 deletions .github/workflows/migrate-s3.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,12 @@ jobs:
# remove all spaces and put one comma between prefixes
cleaned_prefixes=$(echo $prefixes | tr -s '[:space:]' ',' | sed 's/[,/]*$//')
# remove `contentstore.deleted/` from cleaned_prefixes with comma if in list
cleaned_prefixes=$(echo $cleaned_prefixes | sed 's/contentstore.deleted,//')
# remove `contentstore.deleted` from cleaned_prefixes if at the end of the list
cleaned_prefixes=$(echo $cleaned_prefixes | sed 's/contentstore.deleted//')
# None occurs when a file is at the root of the bucket or level 1 directory
# remove None from cleaned_prefixes with comma if in list
cleaned_prefixes=$(echo $cleaned_prefixes | sed 's/None,//')
# remove None from cleaned_prefixes if at the end of the list
cleaned_prefixes=$(echo $cleaned_prefixes | sed 's/None//')
DIRS=""
IFS=','
Expand All @@ -93,7 +95,4 @@ jobs:
--set srcBucket=$SRC_BUCKET \
--set "directories={${DIRS}}"
kubectl wait jobs -l name-prefix=migrate-s3 --for=condition=complete --timeout 10h
- name: Uninstall S3 Refresh chart
run: helm uninstall migrate-s3 --ignore-not-found
kubectl wait jobs -l name-prefix=migrate-s3 --for=condition=ready --timeout 10m
5 changes: 4 additions & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,11 @@
templated/
kustomize/base/charts/
kustomize/**/charts/
kustomize/**/output.yaml
kustomize/**/helm_rendered_spec.yaml
kustomize/**/kustomized_helm_rendered_spec.yaml
kustomize/base/resources.yaml
jobs/reindex/values-reindex-*.yaml
completed.txt
ids.json
alfresco-content-services/
s3-config/
125 changes: 36 additions & 89 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,102 +1,49 @@
### Purpose
# hmpps-delius-alfresco

These example instsructions explain how you would customise one particular dependant Helm chart and publish a newer version to GitHub pages. Instructions assume GitHub pages are already configured for your repo; see the reference section below
## Helm + Kustomize

### Start services
In order to start the alfresco-repository service, we need to make a valid license available in the namespace. A secret containing the license needs to be created:
```bash
ACS_NAMESPACE=hmpps-delius-alfrsco-poc
kubectl create secret generic alfresco-license \
--namespace $ACS_NAMESPACE \
--from-file /example/path/to/license/file.lic
```
This repository contains the Helm and Kustomize configuration for the Delius Alfresco deployment.

Next We will need to ensure all services are up and running.
Start k8s services by executing helm command _(Helm will complain if a random secret is not created)_
Rather than using/modifying the Helm chart directly, we use Kustomize to overlay the Helm chart with our custom configuration.
This allows us to keep the Helm chart as a dependency and only modify the configuration that we need to.

```bash
cd hmpps-delius-alfresco-poc/alfresco-content-services
export SECRET=$(openssl rand -base64 20)
export BUCKET_NAME=$(awk '{print substr($0, 0)}' <<< $(kubectl get secrets s3-bucket-output -o jsonpath='{.data.BUCKET_NAME}' | base64 -d))
helm install alfresco-content-services . --values=./values.yaml \
--set s3connector.config.bucketName=$BUCKET_NAME \
--set global.tracking.sharedsecret=$SECRET
```
### Usage

Note: we use taskfile to simplify the commands. You can install taskfile by running `brew install go-task/tap/go-task`.


To deploy the Delius Alfresco stack, you can use the following command:

### Check the chart file for dependent charts and pull the required version
For the purpose of this demo, we will select the following service from the `Chart.yaml` file
```yaml
- condition: alfresco-sync-service.enabled
name: alfresco-sync-service
repository: https://alfresco.github.io/alfresco-helm-charts/
version: 4.1.0
```
task helm_upgrade ENV=<dev|test|stage|preprod|prod> DEBUG=<true|false>
```
1. Delete existing tar file for the sync service. There will be an error pulling the chart otherwise
rm -rf charts/alfresco-sync-service-4.1.0.tgz

2. Pull a particular version of `alfresco-sync-service` chart
helm pull alfresco-sync-service --repo https://alfresco.github.io/alfresco-helm-charts --version 4.1.0 -d charts --untar
This will deploy the Delius Alfresco stack to the specified environment.
The `DEBUG` flag can be used to enable debug mode, which will enable helm verbose logging + output the templated,
rendered and kustomized manifests to the environment directory.

3. The above command will pull a tar file called `charts/alfresco-sync-service-4.1.0.tgz` and then untar it into a directory called `alfresco-sync-service`. Delete the tar file
rm -rf charts/alfresco-sync-service-4.1.0.tgz
```

### Modify charts

1. Change the chart version in the newly pulled chart. For example change is from `4.1.0` to `4.1.1`
2. Make your changes and then test them by upgrading Helm release
```
- export SECRET=$(awk '{print substr($0, 19)}' <<< $(kubectl get secrets alfresco-content-services-alfresco-repository-properties-secret -o jsonpath='{.data.alfresco-global\.properties}' | base64 -d))
- export BUCKET_NAME=$(awk '{print substr($0, 0)}' <<< $(kubectl get secrets s3-bucket-output -o jsonpath='{.data.BUCKET_NAME}' | base64 -d))
- helm upgrade alfresco-content-services . --values=./values.yaml --set s3connector.config.bucketName=$BUCKET_NAME --set global.tracking.sharedsecret=$SECRET
- NOTE: For the release upgrade, use the existing secret. You will otherwise have to restart pods consuming those secrets
```
4. Once satisfied with your changes, create a package and add it to the docs directory
- "helm package charts/alfresco-sync-service -d ../docs"
5. Create / update an index file in docs directory
- "helm repo index ../docs --url https://ministryofjustice.github.io/hmpps-delius-alfresco-poc"


### Update the lock file and commit changes
Locate the `Chart.yaml` file and modify the repository URL and version. It should now look like the code snippet below after the change:
```yaml
- condition: alfresco-sync-service.enabled
name: alfresco-sync-service
repository: https://ministryofjustice.github.io/hmpps-delius-alfresco-poc/
version: 4.1.1
```
### Configuration

1. Delete `charts/alfresco-sync-service` directory as it is no longer needed
2. Push your changes / docs directory to the feature branch
3. Update your GitHub pages settings so that the `source branch` is pointing to your feature branch
4. Update helm dependencies which will pull the updated charts and will update the lock file
- `helm dependency update .`
5. Push the lock file and charts dirctory to the feature branch and get merge approval
6. Merge into main branch
7. Update your GitHub pages settings so that the `source branch` is pointing to your main branch
8. Upgrade the helm release for the changes to be updated in kubernetes cluster
```
- export SECRET=$(awk '{print substr($0, 19)}' <<< $(kubectl get secrets alfresco-content-services-alfresco-repository-properties-secret -o jsonpath='{.data.alfresco-global\.properties}' | base64 -d))
- helm upgrade alfresco-content-services . --values=./values.yaml --set global.tracking.sharedsecret=$SECRET
- NOTE: For the release upgrade, use the existing secret. You will otherwise have to restart pods consuming those secrets
```
### Alternatively, pull a particular chart either directly from repository URL or by adding it in the local repo
1. Pull a chart with a particular version direcly from the GitHub pages
- `helm pull alfresco-sync-service --repo https://ministryofjustice.github.io/hmpps-delius-alfresco-poc/ --version 4.1.1 -d charts --untar`
2. Or add the updated chart in a local helm repo
```
helm repo add alfresco-sync-service https://ministryofjustice.github.io/hmpps-delius-alfresco-poc/
1. Helm values
The base helm values are stored in the `kustomize/base/values.yaml` file.
Each environment has its own values file, which is stored in the `kustomize/environments/<env>/values.yaml` file.
These values are combined when deploying the stack, with the environment values taking precedence.

helm search repo alfresco-sync-service
NAME CHART VERSION APP VERSION DESCRIPTION
alfresco-sync-service/alfresco-sync-service 4.1.1 3.9.0 Alfresco Sync Service
```
2. Kustomize
The kustomize overlays are stored in the `kustomize/environments/<env>` directory.
These overlays are applied to the Helm chart's resources to modify the configuration as needed for the environment.


### Secrets

A number of secrets are required to deploy the Delius Alfresco stack. Some of these are set by the cloud-platform-environments repository, while others are set manually.

### References to various docs that explain how to set up GitHub pages and how to publish Helm charts
- [The Chart Repository Guide](https://helm.sh/docs/topics/chart_repository/#github-pages-example)
- [Chart Releaser Action to Automate GitHub Page Charts ](https://helm.sh/docs/howto/chart_releaser_action/#github-actions-workflow)
- [Example on how to publish a chart on GitHub pages](https://github.com/technosophos/tscharts)
Table:
| Secret Name | Description | Set By | example/required keys |
| --- | --- | --- | --- |
| amazon-mq-broker-secret | The secret for the Amazon MQ broker | cloud-platform-environments | see [cloud-platform-environments](https://github.com/ministryofjustice/cloud-platform-environments/blob/7968f9c66f6914d33db35b68209c55b2dcb25d7d/namespaces/live.cloud-platform.service.justice.gov.uk/hmpps-delius-alfresco-stage/resources/amq.tf#L218) |
| alfresco-license | The Alfresco license file | manual | `<alfresco-license-file-name> : <base64-encoded-alfresco-license-file>`
| legacy-rds-instance | The RDS instance for the legacy Delius Alfresco stack | manual | `DATABASE_NAME: <database-name>, DATABASE_USERNAME: <database-username>, DATABASE_PASSWORD: <database-password>, RDS_INSTANCE_ADDRESS: <rds-instance-address>` |
| rds-instance-outpur | The RDS instance for the CP Delius Alfresco stack | cloud-platform-environments | see [cloud-platform-environments](https://github.com/ministryofjustice/cloud-platform-environments/blob/7968f9c66f6914d33db35b68209c55b2dcb25d7d/namespaces/live.cloud-platform.service.justice.gov.uk/hmpps-delius-alfresco-stage/resources/rds.tf#L35) |
| quay-registry-secret | The secret for the Quay registry | manual | `.dockerconfigjson: {"auths":{"quay.io":{"username":"<quay-username>","password":"<quay-password>","email":"<quay-email>","auth":"<base64-encoded-auth>"}}}` |
Loading

0 comments on commit 74bf71e

Please sign in to comment.