Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ACC-585] Verify vault secrets workflow #14

Merged
merged 76 commits into from
May 1, 2024
Merged
Show file tree
Hide file tree
Changes from 75 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
61e8559
First draft of a shared github action to verify vault secrets
Conor-TS Mar 1, 2024
a44d52c
JS-based action to verify vault secrets
Conor-TS Mar 6, 2024
194f950
move vault_token dep to secrets
Conor-TS Mar 27, 2024
2e65f00
Fix require
Conor-TS Mar 27, 2024
e14b6c6
Debugging
Conor-TS Apr 6, 2024
9084403
more debugging
Conor-TS Apr 6, 2024
f2ed18a
checking out action repo in parallel
Conor-TS Apr 7, 2024
7aa4b14
npm installing
Conor-TS Apr 7, 2024
d1db156
small fix
Conor-TS Apr 7, 2024
2c438d3
Progress - processing inputs
Conor-TS Apr 7, 2024
0d4c24c
add new inputs to action yaml
Conor-TS Apr 7, 2024
5de78c3
Trying to pull PR changes using Octokit
Conor-TS Apr 7, 2024
764e089
debugging
Conor-TS Apr 7, 2024
556c3b8
debugging
Conor-TS Apr 7, 2024
4afd272
trying to use better REST api dep
Conor-TS Apr 8, 2024
3c041d4
Trying to fix auth for github API
Conor-TS Apr 8, 2024
9cda272
fixes
Conor-TS Apr 8, 2024
0056b03
fixes
Conor-TS Apr 8, 2024
2b1ae1d
debugging
Conor-TS Apr 8, 2024
b786570
trying to fix regex
Conor-TS Apr 8, 2024
14a1390
debugging inputs
Conor-TS Apr 8, 2024
e7f3c86
debugging
Conor-TS Apr 8, 2024
a80626d
debugging
Conor-TS Apr 8, 2024
483d1d1
Fix inputs
Conor-TS Apr 8, 2024
c765d62
Incorporating updated approach to retrieve Vault tokens
Conor-TS Apr 26, 2024
2d03bbc
Try a different runner
Conor-TS Apr 26, 2024
6e3694e
syntax fix
Conor-TS Apr 26, 2024
2eb70cd
revert change to runner
Conor-TS Apr 26, 2024
aad14e0
trying to fix env var issue
Conor-TS Apr 26, 2024
599c533
try a different runner
Conor-TS Apr 26, 2024
3a119d0
debugging
Conor-TS Apr 26, 2024
77f8597
Trying to fix permissions issue
Conor-TS Apr 26, 2024
5b62096
debugging
Conor-TS Apr 26, 2024
6e41baf
debugging
Conor-TS Apr 26, 2024
d8b8184
debugging
Conor-TS Apr 26, 2024
f47f697
Progress, but still debugging
Conor-TS Apr 26, 2024
87b034d
debugging
Conor-TS Apr 26, 2024
3104a55
debugging
Conor-TS Apr 26, 2024
f458e0c
debugging
Conor-TS Apr 26, 2024
a6b5599
Use vault github action to retrieve keys
Conor-TS Apr 27, 2024
80a888b
Fixes
Conor-TS Apr 27, 2024
1b23465
debugging
Conor-TS Apr 27, 2024
919efb3
debugging
Conor-TS Apr 27, 2024
481151c
fix
Conor-TS Apr 27, 2024
8383337
dealing with outputs
Conor-TS Apr 27, 2024
a38aeb1
debugging
Conor-TS Apr 27, 2024
c024ad0
debugging
Conor-TS Apr 27, 2024
99ca7a4
debugging
Conor-TS Apr 27, 2024
5ba1e34
using artifacts to pass keys output
Conor-TS Apr 27, 2024
51aa981
fix keys pull step
Conor-TS Apr 27, 2024
25b60c7
adjust downloading of keys artifacts
Conor-TS Apr 27, 2024
4091063
Download all edge-env keys by wildcard
Conor-TS Apr 27, 2024
f23dca4
Trying to parse keys and feed to JS action
Conor-TS Apr 27, 2024
d4bc576
merging downloaded artifacts
Conor-TS Apr 27, 2024
ef7ad2a
fix keys path
Conor-TS Apr 27, 2024
bb601c1
Cleaning up - trying to fix output formatting
Conor-TS Apr 27, 2024
afb2b8f
Debugging
Conor-TS Apr 27, 2024
a91452a
Debugging
Conor-TS Apr 27, 2024
fa219cc
Debugging
Conor-TS Apr 27, 2024
ea43371
debug
Conor-TS Apr 27, 2024
dcc6885
debugging
Conor-TS Apr 27, 2024
f90fa8e
Actually fail on missing env vars
Conor-TS Apr 27, 2024
150aad8
Add logic to ignore specified environment variables
Conor-TS Apr 29, 2024
6754029
reformatting
Conor-TS Apr 29, 2024
46e30cf
tweaking
Conor-TS Apr 29, 2024
2d82ddd
tweaking
Conor-TS Apr 29, 2024
09ec9fe
Fix
Conor-TS Apr 29, 2024
e9d12ee
Reworking some JS, tests
Conor-TS Apr 29, 2024
e85c54d
small fix
Conor-TS Apr 29, 2024
ed4d2b9
add CI script
Conor-TS Apr 29, 2024
3a60fd7
Merge branch 'master' into ACC-465-verify-vault-secrets-workflow
Conor-TS Apr 29, 2024
130e409
set correct working directory for CI
Conor-TS Apr 29, 2024
4c03f92
Try a different approach to identifying environment variables
Conor-TS Apr 30, 2024
58319ad
Regex update to find fetch_env, fetch_env! and get_env
Conor-TS Apr 30, 2024
6905bba
change the branch for JS checkout to the master branch
Conor-TS May 1, 2024
7e319b2
formatting fix
Conor-TS May 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Run Jest Tests

on:
pull_request:
types: [opened, reopened, synchronize]
push:
branches:
- master

jobs:
test:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./.github/workflows/verify-vault-secrets

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 18

- name: Install dependencies
run: npm install

- name: Run Jest tests
run: npm test
112 changes: 112 additions & 0 deletions .github/workflows/verify-vault-secrets.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
name: Verify Vault Secrets Elixir

on:
workflow_call:
inputs:
service:
required: true
type: string
edges:
required: true
type: string
path_suffixes:
required: true
type: string
environments:
required: false
type: string
default: "['staging','demo','uat','audit1','ps','production']"
ignored_keys:
required: false
type: string
default: ""
vault_addr_prod:
required: false
type: string
default: "https://vault.prod.thescore.is"
vault_addr_non_prod:
required: false
type: string
default: "https://vault.non-prod.thescore.is"

jobs:
retrieve-vault-secret-keys:
runs-on: non-prod-scorebet-org-runner
permissions:
contents: read
id-token: write
strategy:
matrix:
env: ${{ fromJSON(inputs.environments) }}
edge: ${{ fromJSON(inputs.edges) }}
suffix: ${{ fromJSON(inputs.path_suffixes) }}
steps:
- name: Get Vault Keys
id: vault-keys
uses: hashicorp/vault-action@v3
with:
method: jwt
path: github-actions
url: ${{ matrix.env == 'production' && 'https://vault.prod.thescore.is' || 'https://vault.non-prod.thescore.is' }}
role: identity
exportToken: true
secrets: |
scorebet/subkeys/identity/${{ matrix.env }}/${{ matrix.edge }}/${{ matrix.suffix }} subkeys ;
- name: Set Output
id: vault-keys-output
shell: bash
run: |
echo ${{ toJson(steps.vault-keys.outputs.subkeys) }} >> ./${{ matrix.env }}-${{ matrix.edge }}-${{ matrix.suffix }}-keys.json
- name: Upload output artifact
id: vault-keys-artifact
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.env }}-${{ matrix.edge }}-${{ matrix.suffix }}-keys
path: ./${{ matrix.env }}-${{ matrix.edge }}-${{ matrix.suffix }}-keys.json

verify-vault-secrets:
runs-on: ubuntu-latest
needs: retrieve-vault-secret-keys
strategy:
matrix:
env: ${{ fromJson(inputs.environments) }}
edge : ${{ fromJson(inputs.edges) }}
svistoi marked this conversation as resolved.
Show resolved Hide resolved
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Download env-edge keys artifacts
uses: actions/download-artifact@v4
with:
path: keys
pattern: ${{ matrix.env }}-${{ matrix.edge }}-*-keys
merge-multiple: true

- name: Extract keys
id: extract-keys
shell: bash
run: |
keys=$(find ./keys -type f -name "*.json" -exec jq -r 'keys_unsorted[]' {} + | tr '\n' ',')
echo "keys='$keys'" >> $GITHUB_OUTPUT

- uses: actions/checkout@v4
with:
repository: scoremedia/devops-github-workflow
ref: master
path: shared-actions

- uses: actions/setup-node@v4
with:
node-version: 18

- name: npm install
shell: bash
run: |
cd ./shared-actions/.github/workflows/verify-vault-secrets && npm install

- uses: actions/github-script@v7
with:
script: |
const script = require('./shared-actions/.github/workflows/verify-vault-secrets/verifyVaultSecrets.js')
await script({github, context, core})
keys: ${{ fromJSON(steps.extract-keys.outputs.keys) }}
ignored_keys: ${{ inputs.ignored_keys }}
Loading
Loading