Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SECURESIGN-1476 | Add the Redis backfill job to Ansible collection #101

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

SECURESIGN-1476 | Add the Redis backfill job to Ansible collection #101

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e67ba38
add-backfill-redis-job
JasonPowr Nov 4, 2024
262b902
update molecule test
JasonPowr Nov 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions molecule/user_provided/vars/vars.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,11 @@ tas_single_node_rekor_redis:
host: "{{ ansible_default_ipv4.address }}"
port: 6379
password: password
backfill_redis:
JasonPowr marked this conversation as resolved.
Show resolved Hide resolved
enabled: true
schedule:
minute: "0"
hour: "0"
day: "*"
month: "*"
weekday: "*"
20 changes: 19 additions & 1 deletion roles/tas_single_node/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Deploy the [RHTAS](https://docs.redhat.com/en/documentation/red_hat_trusted_arti
|Option|Description|Type|Default|
|---|---|---|---|
| tas_single_node_podman_network | Name of the Podman network for containers to use. | str | `rhtas` |
| tas_single_node_rekor_redis | Details on the Redis connection for Rekor. You can set this to a custom Redis instance. | dict of 'tas_single_node_rekor_redis' options | `{'database_deploy': True, 'redis': {'host': 'rekor-redis-pod', 'port': 6379, 'password': 'password'}}` |
| tas_single_node_rekor_redis | Details on the Redis connection for Rekor. You can set this to a custom Redis instance. | dict of 'tas_single_node_rekor_redis' options | `{'database_deploy': True, 'redis': {'host': 'rekor-redis-pod', 'port': 6379, 'password': 'password'}, 'backfill_redis': {'enabled': True, 'schedule': {'minute': '0', 'hour': '0', 'day': '*', 'month': '*', 'weekday': '*'}}}` |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH I think this shouldn't live under the tas_single_node_rekor_redis key, but be its own key. The tas_single_node_rekor_redis key is supposed to be for redis connection and this kinda stretches it (and, as you see yourself in the user_provided scenario change, once you provide tas_single_node_rekor_redis, you have to define the backfill job parameters, which is not great)

| tas_single_node_trillian | Details on the database connection for Trillian. You can set this to a custom MySQL or MariaDB instance. | dict of 'tas_single_node_trillian' options | `{'database_deploy': True, 'mysql': {'user': 'mysql', 'root_password': 'rootpassword', 'password': 'password', 'database': 'trillian', 'host': 'trillian-mysql-pod', 'port': 3306}}` |
| tas_single_node_rekor_public_key_retries | The number of attempts to retrieve the Rekor public key when constructing the trust root. | int | `5` |
| tas_single_node_rekor_public_key_delay | The number of seconds to wait before retrying the retrieval of the Rekor public key when constructing the trust root. | int | `10` |
Expand Down Expand Up @@ -50,6 +50,7 @@ Deploy the [RHTAS](https://docs.redhat.com/en/documentation/red_hat_trusted_arti
|---|---|---|---|---|
| database_deploy | Whether or not to deploy Redis. | bool | no | |
| redis | Details on the Redis connection. | dict of 'redis' options | no | |
| backfill_redis | Backfill Redis configuration | dict of 'backfill_redis' options | no | |

#### Options for main > tas_single_node_rekor_redis > redis

Expand All @@ -59,6 +60,23 @@ Deploy the [RHTAS](https://docs.redhat.com/en/documentation/red_hat_trusted_arti
| port | The Redis host port number. | int | no | |
| password | The Redis password. | str | no | |

#### Options for main > tas_single_node_rekor_redis > backfill_redis

|Option|Description|Type|Required|Default|
|---|---|---|---|---|
| enabled | Whether or not to deploy the backfill Redis cron job. | bool | no | |
| schedule | Cron schedule for the backfill Redis job. | dict of 'schedule' options | no | |

#### Options for main > tas_single_node_rekor_redis > backfill_redis > schedule

|Option|Description|Type|Required|Default|
|---|---|---|---|---|
| minute | Specifies the minute(s) of each hour when the cron job should run (0-59). | str | no | `*` |
| hour | Specifies the hour(s) of each day when the cron job should run (0-23). | str | no | `*` |
| day | Specifies the day(s) of the month when the cron job should run (1-31). | str | no | `*` |
| month | Specifies the month(s) when the cron job should run (1-12 or month names). | str | no | `*` |
| weekday | Specifies the day(s) of the week when the cron job should run (0-6, where 0=Sunday). | str | no | `*` |

#### Options for main > tas_single_node_trillian

|Option|Description|Type|Required|Default|
Expand Down
10 changes: 10 additions & 0 deletions roles/tas_single_node/defaults/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,14 @@ tas_single_node_rekor_redis:
host: rekor-redis-pod
port: 6379
password: password
backfill_redis:
enabled: true
schedule:
minute: "0"
hour: "0"
day: "*"
month: "*"
weekday: "*"

tas_single_node_trillian:
database_deploy: true
Expand Down Expand Up @@ -65,6 +73,8 @@ tas_single_node_ctlog_image:
"registry.redhat.io/rhtas/certificate-transparency-rhel9@sha256:31e7318a9b19ed04ef0f25949f1f1709d293b532316b27a06f83fa5174547b17"
tas_single_node_rekor_redis_image:
"registry.redhat.io/rhtas/trillian-redis-rhel9@sha256:18820b1fbdbc2cc3e917822974910332d937b03cfe781628bd986fd6a5ee318e"
tas_single_node_backfill_redis_image:
"registry.redhat.io/rhtas/rekor-backfill-redis-rhel9@sha256:c5995c88063bd9875ae61c299bcf549002fcde724aab09807c70934e73daf356"
tas_single_node_trillian_db_image:
"registry.redhat.io/rhtas/trillian-database-rhel9@sha256:501612745e63e5504017079388bec191ffacf00ffdebde7be6ca5b8e4fd9d323"
tas_single_node_tuf_image:
Expand Down
55 changes: 55 additions & 0 deletions roles/tas_single_node/meta/argument_specs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,14 @@ argument_specs:
host: rekor-redis-pod
port: 6379
password: password
backfill_redis:
enabled: true
schedule:
minute: "0"
hour: "0"
day: "*"
month: "*"
weekday: "*"
options:
database_deploy:
description: "Whether or not to deploy Redis."
Expand Down Expand Up @@ -62,6 +70,53 @@ argument_specs:
type: "str"
required: false
version_added: "1.1.0"
backfill_redis:
description: "Backfill Redis configuration"
type: "dict"
required: false
version_added: "1.1.1"
options:
enabled:
description: "Whether or not to deploy the backfill Redis cron job."
type: "bool"
required: false
version_added: "1.1.1"
schedule:
description: "Cron schedule for the backfill Redis job."
type: "dict"
required: false
version_added: "1.1.1"
options:
minute:
description: "Specifies the minute(s) of each hour when the cron job should run (0-59)."
type: "str"
required: false
default: "*"
version_added: "1.1.1"
hour:
description: "Specifies the hour(s) of each day when the cron job should run (0-23)."
type: "str"
required: false
default: "*"
version_added: "1.1.1"
day:
description: "Specifies the day(s) of the month when the cron job should run (1-31)."
type: "str"
required: false
default: "*"
version_added: "1.1.1"
month:
description: "Specifies the month(s) when the cron job should run (1-12 or month names)."
type: "str"
required: false
default: "*"
version_added: "1.1.1"
weekday:
description: "Specifies the day(s) of the week when the cron job should run (0-6, where 0=Sunday)."
type: "str"
required: false
default: "*"
version_added: "1.1.1"
tas_single_node_trillian:
description: "Details on the database connection for Trillian. You can set this to a custom MySQL or MariaDB instance."
type: "dict"
Expand Down
2 changes: 2 additions & 0 deletions roles/tas_single_node/tasks/podman.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
"{{ tas_single_node_rekor_enabled }}",
"{{ tas_single_node_ctlog_enabled }}",
"{{ tas_single_node_rekor_enabled and tas_single_node_rekor_redis.database_deploy }}",
"{{ tas_single_node_rekor_enabled and tas_single_node_rekor_redis.database_deploy and tas_single_node_rekor_redis.backfill_redis.enabled }}",
"{{ tas_single_node_trillian_enabled and tas_single_node_trillian.database_deploy }}",
"{{ tas_single_node_tuf_enabled }}",
"{{ tas_single_node_trillian_enabled }}",
Expand All @@ -61,6 +62,7 @@
- "{{ tas_single_node_rekor_server_image }}"
- "{{ tas_single_node_ctlog_image }}"
- "{{ tas_single_node_rekor_redis_image }}"
- "{{ tas_single_node_backfill_redis_image }}"
- "{{ tas_single_node_trillian_db_image }}"
- "{{ tas_single_node_tuf_image }}"
- "{{ tas_single_node_trillian_netcat_image }}"
Expand Down
30 changes: 30 additions & 0 deletions roles/tas_single_node/tasks/podman/rekor.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,33 @@
configmap_changed: "{{ configmap_result.changed }}"
secret: "{{ tas_single_node_rekor_secret }}"
secret_changed: "{{ secret_result.changed }}"

- name: Get RHTAS network details
ansible.builtin.command:
cmd: "podman network inspect {{ tas_single_node_podman_network }}"
register: tas_podman_network_results
changed_when: false
when: tas_single_node_rekor_redis.backfill_redis.enabled

- name: Set DNS Resolver
ansible.builtin.set_fact:
dns_resolver: "{{ (tas_podman_network_results.stdout | from_json)[0].subnets[0].gateway }}"
when: tas_single_node_rekor_redis.backfill_redis.enabled

- name: Deploy backfill_redis script
ansible.builtin.template:
src: manifests/rekor/backfill_redis.sh.j2
dest: /usr/local/bin/backfill_redis.sh
mode: '0755'
when: tas_single_node_rekor_redis.backfill_redis.enabled

- name: Create Backfill Redis Cronjob
ansible.builtin.cron:
name: "backfill_redis"
minute: "{{ tas_single_node_rekor_redis.backfill_redis.schedule.minute }}"
hour: "{{ tas_single_node_rekor_redis.backfill_redis.schedule.hour }}"
day: "{{ tas_single_node_rekor_redis.backfill_redis.schedule.day }}"
month: "{{ tas_single_node_rekor_redis.backfill_redis.schedule.month }}"
weekday: "{{ tas_single_node_rekor_redis.backfill_redis.schedule.weekday }}"
job: "/usr/local/bin/backfill_redis.sh"
when: tas_single_node_rekor_redis.backfill_redis.enabled
38 changes: 38 additions & 0 deletions roles/tas_single_node/templates/manifests/rekor/backfill_redis.sh.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
#!/bin/sh
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I provided some comments throughout this file, I think a better (although much laborious) route to take would be:

  • Create a systemd timer unit
  • Create a k8s manifest and use our standard approach to create a systemd service unit for it to run
  • Make the timer unit run the job on schedule

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im not familiar with systemd timer unit, so i will take a look :) thanks for the feedback

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

LAST_INDEX_FILE="/tmp/rekor_last_index"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the last index not queryable? Storing in a /tmp file is not great, because that will get wiped out on reboot. Can we either make this dynamic (query for it) or figure out a more reliable approach to store it (would a podman secret work? or at least a better file location somewhere in /var?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK, the last index filled in is not queryable, I will take a look at a better way to store it


endIndex=$(curl -sS http://localhost:3001/api/v1/log | jq -r '.treeSize - 1')
if [ "$endIndex" -lt 0 ]; then
echo "info: no rekor entries found"
exit 0
fi

if [ -f "$LAST_INDEX_FILE" ]; then
startIndex=$(cat "$LAST_INDEX_FILE")
else
startIndex=0
fi

if [ "$startIndex" -gt "$endIndex" ]; then
startIndex=0
fi

if [ "$startIndex" -le "$endIndex" ]; then
podman run --rm --name backfill_redis \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use --force instead of --rm? That should ensure that the previous container always stays on the node (until the cron job runs next time, which will replace it), so if there's a failure, the user has time to capture logs.

-v "{{ tas_single_node_certs_dir }}/ingress-rekor.pem:/usr/local/share/ca-certificates/ingress-rekor.pem:ro" \
-e SSL_CERT_FILE=/usr/local/share/ca-certificates/ingress-rekor.pem \
JasonPowr marked this conversation as resolved.
Show resolved Hide resolved
--add-host="rekor.{{ tas_single_node_base_hostname }}:{{ dns_resolver }}" \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the dns_resolver/add-host is necessary at all. All pods on the same network (use --network) should be able to connect to each other - see e.g. the rekor-server.j2 to see how it connects to trillian and Redis.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did try this first, I didn't have much luck if I am honest, backfill redis is awkward when connecting to the rekor instance it has to be a url so it can query the /api/v1/log/entries?logIndex=x api endpoint

I will take another look, also when I think about it the unable to connect may have been related to the issue fixed in #100, either way I will take a look

--add-host="{{ tas_single_node_rekor_redis.redis.host }}:{{ dns_resolver }}" \
"{{ tas_single_node_backfill_redis_image }}" \
--hostname="{{ tas_single_node_rekor_redis.redis.host }}" \
--password="{{ tas_single_node_rekor_redis.redis.password }}" \
--port="{{ tas_single_node_rekor_redis.redis.port }}" \
--rekor-address="http://rekor.{{ tas_single_node_base_hostname }}" \
--start="$startIndex" --end="$endIndex"

echo "$endIndex" > "$LAST_INDEX_FILE"
else
echo "info: no new entries to process"
fi
2 changes: 1 addition & 1 deletion vm-testing/dev-images.sed
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ s#registry.redhat.io/rhtas/fulcio-rhel9#quay.io/securesign/fulcio-server#
s#registry.redhat.io/rhtas/trillian-redis-rhel9#quay.io/securesign/trillian-redis#
s#registry.redhat.io/rhtas/rekor-server-rhel9#quay.io/securesign/rekor-server#
s#registry.redhat.io/rhtas/rekor-search-ui-rhel9#quay.io/securesign/rekor-search-ui#
s#registry.redhat.io/rhtas/rekor-backfill-redis-rhel9#quay.io/securesign/trillian-redis#
s#registry.redhat.io/rhtas/tuf-server-rhel9#quay.io/securesign/scaffold-tuf-server#
s#registry.redhat.io/rhtas/certificate-transparency-rhel9#quay.io/securesign/certificate-transparency-go#
s#registry.redhat.io/rhtas/client-server-cg-rhel9#quay.io/securesign/cli-client-server-cg#
Expand All @@ -14,3 +13,4 @@ s#registry.redhat.io/rhtas/segment-reporting-rhel9#quay.io/securesign/segment-ba
s#registry.redhat.io/rhtas/timestamp-authority-rhel9#quay.io/securesign/timestamp-authority#
s#registry.redhat.io/rhtas/trillian-createtree-rhel9#quay.io/securesign/trillian-createtree#
s#registry.redhat.io/rhtas/client-server-rhel9#quay.io/securesign/client-server#
s#registry.redhat.io/rhtas/rekor-backfill-redis-rhel9#quay.io/securesign/rekor-backfill-redis#
Loading