SECURESIGN-1476 | Add the Redis backfill job to Ansible collection #101
Conversation
JasonPowr
force-pushed
the
add-backfill-redis-job
branch
from
28586c7
to
ca2ca28
Compare
JasonPowr
force-pushed
the
add-backfill-redis-job
branch
from
ca2ca28
to
e67ba38
Compare
|
Additionally, ideally you could add a workable test for this within |
Im not really sure there is a good way to test this if I am honest, ill take a look, but really we would only be testing to make sure that we can configure it with ansible and thats already done here |
| podman run --rm --name backfill_redis \ | ||
| -v "{{ tas_single_node_certs_dir }}/ingress-rekor.pem:/usr/local/share/ca-certificates/ingress-rekor.pem:ro" \ | ||
| -e SSL_CERT_FILE=/usr/local/share/ca-certificates/ingress-rekor.pem \ | ||
| --add-host="rekor.{{ tas_single_node_base_hostname }}:{{ dns_resolver }}" \ |
There was a problem hiding this comment.
I don't think the dns_resolver/add-host is necessary at all. All pods on the same network (use --network) should be able to connect to each other - see e.g. the rekor-server.j2 to see how it connects to trillian and Redis.
There was a problem hiding this comment.
I did try this first, I didn't have much luck if I am honest, backfill redis is awkward when connecting to the rekor instance it has to be a url so it can query the /api/v1/log/entries?logIndex=x api endpoint
I will take another look, also when I think about it the unable to connect may have been related to the issue fixed in #100, either way I will take a look
| fi | ||
|
|
||
| if [ "$startIndex" -le "$endIndex" ]; then | ||
| podman run --rm --name backfill_redis \ |
There was a problem hiding this comment.
Can we use --force instead of --rm? That should ensure that the previous container always stays on the node (until the cron job runs next time, which will replace it), so if there's a failure, the user has time to capture logs.
| @@ -18,7 +18,7 @@ Deploy the [RHTAS](https://docs.redhat.com/en/documentation/red_hat_trusted_arti | |||
| |Option|Description|Type|Default| | |||
| |---|---|---|---| | |||
| | tas_single_node_podman_network | Name of the Podman network for containers to use. | str | `rhtas` | | |||
| | tas_single_node_rekor_redis | Details on the Redis connection for Rekor. You can set this to a custom Redis instance. | dict of 'tas_single_node_rekor_redis' options | `{'database_deploy': True, 'redis': {'host': 'rekor-redis-pod', 'port': 6379, 'password': 'password'}}` | | |||
| | tas_single_node_rekor_redis | Details on the Redis connection for Rekor. You can set this to a custom Redis instance. | dict of 'tas_single_node_rekor_redis' options | `{'database_deploy': True, 'redis': {'host': 'rekor-redis-pod', 'port': 6379, 'password': 'password'}, 'backfill_redis': {'enabled': True, 'schedule': {'minute': '0', 'hour': '0', 'day': '*', 'month': '*', 'weekday': '*'}}}` | | |||
There was a problem hiding this comment.
TBH I think this shouldn't live under the tas_single_node_rekor_redis key, but be its own key. The tas_single_node_rekor_redis key is supposed to be for redis connection and this kinda stretches it (and, as you see yourself in the user_provided scenario change, once you provide tas_single_node_rekor_redis, you have to define the backfill job parameters, which is not great)
| @@ -0,0 +1,38 @@ | |||
| #!/bin/sh | |||
There was a problem hiding this comment.
While I provided some comments throughout this file, I think a better (although much laborious) route to take would be:
- Create a systemd timer unit
- Create a k8s manifest and use our standard approach to create a systemd service unit for it to run
- Make the timer unit run the job on schedule
There was a problem hiding this comment.
Im not familiar with systemd timer unit, so i will take a look :) thanks for the feedback
| #!/bin/sh | ||
| PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin | ||
|
|
||
| LAST_INDEX_FILE="/tmp/rekor_last_index" |
There was a problem hiding this comment.
Is the last index not queryable? Storing in a /tmp file is not great, because that will get wiped out on reboot. Can we either make this dynamic (query for it) or figure out a more reliable approach to store it (would a podman secret work? or at least a better file location somewhere in /var?)
There was a problem hiding this comment.
AFAIK, the last index filled in is not queryable, I will take a look at a better way to store it
Pr to add the backfill redis, setting as draft for now as it should only be merged after #100
Upstream pr: sigstore/rekor#2280