ci: Improve reliability of ansible tasks #345
Conversation
There was a problem hiding this comment.
Apart from a minor typo in the header of 7f2271d this looks good to me and hopefully the retries do the trick. Thanks!
|
Changes:
|
| @@ -46,6 +46,8 @@ | |||
| url: https://storage.googleapis.com/kubernetes-release/release/{{ k8s_version }}/bin/linux/{{ target_arch }}/{{ item }} | |||
| dest: /usr/local/bin | |||
| mode: '+x' | |||
| retries: 3 | |||
There was a problem hiding this comment.
hi @ldoktor ! Good to know what ansible has now retry for tasks (or exists a long time and I never noticed:(
I noticed that on this file you didn't retry unarchive tasks. Isn't it idempotent?
| @@ -20,6 +20,8 @@ | |||
| repo: https://github.com/bats-core/bats-core.git | |||
| dest: bats-core | |||
| force: yes | |||
| retries: 3 | |||
There was a problem hiding this comment.
Install test dependencies can be retried too I think.
| @@ -13,10 +13,16 @@ | |||
| name: "{{ build_pkgs[ansible_distribution | lower] }}" | |||
| state: present | |||
| - block: | |||
| - name: Download and extract Go tarball | |||
| - name: Download Go tarball | |||
There was a problem hiding this comment.
This answers one of my questions on the previous commit :D
There was a problem hiding this comment.
Perhaps run get_url with force: true to update the tgz in case it exists and the content changed.
There was a problem hiding this comment.
On the other hand the workers should be always clean and locally we might benefit of the caching, what do you think?
There was a problem hiding this comment.
Caching is nice. Then rename go.tar.gz -> go{{ go_version }}.linux-{{ target_arch }}.tar.gz so that a version update will force the download?
There was a problem hiding this comment.
yep, that sounds good
| @@ -37,7 +37,7 @@ | |||
| ignore_errors: yes | |||
| - name: Install kustomize | |||
| shell: | | |||
| curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash | |||
| curl -s --retry 3 --retry-delay 10 "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash | |||
There was a problem hiding this comment.
Luckily the curl version installed on CI supports --retry? I think it was a patch to kata that it didn't work because it has used an old curl, right?
There was a problem hiding this comment.
The --retry option is fairly old, the problem in kata was the --retry-on-all-errors
|
hi @ldoktor ! Good idea using "retries everywhere"; I should make it a habit on my codes for sure. |
|
Changes:
|
|
Changes:
|
|
@wainersm do you find this ready to be merged (after rebase) or is there something I should improve to avoid unnecessary extra rebases? |
the network in CI environment tends to break from time to time, let's allow up to 3 retries for tasks that support it and that use external sources. Fixes: confidential-containers#339 Signed-off-by: Lukáš Doktor <[email protected]>
Network tends to be flaky, let's use get_url module with retries to ensure external resources are fetched and unpack them afterwards (unfortunatelly the unarchive module does not supports retries). Signed-off-by: Lukáš Doktor <[email protected]>
as network tends to be flaky, allow curl to retry in case it fails to fetch the kustomize. Signed-off-by: Lukáš Doktor <[email protected]>
we do want to do multiple steps and doing that in shell makes sense in those cases, ignore the ansible warning explicitly. Signed-off-by: Lukáš Doktor <[email protected]>
|
No changes, just rebased |
Let's allow to re-try tasks that make sense and a few cleanups.