Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retry for urlopen #6077

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bssrikanth
Copy link
Contributor

Many a times open url returns 503 errors, handling it by introducing retries.

Before:

[stdlog] 2024-12-02 16:42:53,814 avocado.utils.download download         L0096 INFO | Fetching https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/snapshot/master.tar.gz -> /root/avocado/data/cache/by_location/fb1b8783e9b605568767b2ad1c7c4e9a264d1085/kselftest.tar.f5c94290-8910-4763-88d7-8583bf659cd0
[stdlog] 2024-12-02 16:42:59,104 avocado.utils.download download         L0051 ERROR| Failed downloading file: The read operation timed out
[stderr] Failed to get file. Probably timeout was reached when connecting to the server.
[stdlog] 2024-12-02 16:42:59,109 avocado.utils.asset asset            L0165 INFO | Temporary asset file unavailable due to failed download attempt.

After:

[stdlog] 2024-12-02 02:00:50,228 avocado.utils.asset asset            L0152 DEBUG| Asset not in cache after lock, fetching it.
[stdlog] 2024-12-02 02:00:50,228 avocado.utils.download download         L0101 INFO | Fetching https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/snapshot/master.tar.gz -> /root/avocado/data/cache/by_location/fb1b8783e9b605568767b2ad1c7c4e9a264d1085/kselftest.tar.1a5eab69-aff0-4f99-a111-2f77e9b30f23
[stdlog] 2024-12-02 02:00:55,500 avocado.utils.download download         L0053 ERROR| Failed downloading file: The read operation timed out
[stdlog] 2024-12-02 02:00:55,501 avocado.utils.download download         L0055 DEBUG| retry attempt: 1
[stdlog] 2024-12-02 02:01:00,716 avocado.utils.download download         L0053 ERROR| Failed downloading file: The read operation timed out
[stdlog] 2024-12-02 02:01:00,717 avocado.utils.download download         L0055 DEBUG| retry attempt: 2
[stdlog] 2024-12-02 02:01:05,813 avocado.utils.download download         L0053 ERROR| Failed downloading file: The read operation timed out
[stdlog] 2024-12-02 02:01:05,813 avocado.utils.download download         L0055 DEBUG| retry attempt: 3
[stdlog] 2024-12-02 02:01:05,814 avocado.utils.download download         L0063 DEBUG| Opened URL "https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/snapshot/master.tar.gz" and received response with headers including: content-length UNKNOWN, date: "Sun, 01 Dec 2024 20:31:16 GMT", last-modified: "Sun, 01 Dec 2024 20:31:06 GMT"

@bssrikanth bssrikanth force-pushed the add_retry_urlopen branch 2 times, most recently from 4378201 to 5d9791b Compare December 2, 2024 12:23
Copy link

codecov bot commented Dec 2, 2024

Codecov Report

Attention: Patch coverage is 58.33333% with 5 lines in your changes missing coverage. Please review.

Project coverage is 68.67%. Comparing base (f452aa5) to head (1ecfb47).
Report is 9 commits behind head on master.

Files with missing lines Patch % Lines
avocado/utils/download.py 58.33% 5 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #6077       +/-   ##
===========================================
+ Coverage   54.47%   68.67%   +14.20%     
===========================================
  Files         202      202               
  Lines       21892    21899        +7     
===========================================
+ Hits        11925    15039     +3114     
+ Misses       9967     6860     -3107     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Many a times open url returns errors including 503,
handling it by introducing retries.

Signed-off-by: Srikanth Aithal <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Review Requested
Development

Successfully merging this pull request may close these issues.

1 participant