Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Getting a lot of failed downloads #3

Open
RandomLegend opened this issue Sep 2, 2024 · 17 comments
Open

Getting a lot of failed downloads #3

RandomLegend opened this issue Sep 2, 2024 · 17 comments
Labels
wontfix This will not be worked on

Comments

@RandomLegend
Copy link

Putting in a playlist of mine i get a lot of failed downloads, i'd say roughly 50% of the playlist is failing, the songs do definetely exist on spotify and are playable.

This is the terminal output for on of them, they all look like this:

[2024-09-02 23:27:53,707 :: spotutils :: /home/RandomLegend/Downloads/onthespot/onthespot/src/onthespot/utils/spotify.py -> 103:       sanitize_data() :: INFO] -> Sanitising string: "Stellar"; Allow path separators: False
[2024-09-02 23:27:53,707 :: spotutils :: /home/RandomLegend/Downloads/onthespot/onthespot/src/onthespot/utils/spotify.py -> 103:       sanitize_data() :: INFO] -> Sanitising string: "Masquerade"; Allow path separators: False
[2024-09-02 23:27:53,707 :: spotutils :: /home/RandomLegend/Downloads/onthespot/onthespot/src/onthespot/utils/spotify.py -> 103:       sanitize_data() :: INFO] -> Sanitising string: "Masquerade"; Allow path separators: False
[2024-09-02 23:27:53,707 :: spotutils :: /home/RandomLegend/Downloads/onthespot/onthespot/src/onthespot/utils/spotify.py -> 103:       sanitize_data() :: INFO] -> Sanitising string: "Stellar"; Allow path separators: False
[2024-09-02 23:27:53,707 :: spotutils :: /home/RandomLegend/Downloads/onthespot/onthespot/src/onthespot/utils/spotify.py -> 103:       sanitize_data() :: INFO] -> Sanitising string: "2022 Stellar"; Allow path separators: False
Audio key error, code: 2
Audio key error, code: 2
[2024-09-02 23:27:54,235 :: worker.downloader.SESSION_DL_TH-768b3b82-2ebe-43d8-88a8-370351388c98 :: /home/RandomLegend/Downloads/onthespot/onthespot/src/onthespot/worker/downloader.py -> 221:      download_track() :: ERROR] -> Download failed for track by id '6UnOxNp0jZfFXyKcAtJwUj', Unexpected error: Traceback (most recent call last):
  File "/home/RandomLegend/Downloads/onthespot/onthespot/src/onthespot/worker/downloader.py", line 122, in download_track
    stream = session.content_feeder().load(track_id, VorbisOnlyAudioQuality(quality), False, None)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/RandomLegend/Downloads/onthespot/onthespot/venv/lib/python3.12/site-packages/librespot/audio/__init__.py", line 739, in load
    return self.load_track(playable_id, audio_quality_picker, preload,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/RandomLegend/Downloads/onthespot/onthespot/venv/lib/python3.12/site-packages/librespot/audio/__init__.py", line 800, in load_track
    return self.load_stream(file, track, None, preload, halt_listener)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/RandomLegend/Downloads/onthespot/onthespot/venv/lib/python3.12/site-packages/librespot/audio/__init__.py", line 754, in load_stream
    return CdnFeedHelper.load_track(self.__session, track, file,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/RandomLegend/Downloads/onthespot/onthespot/venv/lib/python3.12/site-packages/librespot/audio/__init__.py", line 339, in load_track
    key = session.audio_key().get_audio_key(track.gid, file.file_id)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/RandomLegend/Downloads/onthespot/onthespot/venv/lib/python3.12/site-packages/librespot/audio/__init__.py", line 277, in get_audio_key
    return self.get_audio_key(gid, file_id, False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/RandomLegend/Downloads/onthespot/onthespot/venv/lib/python3.12/site-packages/librespot/audio/__init__.py", line 278, in get_audio_key
    raise RuntimeError(
RuntimeError: Failed fetching audio key! gid: (dontknowifthisissecret), fileId: (dontknowifthisissecret)
 !
[2024-09-02 23:27:54,235 :: worker.downloader.SESSION_DL_TH-768b3b82-2ebe-43d8-88a8-370351388c98 :: /home/RandomLegend/Downloads/onthespot/onthespot/src/onthespot/worker/downloader.py -> 384:                 run() :: ERROR] -> Download process returned false: 6UnOxNp0jZfFXyKcAtJwUj

i'm on arch linux, newest kernel, using the portable python version and installed the requirements inside a python venv.
@RandomLegend
Copy link
Author

Alright, either it was completely restarting my PC (didn't think that could solve any issues with that :D) or it was the fact that i readded the playlist completely - either and / or both of those steps fixed the issue.

@RandomLegend
Copy link
Author

Ha nope, just started doing it again!

After processing 66 songs, it started to fail again

@RandomLegend RandomLegend reopened this Sep 2, 2024
@RandomLegend
Copy link
Author

So after searching for that specific error i stumbled accross multiple github entries from various librespot implementations (1, 2, 3) and none of them seemed fixed 🤔

Letting it process the list until it's done, quitting the application and restarting the playlist seems to get me through it pass by pass. So i'd guess it'll take me around 7-10 restarts of the application to get my whole list downloaded.

I take what i can get, but maybe this is fixable?

@justin025
Copy link
Owner

justin025 commented Sep 3, 2024 •
edited
Loading

Sometimes when it fails on a specific song, in your case 67, it means there is something wrong with that song. From the people I've spoke to it could be a podcast duplicate or special characters in the title/metadata. You can try removing that song from your playlist, bulk downloading and then downloading that song separately.

I hear the original dev is working on a fix for the playlist api but it will take some time to be pushed upstream.

@RandomLegend
Copy link
Author

Alright, thanks for that.

Would it be possible for you to make a button that would allow users to copy the links of all failed songs?
So in my example i would paste my playlist link, let it go through, start to fail at some point and then there would be a button and with that button i'd have a list of all failed songs that i could paste again so it doesn't have to go through the playlist again?

@maguirreg
Copy link

The issue occurs with both playlists and podcasts. I traced it down to the call that retrieves information about the media item here. The problem isn't due to an error with the media itself; instead, I suspect it might be related to the interface with the source. It's possible they’ve implemented a request limit within a certain time frame. Even after increasing the download delay to 10, the error persists. Running with download delay of 15 gave me better results for a moment allowing me to download more items, but started to fail again after a couple more.
I believe one solution would be to save all failed ones to retry them after, or implement something like a database with the MediaID and details to avoid making too many requests for the same unchanged data.

@RandomLegend
Copy link
Author

Yeah after looking into my failed downloads i saw that there are some tracks in there that don't have any special character or anything.

Examaple - Runaway from The Corrs

Decreasing the amount of workers to "1" and increasing the wait time to "15" made it much more stable for me. I was able to download a whole playlist without a single fail now.

@justin025
Copy link
Owner

The function you linked above seems to be for downloading episodes which are a little different, download_track is at lines 33-222 in the same file. It could definitely be a rate limit error but we would probably see a '400' http request somewhere in the log, see 204 for example. I do know a 'request cache' already exists but the problem is not metadata and songs in a playlist but the ogg file itself. If you guys are right its more likely that spotify has placed limits on the most bandwidth intensive media, that being streams.

I'll need to look into errors like these at some point. Thank you for sharing.

@justin025
Copy link
Owner

I know some people are sensitive about sharing their playlists but would you mind sharing links to any playlists that break so I can compile a list to test. If you don't feel comfortable sharing publicly you can dm me on discord j075046

@maguirreg
Copy link

This is my last attempt:

  1. Download delay = 5, Max download Workers = 3
  2. Tried to download Case 63, for the first time after 5 hours without using the app.
  3. First run, it downloaded all 22 tracks. Cleared the completed/cancelled items and tried again
  4. Second run, failed on the 3rd, 7th, 14th, 15th, 17th-22nd tracks, same "Failed fetching audio key" error
    It suppose to skip the already downloaded items, but when is trying to get the info from the media it fails.
    image
    image
    output.log
    I'm attaching the logs for another execution where several others failed for the same list

@justin025
Copy link
Owner

justin025 commented Sep 3, 2024 •
edited
Loading

Definitely a rate limit issue. I downloaded the podcast and the first time it worked flawlessly. The second time it downloaded like so:
2024-09-03_10-37
I'm afraid I have no solution to this at the moment so I will reopen the issue until I can either add logging informing the user they are being rate limited or figure out a solution.

For any users experiencing this, just try closing and opening the app or waiting a little while and try again.

@justin025 justin025 reopened this Sep 3, 2024
@pdtan pdtan mentioned this issue Sep 4, 2024
Open
@justin025 justin025 mentioned this issue Sep 12, 2024
Open
@zerog13
Copy link

zerog13 commented Sep 13, 2024

Changing the Download Delay to 25 has fixed my download rates 100%, even the metadata errors.

@zerog13
Copy link

zerog13 commented Sep 13, 2024

This is my last attempt:

1. Download delay = 5, Max download Workers = 3

2. Tried to download [Case 63](https://open.spotify.com/show/4c9ZKaFtEKweSYOlYvxfvp), for the first time after 5 hours without using the app.

3. First run, it downloaded all 22 tracks. Cleared the completed/cancelled items and tried again

4. Second run, failed on the 3rd, 7th, 14th, 15th, 17th-22nd tracks, same "Failed fetching audio key" error
   It suppose to skip the already downloaded items, but when is trying to get the info from the media it fails.
   ![image](https://private-user-images.githubusercontent.com/734215/363984409-bba9ceed-a833-4301-a07b-52112a78e6bf.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYyMzM3NDUsIm5iZiI6MTcyNjIzMzQ0NSwicGF0aCI6Ii83MzQyMTUvMzYzOTg0NDA5LWJiYTljZWVkLWE4MzMtNDMwMS1hMDdiLTUyMTEyYTc4ZTZiZi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxM1QxMzE3MjVaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1mMTQ0YWMwZTg2OWFlMzU3MmQxMDk4ZjhlMDU4ZWQ1ZTBlYjhlMTRjYmFkMzllYzllZWIwMjUyZDhmNTIyZGJjJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.DYz5mNU49eb6ZH1N691Or8MkregCHu9M9NnOjrAddDo)
   ![image](https://private-user-images.githubusercontent.com/734215/363988210-2fb95f51-c2d8-40e2-a8ac-28fafa276688.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYyMzM3NDUsIm5iZiI6MTcyNjIzMzQ0NSwicGF0aCI6Ii83MzQyMTUvMzYzOTg4MjEwLTJmYjk1ZjUxLWMyZDgtNDBlMi1hOGFjLTI4ZmFmYTI3NjY4OC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEzJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxM1QxMzE3MjVaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0yZjNmYTEwMzNlZjgxMGIyZjFkMTZkZTVmZWY4NGUyMzQ5MmFkOTcxOGJmYmRlYmRlMGEzYjg0NjFiNThmODdkJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.DmOLKiDPIm7eVHGVoSMU_X4g7aO7S30PZsB-skWmuQE)
   [output.log](https://github.com/user-attachments/files/16848974/output.log)
   I'm attaching the logs for another execution where several others failed for the same list

Change download rate to 25, it fixed my issue 100%. no errors on the last 350 tracks downloaded.

@zerog13
Copy link

zerog13 commented Sep 14, 2024

This is my last attempt:

1. Download delay = 5, Max download Workers = 3

2. Tried to download [Case 63](https://open.spotify.com/show/4c9ZKaFtEKweSYOlYvxfvp), for the first time after 5 hours without using the app.

3. First run, it downloaded all 22 tracks. Cleared the completed/cancelled items and tried again

4. Second run, failed on the 3rd, 7th, 14th, 15th, 17th-22nd tracks, same "Failed fetching audio key" error
   It suppose to skip the already downloaded items, but when is trying to get the info from the media it fails.
   ![image](https://private-user-images.githubusercontent.com/734215/363984409-bba9ceed-a833-4301-a07b-52112a78e6bf.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYzMDE5NTEsIm5iZiI6MTcyNjMwMTY1MSwicGF0aCI6Ii83MzQyMTUvMzYzOTg0NDA5LWJiYTljZWVkLWE4MzMtNDMwMS1hMDdiLTUyMTEyYTc4ZTZiZi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTE0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxNFQwODE0MTFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1mOTkyYTczZThmNTBhYjljMzJjNjNlNzlkODE0ZjBlNmVkZDAzOThjN2RhN2VkZTA2NWU0NjY3YzliNTg2NTVlJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.wqNRGXCHCztU35kWLOerkUNi0-jf5iHlWFR7v45UHEI)
   ![image](https://private-user-images.githubusercontent.com/734215/363988210-2fb95f51-c2d8-40e2-a8ac-28fafa276688.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYzMDE5NTEsIm5iZiI6MTcyNjMwMTY1MSwicGF0aCI6Ii83MzQyMTUvMzYzOTg4MjEwLTJmYjk1ZjUxLWMyZDgtNDBlMi1hOGFjLTI4ZmFmYTI3NjY4OC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTE0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxNFQwODE0MTFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lZWU2YmE4ODFhMDIyZDliOGQ3NmU1ODcwMTA2NDI3YzE4N2E2NDNlMGRlOGRkYzAxMjRiZWIxNTFiYTM2OTdmJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.SmTogjukA8sy4iPxkHn6bJiD0KVK5T-IOZMcoWZ7DpE)
   [output.log](https://github.com/user-attachments/files/16848974/output.log)
   I'm attaching the logs for another execution where several others failed for the same list

try this plugin for podcasts - Podcasts - A player, downloader, transcriber https://podcasts.bluepill.life/index.html

@justin025 justin025 added bug Something isn't working wontfix This will not be worked on and removed bug Something isn't working labels Sep 24, 2024
@justin025 justin025 pinned this issue Sep 24, 2024
@justin025
Copy link
Owner

Although this doesn't necessarily fix the issue, in the latest release I've added the feature to rotate the currently downloading account in an effort to reduce rate limits

@justin025 justin025 mentioned this issue Oct 4, 2024
Closed
@bulasag
Copy link

bulasag commented Nov 20, 2024

I'm getting failed on all songs. Tried closing, waiting 15mins, opening and trying again but same thing happens. How do I see terminal output on the exe in W11?

@justin025
Copy link
Owner

You can copy the exe into git bash and can see the output there or its available at %localappdata%/Temp/onthespot/logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
wontfix This will not be worked on
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@maguirreg @bulasag @RandomLegend @zerog13 @justin025