Regular HTTP Bans #1182
Comments
|
As I said
So I went and looked into my logs to find the series "Shin Tennis no..." and found the following log. |
|
The point of using the calendar is to add/update the seasonal anime entries. Also it seems like you have enabled fetching related anime entries with a depth above 0 (where 0 means disabled), so it will fetch all the entries currently in the calendar in addition to any related anime entries. This seems to me like a misconfiguration on your part if you don't want it to update seasonal data, but do correct me if I'm assuming wrong here. |
|
Didn't touch those settings in the last couple of years, so I couldn't even say, if I did those settings. Even if, shouldn't there be a sanity check for updating "related" items eg. don't update items that are completed series. eg. Inuyasha. Inuyasha is long over and the chances it has new data is slim if not 0, and how would it be related to the current season? What I did now,
|
|
I'll just put a screenshot here because I don't want to copy-paste 😔 |
|
@Rand-Random you can also disable the related anime checkbox if you want. It will disable the depth setting when you do. |
|
this is same issue as the existence of JMM/Shoko. Until there is system defaults/built-in system to mitigate this issue new people with bigger collection will auto-ban them-self, over and over again. The "best strategy" was/will be adding single small series to import folder, and process it manually. or in batch-of-10. |
|
I'll share my log. server restarted at line 9560 to force anidb check and started to update only to get banned some minutes later. I think something may have changed on anidb side. Or shoko is updating faster than before. |
|
From the logs its seams that you leeched off anidb with requests and 3 minute later got ban. As it always been (more or less). That is sadly the "first" experience with bigger collections. Also it looks like HttpRateLimiter did fire just before ban, but then luck weren't on your side. But for real, @ElementalCrisis from this log it looks like he been asking anidb for 2 series per 1second. (00:31:40, 00:31:38, 00:31:34), 3series per 1 second (00:31:36, 00:31:28), 4 series per 1 second (00:31:20) - I wont 100% put my life for that log as its verbose if not clutter, but there is something there that can help with ban issue. I'm maybe old and rusty, but I don't recall being able to leech that fast from anidb in the best years of my life ;-) |
Where? 🤔 |
Sorry, I probably took 00:31:45 and 00:31:46 are 1 each second. Maybe that trigger something ? |
|
Update on my side: After disabling the "related anime" feature as @revam suggested, the ban didn't occur. I got curious and wanted to re-enable the feature and see if the ban again happens or not. 6 days later with the setting enabled, a ban again occurred. Currently there are 100 items in the queue to be updated Some "relations" are rather odd, eg. this anime is "related" (depth 3) to this anime: anyone has any idea why? and if I have some settings wrong, that those two are considered related. I have 2 files attached. Also rather strange IMHO is that "Dr." was added with the timestamp 16th of October why would it sill be in my current queue?
2 days later the ban happend. |
|
implementation detail of the calendar update job is that it won't add the entry to the queue if it has been updated within 2 days. and related anime won't be updated if they have been recently updated within 24h. |
after the initial burst of commands to hopefully prevent API bans. The long delay for UDP has been changed from two times the short delay to three times (4s → 6s), and HTTP has been changed from two times to fifteen times (4s → 30s). To compensate, then the reset period is down from 3m to 2m. If these numbers are too preservative then we can tweak them again. Related to #1182
|
@Rand-Random tell me if the bans became less frequent or not after updating to 5.0.0.30 please. |
|
@revam I am currently occupied with currently switching to shoko relay, there is also 1 issue you may be the person to talk with
|
Basically running into a ban almost immediatly after a ban is lifted, and this over and over again.
I have no clue what shoko is actually currently doing at all, it seems trying to fetch rather random series.
Shoko seems to queue more series to update, than it is able to fetch, already tried to cleared the queue only for a new file of the current season to again trigger this update cycle.
Current queue is:
No idea why it tries to update eg. "Inuyasha" or "Shin Tennis no Ouji-sama: U-17 World Cup Semifinal."
Didn't recently add anything to Inuyasha, why is it fetching data?
And the series "Shin Tennis no Ouji-sama: U-17 World Cup Semifinal" never heard about it and I don't have it in my library, so why?
As I said, I already once cleared the queue, but it again filled up, so I currently don't know what to do?
20:17 (8:17 pm) ban was lifted, 20:21 (8:21 pm) again banned.
Full log: log.txt
The text was updated successfully, but these errors were encountered: