Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automatically detect the rate limits #233

Open
RyanMarten opened this issue Dec 9, 2024 · 0 comments
Open

Automatically detect the rate limits #233

RyanMarten opened this issue Dec 9, 2024 · 0 comments

Comments

@RyanMarten
Copy link
Contributor

By sending requests and seeing when you get 429s back
This was referenced Dec 9, 2024
Closed
Merged
@RyanMarten RyanMarten mentioned this issue Dec 17, 2024
Open
4 tasks
devin-ai-integration bot added a commit that referenced this issue Dec 29, 2024
@devin-ai-integration
feat: implement dynamic rate limit detection using Retry-After headers
a7667ee 
- Add retry_after_seconds to StatusTracker for dynamic wait times
- Extract Retry-After header from OpenAI rate limit responses
- Extract Retry-After from LiteLLM RateLimitError
- Update base processor to use dynamic retry times
- Fall back to default pause time if no Retry-After header

Fixes #233
@devin-ai-integration devin-ai-integration bot mentioned this issue Dec 29, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@RyanMarten