Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Configurable Timeouts for Operations and Ollama Integration Documentation #24

Merged
merged 2 commits into from
Sep 29, 2024

Conversation

shreyashankar
Copy link
Collaborator

This PR introduces configurable timeouts for operations and adds comprehensive documentation for Ollama integration, enhancing our system's reliability and user guidance. This will close #23.

Changes

  1. Added support for configurable timeouts in operations
  2. Implemented a new max_retries_per_timeout parameter (default: 2)
  3. Created a new test in test_map.py to verify the timeout functionality
  4. Added documentation for Ollama integration with timeout examples

Details

Timeout Implementation

  • Operations now accept a timeout parameter in their configuration
  • If a timeout occurs, the operation will retry up to max_retries_per_timeout times

New Parameter

  • max_retries_per_timeout: Controls the number of retry attempts after a timeout (default: 2)

Testing

  • Added a new test case in test_map.py to ensure proper functionality of the timeout feature

Documentation

  • Created new documentation for Ollama integration
  • Included example of using timeouts with Ollama operations
  • Provided guidance on estimating appropriate timeout values

Usage Example

operations:
  - name: classify_medical_record
    type: map
    timeout: 20000  # Example timeout for Ollama operation
    max_retries_per_timeout: 3
    # ... other configuration ...

Benefits

  • Improved handling of long-running operations, especially for Ollama integrations
  • Enhanced system stability by preventing indefinite hangs
  • Greater control over resource allocation and task management
  • Clear guidance for users on integrating Ollama and managing timeouts

@shreyashankar shreyashankar merged commit 552b65e into main Sep 29, 2024
3 checks passed
@shreyashankar shreyashankar deleted the shreyashankar/blockingerr branch September 29, 2024 23:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ollama Models Timeout When Processing Many Documents
1 participant