Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make event handling faster and resilient to missing events #625

Merged
merged 40 commits into from
Feb 5, 2024

Conversation

cortadocodes
Copy link
Member

@cortadocodes cortadocodes commented Jan 30, 2024

Contents (#625)

IMPORTANT: There is 1 breaking change.

Enhancements

  • Allow setting of maximum number of workers for parallel questions in Child.ask_multiple
  • Pull up to 50 messages from answer subscriptions at once instead of 1
  • Allow skipping of any missing message after a 10s delay in OrderedMessageHandler
  • Suppress name/namespace override warning if the value is the same in the environment and service configuration file
  • Speed up event validation by caching service communication JSON schema
  • 💥 BREAKING CHANGE: Extract SRUID for child logs context from subscription in message handler

Fixes

  • Exit early from message pulling if heartbeat check fails
  • Make Manifest.update_dataset_paths method thread-safe

Refactoring

  • Factor out multiple checks of package version in message handler

Testing

  • Improve message handling tests by not mocking _pull_and_enqueue_available_messages method and removing MockMessagePuller

Upgrade instructions

💥 Extract SRUID for child logs from subscription in message handler

This removes the service_name argument from Service.wait_for_answer. If you were using this argument, simply remove it; logs from children shown in a parent will now have the full and correct SRUID automatically.

@cortadocodes cortadocodes self-assigned this Jan 31, 2024
@codecov-commenter
Copy link

codecov-commenter commented Jan 31, 2024

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (04364f2) 94.90% compared to head (e9d5d92) 94.86%.
Report is 5 commits behind head on main.

Files Patch % Lines
octue/cloud/pub_sub/message_handler.py 97.22% 1 Missing ⚠️
octue/diagnostics.py 75.00% 1 Missing ⚠️
octue/resources/child.py 83.33% 1 Missing ⚠️
octue/runner.py 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #625      +/-   ##
==========================================
- Coverage   94.90%   94.86%   -0.04%     
==========================================
  Files          77       77              
  Lines        3394     3409      +15     
==========================================
+ Hits         3221     3234      +13     
- Misses        173      175       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@cortadocodes cortadocodes requested a review from thclark February 5, 2024 14:48
@cortadocodes cortadocodes changed the title Improve questions asking and event handling Make event handling faster and resilient to missing events Feb 5, 2024
@cortadocodes cortadocodes merged commit 2e6c23f into main Feb 5, 2024
4 of 7 checks passed
@cortadocodes cortadocodes deleted the improve-questions-asking-and-event-handling branch February 5, 2024 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants