Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/configurable_backlog #310

Merged
merged 3 commits into from
May 21, 2024

Conversation

tom-binary
Copy link
Contributor

Allow Emitters to request a specific queue length, via max_len. This is then used when deciding when to pause or clean up, and allows shorter/longer backlog than the 10k default.

To make this useful, the cleanup via XTRIM now applies the cleanup in batches, to avoid locking the server for extended amounts of time on larger streams. It also applies exact cleanup if the approximate cleanup couldn't find anything to remove and we're blocked waiting for progress, so that smaller streams can be used without deadlock.

- we have lagged too far behind
- or could not use approximate algorithm to remove enough items to be useful
@tom-binary tom-binary merged commit a58ea7b into deriv-com:master May 21, 2024
1 check passed
@tom-binary tom-binary deleted the feature/configurable_backlog branch May 21, 2024 04:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant