Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Route ListTransactions request #1797

Merged
merged 1 commit into from
Nov 6, 2024
Merged

Conversation

rukai
Copy link
Member

@rukai rukai commented Nov 3, 2024

This PR implements routing for ListTransactions requests by applying the same routing logic to ListTransactions as we did for ListGroups in #1790

However, I found that while the java driver deduplicates entries in ListGroups responses it does not do the same for ListTransactions responses.

So I had to alter the logic in split_request_by_routing_to_all_brokers to avoid duplicates.
As a result ListGroups is actually more efficient as we better distribute the load across shotover instances by avoiding querying the same node twice.

The comment on split_request_by_routing_to_all_brokers explains the new logic pretty well:

/// Route broadcast requests to all brokers split across all shotover nodes.
/// That is, each shotover node in a rack will deterministically be assigned a portion of the rack to route the request to.
/// If a shotover node is the only node in its rack it will route to all kafka brokers in the rack.
/// When combined with a client that is routing this request to all shotover nodes, the request will reach each kafka broker exactly once.

Copy link

codspeed-hq bot commented Nov 3, 2024

CodSpeed Performance Report

Merging #1797 will not alter performance

Comparing rukai:list_transactions (fd59ddb) with main (f81fbdb)

Summary

✅ 38 untouched benchmarks

@rukai rukai force-pushed the list_transactions branch 7 times, most recently from 8c4379d to a5a9d0d Compare November 4, 2024 04:01
@rukai rukai marked this pull request as ready for review November 4, 2024 04:01
@rukai rukai force-pushed the list_transactions branch from a5a9d0d to c7a35db Compare November 4, 2024 05:00
@rukai rukai force-pushed the list_transactions branch from c7a35db to b670eb9 Compare November 5, 2024 23:02
@rukai rukai force-pushed the list_transactions branch from b670eb9 to fd59ddb Compare November 5, 2024 23:04
@rukai rukai enabled auto-merge (squash) November 5, 2024 23:54
@rukai rukai merged commit 79f47ab into shotover:main Nov 6, 2024
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants