Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KafkaSinkCluster: Support new consumer protocol #1825

Merged
merged 1 commit into from
Nov 21, 2024

Conversation

rukai
Copy link
Member

@rukai rukai commented Nov 20, 2024

Kafka is introducing a new consumer group protocol, at the moment nothing uses it, but having now researched it its fairly straightforward to implement, so lets do so.

Details:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-848:+The+Next+Generation+of+the+Consumer+Rebalance+Protocol#KIP848:TheNextGenerationoftheConsumerRebalanceProtocol-ConsumerGroupHeartbeatAPI

But the TL;DR is just that there is a new message type ConsumerGroupHeartbeat that we need to route to the group coordinator.

To actually test the message type we need to set a broker config to enable this functionality as a kind of "early access", since we already have a kafka 3.9 specific test case, I added this config to the kakfa 3.9 test and added a specific test case to the 3.9 test, to test the new consumer protocol.

@rukai rukai force-pushed the consumer_protocol branch 2 times, most recently from 69c32db to c4178d6 Compare November 20, 2024 00:54
Copy link

codspeed-hq bot commented Nov 20, 2024

CodSpeed Performance Report

Merging #1825 will not alter performance

Comparing rukai:consumer_protocol (4f6169a) with main (229c9ce)

Summary

✅ 38 untouched benchmarks

@rukai rukai marked this pull request as ready for review November 20, 2024 01:49
@rukai rukai force-pushed the consumer_protocol branch from c4178d6 to 4f6169a Compare November 20, 2024 04:19
@rukai rukai merged commit 6d73da1 into shotover:main Nov 21, 2024
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants