Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor kafka_int_tests to use connection creator helper #1495

Merged
merged 2 commits into from
Feb 23, 2024

Conversation

rukai
Copy link
Member

@rukai rukai commented Feb 22, 2024

The driving factor for this was the ability to abstract over connections created with TLS, TLS+SASL,plaintext+SASL or just plaintext.

In the future I expect this to also be useful for abstracting over different kafka drivers.

The consumer and producer APIs are completely abstracted and the tests no longer directly interact with rdkafka.
However the rdkafka admin API is still directly exposed since that has a lot more surface area.
We can wrap that up if and when we need to abstract over multiple kafka drivers.

@rukai rukai force-pushed the kafka_connection_builder branch from 80a1ca9 to aa15c88 Compare February 22, 2024 06:05
@rukai rukai requested a review from conorbros February 22, 2024 06:13
Copy link

2 benchmark regressed. 0 benchmark improved. Please check the benchmark workflow logs for full details: https://github.com/shotover/shotover-proxy/actions/runs/8000182500

                        change: [+5.8377% +22.807% +51.668%] (p = 0.04 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
kafka_codec/encode_all  time:   [2.5030 µs 2.5451 µs 2.5862 µs]
                        change: [+29.058% +35.012% +40.893%] (p = 0.00 < 0.05)
                        Performance has regressed.
--
  3 (3.00%) low mild
  6 (6.00%) high mild
  5 (5.00%) high severe
cassandra_codec/encode_system.local_result_v4_no_compression
                        time:   [32.973 µs 34.034 µs 34.994 µs]
                        change: [+26.121% +31.687% +37.149%] (p = 0.00 < 0.05)
                        Performance has regressed.

Copy link

1 benchmark regressed. 0 benchmark improved. Please check the benchmark workflow logs for full details: https://github.com/shotover/shotover-proxy/actions/runs/8010223586

  2 (2.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe
kafka_codec/decode_request_produce_drop
                        time:   [491.73 ns 523.86 ns 559.20 ns]
                        change: [+27.341% +33.654% +40.816%] (p = 0.00 < 0.05)
                        Performance has regressed.

@rukai rukai force-pushed the kafka_connection_builder branch from a36d881 to 90bdcb0 Compare February 22, 2024 22:11
Copy link

1 benchmark regressed. 0 benchmark improved. Please check the benchmark workflow logs for full details: https://github.com/shotover/shotover-proxy/actions/runs/8011642761

  1 (1.00%) low mild
  1 (1.00%) high mild
  3 (3.00%) high severe
kafka_codec/encode_request_list_offsets
                        time:   [762.93 ns 770.68 ns 777.65 ns]
                        change: [+21.384% +53.449% +110.44%] (p = 0.02 < 0.05)
                        Performance has regressed.

@rukai rukai force-pushed the kafka_connection_builder branch from 90bdcb0 to 22e3bee Compare February 22, 2024 22:40
@rukai rukai merged commit 74adab8 into shotover:main Feb 23, 2024
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants