Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cluster): support compression for query flight data #13934

Merged
merged 6 commits into from
Dec 7, 2023

Conversation

zhang2014
Copy link
Member

@zhang2014 zhang2014 commented Dec 6, 2023

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

feat(cluster): support compression for query flight data


This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Dec 6, 2023
@zhang2014 zhang2014 added the ci-benchmark Benchmark: run all test label Dec 6, 2023
@zhang2014 zhang2014 added ci-benchmark Benchmark: run all test and removed ci-benchmark Benchmark: run all test labels Dec 6, 2023
Copy link
Contributor

github-actions bot commented Dec 6, 2023

Docker Image for PR

  • tag: pr-13934-ea616b2

note: this image tag is only available for internal use,
please check the internal doc for more details.

Copy link
Contributor

github-actions bot commented Dec 6, 2023

@zhang2014 zhang2014 marked this pull request as ready for review December 6, 2023 13:55
@BohuTANG BohuTANG requested a review from sundy-li December 6, 2023 13:59
@BohuTANG BohuTANG added ci-benchmark Benchmark: run all test and removed ci-benchmark Benchmark: run all test labels Dec 6, 2023
Copy link
Contributor

github-actions bot commented Dec 6, 2023

Docker Image for PR

  • tag: pr-13934-137fea3

note: this image tag is only available for internal use,
please check the internal doc for more details.

Copy link
Contributor

github-actions bot commented Dec 6, 2023

@zhang2014 zhang2014 added ci-benchmark Benchmark: run all test and removed ci-benchmark Benchmark: run all test labels Dec 7, 2023
Copy link
Contributor

github-actions bot commented Dec 7, 2023

Docker Image for PR

  • tag: pr-13934-1b647cd

note: this image tag is only available for internal use,
please check the internal doc for more details.

Comment on lines +28 to +31
pub enum FlightCompression {
Lz4,
Zstd,
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub enum FlightCompression {
Lz4,
Zstd,
}
pub enum FlightCompression {
LZ4,
ZSTD,
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This display will the enum?
image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this's the warehouse of pr-13934-1b647cd version

Copy link
Member

@BohuTANG BohuTANG Dec 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the settings show, user may set as:

set query_flight_compression='Lz4';

not

set query_flight_compression='LZ4';

But possible values are:

possible_values: Some(vec!["None", "LZ4", "ZSTD"]),

Updated:
It's only a display issue, the query_flight_compression is case insensitive.

Copy link
Contributor

github-actions bot commented Dec 7, 2023

@zhang2014 zhang2014 enabled auto-merge December 7, 2023 13:20
@zhang2014 zhang2014 added this pull request to the merge queue Dec 7, 2023
Merged via the queue into databendlabs:main with commit 341ebe8 Dec 7, 2023
68 checks passed
@zhang2014 zhang2014 deleted the feat/flight_compression branch December 7, 2023 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-benchmark Benchmark: run all test pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: compression when data shuffle via flight RPC
3 participants