refactor(query): improve hash join #12928
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
Docker Image for PR
|
Docker Image for PR
|
Dousir9
added
ci-benchmark
Docker Image for PR
|
Dousir9
added
ci-benchmark
Dousir9
added
ci-benchmark
Docker Image for PR
|
sundy-li
reviewed
Sep 28, 2023
Dousir9
added
ci-benchmark
Docker Image for PR
|
sundy-li
approved these changes
Oct 8, 2023
zhang2014
approved these changes
Oct 8, 2023
andylokandy
pushed a commit
to andylokandy/databend
that referenced
this pull request
Nov 27, 2023
* improve hash join * improve concat * improve take_string and add take_boolean * fix * improve concat * improve concat_string_types * improve take * improve filter * update * remove get_function_context * improve settings * allow too_many_arguments * merge * merge * refine primitive comments * refine * refine * refine take_compact * fix take_compact * add safety comment * fix take_compact_string * refine: use extend from iter and get_unchecked_mut * refine concat_primitive_types * reduce pr size * reduce pr size
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/
Summary
Summary about this PR
Improve the
kernels(take.rs,take_chunks.rs,take_compact.rs,concat.rs,filter.rs) and reduce memory usage:StringColumn, we first scaniterand calculate the space required by it. This can reduce the memory usage ofStringColumnand avoid theresizeandgrowoperations ofVec;StringColumn, we addstring_items_bufto avoid allocating memory frequently inkernels.ctx.get_function_context()andctx.get_settings()have a certain amount of overhead, so we should call them as little as possible when building pipelines.The ci-benchmark TPC-H standalone results:
This change is