Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(query): improve build keys state #13004

Merged
merged 18 commits into from
Sep 27, 2023
Merged

chore(query): improve build keys state #13004

merged 18 commits into from
Sep 27, 2023

Conversation

Dousir9
Copy link
Member

@Dousir9 Dousir9 commented Sep 25, 2023 •
edited
Loading

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Use serialize_size to get the exact memory bytes required by StringColumn to avoid the resize and grow operations of Vec.
Benchmark: M1 Pro(10C32GB): TPCH-SF100 Q1:
15.4s(v1.2.130) -> 14.3s(PR)

Perf before:
截屏2023-09-26 23 43 40

Perf after:
截屏2023-09-26 23 45 39

  • Closes #issue

This change is Reviewable

@vercel
Copy link

vercel bot commented Sep 25, 2023 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
databend ⬜️ Ignored (Inspect) Visit Preview Sep 27, 2023 3:03am

@BohuTANG BohuTANG added the ci-cloud Build docker image for cloud test label Sep 25, 2023
@github-actions
Copy link
Contributor

Docker Image for PR

  • tag: pr-13004-cbc04b8

note: this image tag is only available for internal use,
please check the internal doc for more details.

@Dousir9 Dousir9 marked this pull request as ready for review September 26, 2023 01:40
@github-actions github-actions bot added the pr-chore this PR only has small changes that no need to record, like coding styles. label Sep 26, 2023
@BohuTANG BohuTANG removed the ci-cloud Build docker image for cloud test label Sep 26, 2023
@BohuTANG BohuTANG added the ci-cloud Build docker image for cloud test label Sep 26, 2023
@github-actions
Copy link
Contributor

Docker Image for PR

  • tag: pr-13004-b140b2f

note: this image tag is only available for internal use,
please check the internal doc for more details.

@BohuTANG BohuTANG merged commit 84b0451 into databendlabs:main Sep 27, 2023
58 checks passed
andylokandy pushed a commit to andylokandy/databend that referenced this pull request Nov 27, 2023
@Dousir9 @andylokandy
chore(query): improve build keys state (databendlabs#13004)
9ed8bdc 
* improve serialize_column_binary

* improve serialize_column_binary

* improve serialize string

* fix data_size

* add #Safety comments

* unsafe serialize_column_binary

* add some comments

* group by other_columns

* refine other_columns

* add Safety

* make lint

* remove unsafe

* add comments

* refine unsafe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BohuTANG BohuTANG BohuTANG left review comments

@sundy-li sundy-li sundy-li approved these changes

@zhang2014 zhang2014 Awaiting requested review from zhang2014

@andylokandy andylokandy Awaiting requested review from andylokandy

Assignees
No one assigned
Labels
ci-cloud Build docker image for cloud test pr-chore this PR only has small changes that no need to record, like coding styles.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Dousir9 @BohuTANG @sundy-li