Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance: [cherry-pick] Move compactor into sub package #34098

Merged
merged 3 commits into from
Jun 25, 2024

Conversation

XuanYang-cn
Copy link
Contributor

@XuanYang-cn XuanYang-cn commented Jun 24, 2024

This PR consists of the following commits:

See also: #32451, #33547, #33998, #31679
pr: #32198, #33554, #33997, #32612

This PR consists of the following commits:

- enhance: Tidy compactor and remove dup codes (milvus-io#32198)
- fix: Fix l0 compactor may cause DN from OOM (milvus-io#33554)
- enhance: Add deltaRowCount in l0 compaction (milvus-io#33997)
- enhance: enable stream writer in compactions (milvus-io#32612)

See also: milvus-io#32451, milvus-io#33547, milvus-io#33998, milvus-io#31679
pr: milvus-io#32198, milvus-io#33554, milvus-io#33997, milvus-io#32612

Signed-off-by: yangxuan <[email protected]>
@sre-ci-robot sre-ci-robot added area/compilation size/XXL Denotes a PR that changes 1000+ lines. labels Jun 24, 2024
Copy link
Contributor

mergify bot commented Jun 24, 2024

@XuanYang-cn Please associate the related pr of master to the body of your Pull Request. (eg. “pr: #”)

@mergify mergify bot added the kind/enhancement Issues or changes related to enhancement label Jun 24, 2024
@XuanYang-cn
Copy link
Contributor Author

See also: #34085

Copy link
Contributor

mergify bot commented Jun 24, 2024

@XuanYang-cn ut workflow job failed, comment rerun ut can trigger the job again.

@mergify mergify bot added the ci-passed label Jun 24, 2024
Copy link

codecov bot commented Jun 24, 2024

Codecov Report

Attention: Patch coverage is 75.83466% with 152 lines in your changes missing coverage. Please review.

Project coverage is 81.71%. Comparing base (22e6807) to head (b1dbbd4).
Report is 3 commits behind head on 2.4.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##              2.4   #34098      +/-   ##
==========================================
- Coverage   81.80%   81.71%   -0.09%     
==========================================
  Files        1005     1012       +7     
  Lines      130104   129741     -363     
==========================================
- Hits       106435   106022     -413     
- Misses      19676    19705      +29     
- Partials     3993     4014      +21     
Files Coverage Δ
internal/datanode/compaction_executor.go 93.39% <100.00%> (ø)
internal/datanode/data_node.go 82.20% <ø> (ø)
internal/datanode/io/binlog_io.go 85.71% <100.00%> (-14.29%) ⬇️
internal/datanode/writebuffer/insert_buffer.go 100.00% <100.00%> (ø)
internal/datanode/data_sync_service.go 77.04% <66.66%> (+9.16%) ⬆️
internal/datanode/services.go 88.32% <83.33%> (-0.04%) ⬇️
internal/datacoord/compaction_view.go 88.37% <0.00%> (-1.04%) ⬇️
internal/storage/serde.go 88.01% <90.90%> (+0.79%) ⬆️
internal/datanode/compaction/l0_compactor.go 90.74% <76.47%> (ø)
internal/datanode/io/io_pool.go 71.92% <58.33%> (-9.89%) ⬇️
... and 3 more

... and 214 files with indirect coverage changes

@czs007
Copy link
Collaborator

czs007 commented Jun 25, 2024

/approve
/lgtm

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: czs007, XuanYang-cn

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot merged commit a33b686 into milvus-io:2.4 Jun 25, 2024
13 of 15 checks passed
@XuanYang-cn XuanYang-cn deleted the c/compaction-refinement branch June 25, 2024 03:29
@XuanYang-cn
Copy link
Contributor Author

See also: #33435

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved area/compilation ci-passed dco-passed DCO check passed. kind/branch-feature kind/enhancement Issues or changes related to enhancement lgtm size/XXL Denotes a PR that changes 1000+ lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants