Reduce go routine and thread numbers #27457

xiaofan-luan · 2023-09-30T09:03:22Z

fix #27456

sre-ci-robot · 2023-09-30T09:03:37Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: xiaofan-luan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [xiaofan-luan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mergify · 2023-09-30T09:17:46Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2023-09-30T09:23:02Z

@xiaofan-luan ut workflow job failed, comment rerun ut can trigger the job again.

mergify · 2023-09-30T12:22:49Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2023-09-30T12:25:56Z

@xiaofan-luan ut workflow job failed, comment rerun ut can trigger the job again.

mergify · 2023-09-30T13:23:04Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

pkg/util/conc/future.go

yah01 · 2023-09-30T14:28:55Z

pkg/util/conc/future.go

@@ -71,6 +71,11 @@ func (future *Future[T]) Err() error {
 	return future.err
 }

+func (future *Future[T]) SetErr(err error) {


This method may be abused to set error of future multiple times, but the channel can't be closed twice

yah01 · 2023-09-30T14:29:34Z

pkg/util/paramtable/component_param.go

@@ -1777,7 +1776,7 @@ Max read concurrency must greater than or equal to 1, and less than or equal to
 	p.MaxGroupNQ = ParamItem{
 		Key:          "queryNode.grouping.maxNQ",
 		Version:      "2.0.0",
-		DefaultValue: "1000",
+		DefaultValue: "100",


@liliu-z does this fit Knowhere?

this is to avoid merge cost latency up.

@yah01 big NQ will introduce big latency in nowadays search arch. Let’s reduce it for now until we can find some other solutions

mergify · 2023-10-01T02:57:49Z

@xiaofan-luan ut workflow job failed, comment rerun ut can trigger the job again.

mergify · 2023-10-01T03:16:32Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2023-10-01T13:46:01Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

codecov · 2023-10-01T16:59:36Z

Codecov Report

Merging #27457 (9001b50) into master (0c69f48) will decrease coverage by 0.31%.
Report is 333 commits behind head on master.
The diff coverage is 86.44%.

❗ Current head 9001b50 differs from pull request most recent head 0156bc0. Consider uploading reports for the commit 0156bc0 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #27457      +/-   ##
==========================================
- Coverage   81.78%   81.47%   -0.31%     
==========================================
  Files         833      804      -29     
  Lines      117802   114194    -3608     
==========================================
- Hits        96340    93040    -3300     
+ Misses      18265    18024     -241     
+ Partials     3197     3130      -67

Files	Coverage Δ
internal/querynodev2/segments/collection.go	`97.58% <100.00%> (ø)`
internal/querynodev2/segments/plan.go	`70.17% <100.00%> (-4.39%)`	⬇️
internal/querynodev2/segments/search.go	`89.09% <100.00%> (+4.47%)`	⬆️
internal/querynodev2/tasks/search_task.go	`42.17% <ø> (ø)`
pkg/metrics/metrics.go	`100.00% <ø> (ø)`
pkg/metrics/querynode_metrics.go	`100.00% <ø> (ø)`
pkg/util/conc/future.go	`100.00% <100.00%> (ø)`
pkg/util/indexparams/disk_index_params.go	`87.19% <ø> (ø)`
pkg/util/paramtable/component_param.go	`98.08% <33.33%> (-0.05%)`	⬇️
internal/querynodev2/segments/retrieve.go	`94.44% <94.23%> (+9.75%)`	⬆️
... and 1 more

... and 230 files with indirect coverage changes

mergify · 2023-10-01T17:18:14Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2023-10-01T17:19:55Z

@xiaofan-luan ut workflow job failed, comment rerun ut can trigger the job again.

mergify · 2023-10-02T03:28:54Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

xiaofan-luan · 2023-10-02T03:42:30Z

/run-cpu-e2e

mergify · 2023-10-02T04:27:33Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

xiaofan-luan · 2023-10-02T04:42:36Z

/run-cpu-e2e

mergify · 2023-10-02T05:27:43Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2023-10-02T10:05:03Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2023-10-08T03:01:51Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2023-10-08T09:12:14Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

xiaofan-luan · 2023-10-08T09:38:06Z

/run-cpu-e2e

mergify · 2023-10-08T10:30:37Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

xiaofan-luan · 2023-10-08T11:59:59Z

/run-cpu-e2e

mergify · 2023-10-08T12:54:15Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2023-10-08T13:50:10Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2023-10-09T13:19:50Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

xiaofan-luan · 2023-10-09T14:10:43Z

/run-cpu-e2e

mergify · 2023-10-09T15:20:24Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Signed-off-by: xiaofan-luan <[email protected]>

mergify · 2023-11-01T09:31:15Z

@xiaofan-luan E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2023-11-16T08:01:37Z

@xiaofan-luan

Invalid PR Title Format Detected

Your PR submission does not adhere to our required standards. To ensure clarity and consistency, please meet the following criteria:

Title Format: The PR title must begin with one of these prefixes:

feat: for introducing a new feature.
enhance: for improvements to existing functionality.
fix: for bug fixes.

Description Requirement: The PR must include a non-empty description, detailing the changes and their impact.

Required Title Structure:

[Type]: [Description of the PR]

Where Type is one of feat, enhance, or fix.

Example:

feat: Enhance search functionality within the user interface

Please review and update your PR to comply with these guidelines.

stale · 2023-12-16T17:17:54Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

sre-ci-robot added approved size/L Denotes a PR that changes 100-499 lines. labels Sep 30, 2023

sre-ci-robot requested review from aoiasd and godchen0212 September 30, 2023 09:03

mergify bot added the dco-passed DCO check passed. label Sep 30, 2023

xiaofan-luan force-pushed the ReduceGoRoutine branch from cb00127 to 70441ea Compare September 30, 2023 12:08

sre-ci-robot added size/XL Denotes a PR that changes 500-999 lines. and removed size/L Denotes a PR that changes 100-499 lines. labels Sep 30, 2023

xiaofan-luan force-pushed the ReduceGoRoutine branch from 70441ea to 0b2a943 Compare September 30, 2023 13:09

yah01 reviewed Sep 30, 2023

View reviewed changes

xiaofan-luan force-pushed the ReduceGoRoutine branch from 0b2a943 to 639714d Compare October 1, 2023 02:40

xiaofan-luan force-pushed the ReduceGoRoutine branch from 639714d to e7ebb12 Compare October 1, 2023 13:11

xiaofan-luan force-pushed the ReduceGoRoutine branch from e7ebb12 to 6e45af9 Compare October 1, 2023 16:58

xiaofan-luan force-pushed the ReduceGoRoutine branch from 6e45af9 to 02fec89 Compare October 2, 2023 02:35

xiaofan-luan force-pushed the ReduceGoRoutine branch from 02fec89 to ea59f23 Compare October 2, 2023 09:23

xiaofan-luan force-pushed the ReduceGoRoutine branch from ea59f23 to b81db02 Compare October 8, 2023 02:34

xiaofan-luan force-pushed the ReduceGoRoutine branch from b81db02 to 9cef2cc Compare October 8, 2023 08:23

xiaofan-luan force-pushed the ReduceGoRoutine branch from 9cef2cc to b4fae5f Compare October 8, 2023 12:57

xiaofan-luan force-pushed the ReduceGoRoutine branch from b4fae5f to 9001b50 Compare October 9, 2023 12:21

Reduce go routine and thread numbers

0156bc0

Signed-off-by: xiaofan-luan <[email protected]>

xiaofan-luan force-pushed the ReduceGoRoutine branch from 9001b50 to 0156bc0 Compare November 1, 2023 08:29

mergify bot added the do-not-merge/invalid-pr-format label Nov 16, 2023

stale bot added the stale indicates no udpates for 30 days label Dec 16, 2023

stale bot closed this Dec 23, 2023

Reduce go routine and thread numbers #27457

Reduce go routine and thread numbers #27457

Conversation

xiaofan-luan commented Sep 30, 2023

sre-ci-robot commented Sep 30, 2023

mergify bot commented Sep 30, 2023

mergify bot commented Sep 30, 2023

mergify bot commented Sep 30, 2023

mergify bot commented Sep 30, 2023

mergify bot commented Sep 30, 2023

yah01 Sep 30, 2023

Choose a reason for hiding this comment

yah01 Sep 30, 2023

Choose a reason for hiding this comment

xiaofan-luan Oct 1, 2023

Choose a reason for hiding this comment

liliu-z Oct 1, 2023 • edited Loading

Choose a reason for hiding this comment

mergify bot commented Oct 1, 2023

mergify bot commented Oct 1, 2023

mergify bot commented Oct 1, 2023

codecov bot commented Oct 1, 2023 • edited Loading

Codecov Report

mergify bot commented Oct 1, 2023

mergify bot commented Oct 1, 2023

mergify bot commented Oct 2, 2023

xiaofan-luan commented Oct 2, 2023

mergify bot commented Oct 2, 2023

xiaofan-luan commented Oct 2, 2023

mergify bot commented Oct 2, 2023

mergify bot commented Oct 2, 2023

mergify bot commented Oct 8, 2023

mergify bot commented Oct 8, 2023

xiaofan-luan commented Oct 8, 2023

mergify bot commented Oct 8, 2023

xiaofan-luan commented Oct 8, 2023

mergify bot commented Oct 8, 2023

mergify bot commented Oct 8, 2023

mergify bot commented Oct 9, 2023

xiaofan-luan commented Oct 9, 2023

mergify bot commented Oct 9, 2023

mergify bot commented Nov 1, 2023

mergify bot commented Nov 16, 2023

stale bot commented Dec 16, 2023

liliu-z Oct 1, 2023 •

edited

Loading

codecov bot commented Oct 1, 2023 •

edited

Loading