enhance: avoid too many gorountines when calc distance

[CocaineCong]

the pr is resubmit for this #32776 pr

sorry for my mistake to close the pr #32776 and I can't reopen it again... so I have to resubmit this new pr 😭

I'm not sure if that's the right way to modify it ?

or use channel to control the number of goroutines instead of ants pool, like following:

bufSize := int(leftNum / 3)
ch := make(chan struct{}, bufSize)
for i := int64(0); i < leftNum; i++ {
waitGroup.Add(1)
ch <- struct{}{}
go func(i int64) {
	defer waitGroup.Done()
	CalcFFBatch(dim, left, i, right, metricUpper, &distArray)
	<-ch
    }(i)
}

[mergify]

@CocaineCong Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

[chasingegg]

Thanks for your contribution! It will be nice of you to show some performance numbers under very large batch sizes between old way with the new approach and add it to UT

[chasingegg]

Refer to milvus pkg/util/conc/pool.go, where wraps the ants.pool, you could use the pool by referring to other usages.
Also make this pool as static, not init it every time when we compute distances.

[CocaineCong]

ok, i got it

[CocaineCong]

hey, @chasingegg last week I was used sync.Once to make the pool init once. just like that:

var (
	calcPool         *conc.Pool[any]
	calcPoolInitOnce = new(sync.Once)
)

func initCalcPool() {
	calcPool = conc.NewDefaultPool[any]()
}

func GetCalcPool() *conc.Pool[any] {
	calcPoolInitOnce.Do(initCalcPool)
	return calcPool
}

but I find it will going deep into a dead cycle when I run the unit test.

I haven't found the reason for the dead cycle, so I have to make the new pool first in every time.

[chasingegg]

Can I ask how can I reproduce your dead cycle problem? which unit test?

[CocaineCong]

Can I ask how can I reproduce your dead cycle problem? which unit test?

You can reproduce it by following these steps：

1. copy the sync.Once code to make the pool init once.

2. replace pool := conc.NewDefaultPool[any]() to pool := GetCalcPool()

3. run the ut where in pkg/util/distance/calc_distance_test.go:155 , and find will go to the dead cycle.

4. I'm guessing this would be a problem caused by me, so I'm still debuging this problem. But I've been so busy these past few days that I've procrastinated a lot. I will find out the reason why this weekend.

[chasingegg]

Thanks! I will also look into it and hopefully could reproduce the problem

[mergify]

@CocaineCong Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

[mergify]

@CocaineCong E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

[mergify]

@CocaineCong E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

[chasingegg]

/run-cpu-e2e

[CocaineCong]

@chasingegg PTAL

[chasingegg]

@chasingegg PTAL

Commented

[mergify]

@CocaineCong ut workflow job failed, comment rerun ut can trigger the job again.

[CocaineCong]

hey, @chasingegg I found the reason why there is a dead loop.
If we use sync.Once for lazy loading, it will be released after the first call and then block continuously.
Therefore, there are three solutions:

• one is not to release, but it may cause memory issues.
• two is initialize a pool every time when it goes in.
• three is don't make changes, keep the status quo.

[mergify]

@CocaineCong Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

[CocaineCong]

@chasingegg PTAL.

[mergify]

@CocaineCong E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: CocaineCong
To complete the pull request process, please assign wxyucs after the PR has been reviewed.
You can assign the PR to them by writing /assign @wxyucs in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• pkg/util/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment