Ann Benchmark threw error running SIFT

[pauldintel]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version: 2.3.1
- Deployment mode(standalone or cluster): Standalone
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2): pymilvus
- OS(Ubuntu or CentOS): Ubuntu 22.04
- CPU/Memory: 16 vCPU, 64GB
- GPU: 
- Others:

Current Behavior

While running ANN benchmark ended up with error :
:~/ann-benchmarks$ sudo python3 run.py --dataset sift-128-euclidean --algorithm milvus --local
2023-09-27 22:15:01,502 - annb - INFO - running only milvus
Traceback (most recent call last):
File "/home/ubuntu/ann-benchmarks/run.py", line 7, in
main()
File "/home/ubuntu/ann-benchmarks/ann_benchmarks/main.py", line 341, in main
raise Exception("Nothing to run")
Exception: Nothing to run

Expected Behavior

Run without error and show results

Steps To Reproduce

sudo python3 run.py --dataset sift-128-euclidean --algorithm milvus

Milvus Log

No response

Anything else?

No response

[yanliang567]

it seems that not only milvus but also the other engines are not working. I think there are a few issues in the ann benchmark.
@wangting0128 could you please help to take a look?

/assign @wangting0128
/unassign

[wangting0128]

Please try executing the following command to see if it works:
sudo python3 run.py --dataset glove-25-angular --algorithm 'Milvus(Knowhere)'

[yanliang567]

/assign @pauldintel

[pauldintel]

thanks @wangting0128 . this works but how do I change connection to remote node ?
I tried below modification at module.py
import milvus
self._client = milvus.Milvus(host='IPaddress', port='19530', try_connect=True, pre_ping=False)
but it doesn't work.

[wangting0128]

thanks @wangting0128 . this works but how do I change connection to remote node ? I tried below modification at module.py import milvus self._client = milvus.Milvus(host='IPaddress', port='19530', try_connect=True, pre_ping=False) but it doesn't work.

You need to rewrite the entire module.py file using the pymilvus package to support the function of connecting to remote milvus

[yanliang567]

@pauldintel I'd suggest you use https://github.com/zilliztech/VectorDBBench if you are interested in Milvus service performance instead the Ann search engine.

[yanliang567]

@pauldintel I'd suggest you use https://github.com/zilliztech/VectorDBBench if you are interested in Milvus service performance instead of in Ann search engine.

[pauldintel]

sure @yanliang567 - I will git it a shot to try Milvus from there .. any chande @wangting0128 to update the code from knowwhere to pymilvus ? looks like the Dockerfile using 7 months old Milvus package that is no more as standard .

[yanliang567]

sure @yanliang567 - I will git it a shot to try Milvus from there .. any chande @wangting0128 to update the code from knowwhere to pymilvus ? looks like the Dockerfile using 7 months old Milvus package that is no more as standard .

No, pymilvus is working only for milvus.

[pauldintel]

sure @yanliang567 - I will git it a shot to try Milvus from there .. any chande @wangting0128 to update the code from knowwhere to pymilvus ? looks like the Dockerfile using 7 months old Milvus package that is no more as standard .

No, pymilvus is working only for milvus.

Okay thanks ... so how accurate will be knowhere benchmarking ? does it close to Milvus project or different ? @yanliang567 and if the remote connect option is not available through knowhere then only option to run benchmark on standalone machine correct ?

[yanliang567]

knowhere is the engine in Milvus, you can run it as a lib, and milvus is a vector db system. thats the difference. If you want to knowhere the system performance, you may run https://github.com/zilliztech/VectorDBBench

[pauldintel]

sorry @yanliang567 . My plan is to benchmark Milvus as vector DB so eventually this will benchmark knowhere as engine too correct ? I am stuck with VectorDBBench for different issue on library ...

[yanliang567]

correct. please free to open an issue in the VectorDBBench repo if you any issue. I'd close this one if you agree

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.