Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: [benchmark][standalone] build IVF_FLAT index failed rpc error: code = DeadlineExceeded desc = context deadline exceeded #32400

Closed
1 task done
wangting0128 opened this issue Apr 18, 2024 · 6 comments
Assignees
Labels
2.4-features kind/bug Issues or changes related a bug test/benchmark benchmark test triage/accepted Indicates an issue or PR is ready to be actively worked on.
Milestone

Comments

@wangting0128
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: test_hybrid_search_serial_ivf_flat_hnsw_standalone
- Deployment mode(standalone or cluster): standalone
- MQ type(rocksmq, pulsar or kafka): rocksmq   
- SDK version(e.g. pymilvus v2.0.0rc2): 2.4.0rc66
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

argo task: multi-vector-based-scene1-f8pw5
test case name: test_hybrid_search_serial_ivf_flat_hnsw_standalone

server:
init stats

NAME                                                              READY   STATUS                            RESTARTS        AGE     IP              NODE         NOMINATED NODE   READINESS GATES
multi-vector-based-scene1-f8pw5-etcd-0                            1/1     Running                           0               2m33s   10.104.15.16    4am-node20   <none>           <none>
multi-vector-based-scene1-f8pw5-milvus-standalone-65cf6f86z2s9q   1/1     Running                           0               2m33s   10.104.26.10    4am-node32   <none>           <none>
multi-vector-based-scene1-f8pw5-minio-6f9756b97c-mfzmr            1/1     Running                           0               2m33s   10.104.15.19    4am-node20   <none>           <none>

after testing

NAME                                                              READY   STATUS                            RESTARTS        AGE     IP              NODE         NOMINATED NODE   READINESS GATES
multi-vector-based-scene1-f8pw5-etcd-0                            1/1     Running                           0               24h     10.104.15.16    4am-node20   <none>           <none>
multi-vector-based-scene1-f8pw5-milvus-standalone-65cf6f86z2s9q   1/1     Running                           3 (17h ago)     24h     10.104.26.10    4am-node32   <none>           <none>
multi-vector-based-scene1-f8pw5-minio-6f9756b97c-mfzmr            1/1     Running                           0               24h     10.104.15.19    4am-node20   <none>           <none>
截屏2024-04-17 11 29 49

client pod name: multi-vector-based-scene1-f8pw5-19328334
client log:
截屏2024-04-17 11 24 41

Expected Behavior

No response

Steps To Reproduce

1. create a collection, 8 fields: "id", "float_vector", "float_vector_1", "int64_1", "int64_2", "float_1", "double_1", "varchar_1"
2. build index
   IVF_FLAT: float_vector
   HNSW: float_vector_1
   INVERTED: "int64_1", "int64_2", "float_1", "double_1", "varchar_1"
3. insert 25m data
4. flush collection
5. build index again with the same params <- failed

Milvus Log

No response

Anything else?

server config:
截屏2024-04-17 11 29 28

@wangting0128 wangting0128 added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. test/benchmark benchmark test 2.4-features labels Apr 18, 2024
@wangting0128 wangting0128 added this to the 2.4.0 milestone Apr 18, 2024
@xiaofan-luan
Copy link
Collaborator

/assign @xiaocai2333

@xiaofan-luan
Copy link
Collaborator

/assign @xiaocai2333

@yanliang567
Copy link
Contributor

/unassign

@yanliang567 yanliang567 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Apr 18, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.0, 2.4.1 Apr 18, 2024
@xiaocai2333
Copy link
Contributor

It seems that RootCoord crashed due to some other reasons.

@yanliang567 yanliang567 modified the milestones: 2.4.1, 2.4.2 May 7, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.2, 2.4.3, 2.4.4 May 24, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.4, 2.4.5 Jun 5, 2024
@yanliang567 yanliang567 removed this from the 2.4.5 milestone Jun 26, 2024
@yanliang567 yanliang567 added this to the 2.4.6 milestone Jun 26, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.6, 2.4.7 Jul 19, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.7, 2.4.8 Aug 12, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.8, 2.4.10 Aug 19, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.10, 2.4.11 Sep 5, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.11, 2.4.12 Sep 18, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.12, 2.4.13 Sep 27, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.13, 2.4.14 Oct 15, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.14, 2.4.16 Nov 14, 2024
@yanliang567 yanliang567 modified the milestones: 2.4.16, 2.4.17, 2.4.18 Nov 21, 2024
@xiaocai2333
Copy link
Contributor

@wangting0128 Does this issue reproduce recently? Can we close it?

@wangting0128
Copy link
Contributor Author

not reproduce recently, close it now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.4-features kind/bug Issues or changes related a bug test/benchmark benchmark test triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

No branches or pull requests

5 participants