Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: No etcd Connection with Docker-Compose Standalone #31590

Closed
1 task done
hno2 opened this issue Mar 25, 2024 · 4 comments
Closed
1 task done

[Bug]: No etcd Connection with Docker-Compose Standalone #31590

hno2 opened this issue Mar 25, 2024 · 4 comments
Assignees
Labels
kind/bug Issues or changes related a bug stale indicates no udpates for 30 days triage/accepted Indicates an issue or PR is ready to be actively worked on.

Comments

@hno2
Copy link

hno2 commented Mar 25, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version:milvus:v2.4.0-rc.1-gpu
- Deployment mode(standalone or cluster):standalone/docker-compose
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2): Not Relevant
- OS(Ubuntu or CentOS): Ubuntu
- CPU/Memory:  AMD Ryzen Threadripper PRO 7975WX 32-Cores/256GB
- GPU: 4090 via nvidia-docker
- Others:

Current Behavior

When starting milvus standalone via the released milvus-standalone-docker-compose-gpu.yml, the milvus-standalone container crashes after a couple of minutes due to a failing etcd connection.
Full log attached below, the standalone throws several errors of `

milvus-standalone  | {"level":"warn","ts":"2024-03-25T09:40:56.830Z","logger":"etcd-client","caller":"[email protected]/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0015b2fc0/etcd:2379","attempt":0,"error":"rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: PROTOCOL_ERROR"}

The etcd container can be pinged from the standalone container, and the etcd service is up and running and values can be put via etcdctl.

Expected Behavior

The docker-compose file should just work out of the box with no configuration.

Steps To Reproduce

1. Download the docker-compose of the latest release
2. Run it via `docker compose up`
3. etcd and minio work with healty healthcheck, milvus standalone crashed after a couple of minutes due to no connection to etcd.

Milvus Log

log.txt

Anything else?

No response

@hno2 hno2 added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Mar 25, 2024
@yanliang567
Copy link
Contributor

/assign @Presburger
please help to take a look
/unassign

@yanliang567 yanliang567 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Mar 27, 2024
Copy link

stale bot commented May 1, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

@stale stale bot added the stale indicates no udpates for 30 days label May 1, 2024
@stale stale bot closed this as completed May 12, 2024
@jdgiddings
Copy link

Having this exact issue, too. Any luck?

@yanliang567
Copy link
Contributor

Having this exact issue, too. Any luck?

this issue is about rc and we have already release v2.4.3, please retry. If it reproduced to you, feel free to file a new issue and offer error, logs and configs for investigations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Issues or changes related a bug stale indicates no udpates for 30 days triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

No branches or pull requests

4 participants