[Bug]: Standalone or datanode pod crash during rolling update

[zhuwenxing]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version:2.2.0-20231024-4be4557f --> master-20231023-8c605ca8-amd64
- Deployment mode(standalone or cluster): standalone
- MQ type(rocksmq, pulsar or kafka):  kafka  
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

[2023/10/24 07:34:57.385 +00:00] [WARN] [proxy/service.go:366] ["init Proxy server failed"] [error="attempt #0: stack trace: /go/src/github.com/milvus-io/milvus/pkg/tracer/stack_trace.go:51 github.com/milvus-io/milvus/pkg/tracer.StackTrace\n/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:491 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).Call\n/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:505 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).ReCall\n/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:107 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.wrapGrpcCall[...]\n/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:121 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.(*Client).GetComponentStates\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:37 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...].func1\n/go/src/github.com/milvus-io/milvus/pkg/util/retry/retry.go:44 github.com/milvus-io/milvus/pkg/util/retry.Do\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:61 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...]\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:82 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentHealthy[...]\n/go/src/github.com/milvus-io/milvus/internal/distributed/proxy/service.go:533 github.com/milvus-io/milvus/internal/distributed/proxy.(*Server).init: attempt #0: err: rpc error: code = Unimplemented desc = grpc: Decompressor is not installed for grpc-encoding \"zstd\": service unimplemented: unrecoverable error"]
[2023/10/24 07:34:57.386 +00:00] [ERROR] [components/proxy.go:53] ["Proxy starts error"] [error="attempt #0: stack trace: /go/src/github.com/milvus-io/milvus/pkg/tracer/stack_trace.go:51 github.com/milvus-io/milvus/pkg/tracer.StackTrace\n/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:491 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).Call\n/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:505 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).ReCall\n/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:107 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.wrapGrpcCall[...]\n/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:121 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.(*Client).GetComponentStates\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:37 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...].func1\n/go/src/github.com/milvus-io/milvus/pkg/util/retry/retry.go:44 github.com/milvus-io/milvus/pkg/util/retry.Do\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:61 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...]\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:82 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentHealthy[...]\n/go/src/github.com/milvus-io/milvus/internal/distributed/proxy/service.go:533 github.com/milvus-io/milvus/internal/distributed/proxy.(*Server).init: attempt #0: err: rpc error: code = Unimplemented desc = grpc: Decompressor is not installed for grpc-encoding \"zstd\": service unimplemented: unrecoverable error"] [stack="github.com/milvus-io/milvus/cmd/components.(*Proxy).Run\n\t/go/src/github.com/milvus-io/milvus/cmd/components/proxy.go:53\ngithub.com/milvus-io/milvus/cmd/roles.runComponent[...].func1\n\t/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:113"]
panic: attempt #0: stack trace: /go/src/github.com/milvus-io/milvus/pkg/tracer/stack_trace.go:51 github.com/milvus-io/milvus/pkg/tracer.StackTrace
/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:491 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).Call
/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:505 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).ReCall
/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:107 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.wrapGrpcCall[...]
/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:121 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.(*Client).GetComponentStates
/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:37 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...].func1
/go/src/github.com/milvus-io/milvus/pkg/util/retry/retry.go:44 github.com/milvus-io/milvus/pkg/util/retry.Do
/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:61 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...]
/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:82 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentHealthy[...]
/go/src/github.com/milvus-io/milvus/internal/distributed/proxy/service.go:533 github.com/milvus-io/milvus/internal/distributed/proxy.(*Server).init: attempt #0: err: rpc error: code = Unimplemented desc = grpc: Decompressor is not installed for grpc-encoding "zstd": service unimplemented: unrecoverable error

goroutine 300 [running]:
panic({0x4347a20, 0xc000625728})
	/usr/local/go/src/runtime/panic.go:987 +0x3bb fp=0xc00067bf58 sp=0xc00067be98 pc=0x1781c1b
github.com/milvus-io/milvus/cmd/roles.runComponent[...].func1()
	/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:114 +0x186 fp=0xc00067bfe0 sp=0xc00067bf58 pc=0x3d3d926
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc00067bfe8 sp=0xc00067bfe0 pc=0x17bb9a1
created by github.com/milvus-io/milvus/cmd/roles.runComponent[...]
	/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:100 +0x158

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

setup command

Containers:
  standalone:
    Container ID:  containerd://8625463c9f881c037945a2bffe0503687577fbed5b7beb313ccdcd14f292904e
    Image:         harbor.milvus.io/dockerhub/milvusdb/milvus:master-20231023-8c605ca8-amd64
    Image ID:      harbor.milvus.io/dockerhub/milvusdb/milvus@sha256:6469f9b1c2eb284dee7551905eb8189195538e3c18fdb095154ddfc7e2d5c600
    Ports:         19530/TCP, 9091/TCP
    Host Ports:    0/TCP, 0/TCP
    Args:
      /milvus/tools/run.sh
      milvus
      run
      standalone
      --run-with-subprocess
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Tue, 24 Oct 2023 15:34:57 +0800
      Finished:     Tue, 24 Oct 2023 15:34:59 +0800

failed job:
https://qa-jenkins.milvus.io/blue/organizations/jenkins/rolling_update_for_operator_test_simple/detail/rolling_update_for_operator_test_simple/144/pipeline
log:

artifacts-kafka-standalone-144-pytest-logs.tar.gz
artifacts-kafka-standalone-144-server-logs.tar.gz

Anything else?

No response

[yanliang567]

/assign @weiliu1031
/unassign

[zhuwenxing]

When using mixcoord deploy mode, the datanode will crash

2023/10/24 07:44:36.096 +00:00] [DEBUG] [sessionutil/session_util.go:249] ["Session connect to etcd success"]
[2023/10/24 07:44:36.096 +00:00] [DEBUG] [sessionutil/session_util.go:566] ["SessionUtil GetSessions"] [prefix=rootcoord] [key=rootcoord] [address=10.104.20.181:53100]
[2023/10/24 07:44:36.096 +00:00] [DEBUG] [sessionutil/session_util.go:566] ["SessionUtil GetSessions"] [prefix=rootcoord] [key=rootcoord-3] [address=10.104.20.181:53100]
[2023/10/24 07:44:36.096 +00:00] [DEBUG] [client/client.go:93] ["RootCoordClient GetSessions success"] [address=10.104.20.181:53100] [serverID=3]
[2023/10/24 07:44:36.098 +00:00] [WARN] [grpcclient/client.go:422] ["call received grpc error"] [clientRole=rootcoord] [error="rpc error: code = Unimplemented desc = grpc: Decompressor is not installed for grpc-encoding \"zstd\""]
[2023/10/24 07:44:36.098 +00:00] [ERROR] [grpcclient/client.go:411] ["retry func failed"] ["retry time"=0] [error="err: rpc error: code = Unimplemented desc = grpc: Decompressor is not installed for grpc-encoding \"zstd\": service unimplemented: unrecoverable error"] [stack="github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).call\n\t/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:411\ngithub.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).Call\n\t/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:489\ngithub.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).ReCall\n\t/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:505\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord/client.wrapGrpcCall[...]\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:107\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord/client.(*Client).GetComponentStates\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:121\ngithub.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...].func1\n\t/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:37\ngithub.com/milvus-io/milvus/pkg/util/retry.Do\n\t/go/src/github.com/milvus-io/milvus/pkg/util/retry/retry.go:44\ngithub.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...]\n\t/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:61\ngithub.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentHealthy[...]\n\t/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:82\ngithub.com/milvus-io/milvus/internal/distributed/datanode.(*Server).init\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:261\ngithub.com/milvus-io/milvus/internal/distributed/datanode.(*Server).Run\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:187\ngithub.com/milvus-io/milvus/cmd/components.(*DataNode).Run\n\t/go/src/github.com/milvus-io/milvus/cmd/components/data_node.go:53\ngithub.com/milvus-io/milvus/cmd/roles.runComponent[...].func1\n\t/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:113"]
[2023/10/24 07:44:36.098 +00:00] [WARN] [grpcclient/client.go:505] ["ClientBase Call grpc call get error"] [role=rootcoord] [address=10.104.20.181:53100] [error="stack trace: /go/src/github.com/milvus-io/milvus/pkg/tracer/stack_trace.go:51 github.com/milvus-io/milvus/pkg/tracer.StackTrace\n/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:491 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).Call\n/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:505 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).ReCall\n/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:107 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.wrapGrpcCall[...]\n/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:121 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.(*Client).GetComponentStates\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:37 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...].func1\n/go/src/github.com/milvus-io/milvus/pkg/util/retry/retry.go:44 github.com/milvus-io/milvus/pkg/util/retry.Do\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:61 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...]\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:82 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentHealthy[...]\n/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:261 github.com/milvus-io/milvus/internal/distributed/datanode.(*Server).init: attempt #0: err: rpc error: code = Unimplemented desc = grpc: Decompressor is not installed for grpc-encoding \"zstd\": service unimplemented: unrecoverable error"]
[2023/10/24 07:44:36.098 +00:00] [ERROR] [componentutil/componentutil.go:61] ["retry func failed"] ["retry time"=0] [error="stack trace: /go/src/github.com/milvus-io/milvus/pkg/tracer/stack_trace.go:51 github.com/milvus-io/milvus/pkg/tracer.StackTrace\n/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:491 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).Call\n/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:505 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).ReCall\n/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:107 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.wrapGrpcCall[...]\n/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:121 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.(*Client).GetComponentStates\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:37 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...].func1\n/go/src/github.com/milvus-io/milvus/pkg/util/retry/retry.go:44 github.com/milvus-io/milvus/pkg/util/retry.Do\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:61 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...]\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:82 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentHealthy[...]\n/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:261 github.com/milvus-io/milvus/internal/distributed/datanode.(*Server).init: attempt #0: err: rpc error: code = Unimplemented desc = grpc: Decompressor is not installed for grpc-encoding \"zstd\": service unimplemented: unrecoverable error"] [stack="github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...]\n\t/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:61\ngithub.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentHealthy[...]\n\t/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:82\ngithub.com/milvus-io/milvus/internal/distributed/datanode.(*Server).init\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:261\ngithub.com/milvus-io/milvus/internal/distributed/datanode.(*Server).Run\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:187\ngithub.com/milvus-io/milvus/cmd/components.(*DataNode).Run\n\t/go/src/github.com/milvus-io/milvus/cmd/components/data_node.go:53\ngithub.com/milvus-io/milvus/cmd/roles.runComponent[...].func1\n\t/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:113"]
[2023/10/24 07:44:36.098 +00:00] [ERROR] [datanode/service.go:262] ["failed to wait for RootCoord client to be ready"] [error="attempt #0: stack trace: /go/src/github.com/milvus-io/milvus/pkg/tracer/stack_trace.go:51 github.com/milvus-io/milvus/pkg/tracer.StackTrace\n/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:491 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).Call\n/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:505 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).ReCall\n/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:107 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.wrapGrpcCall[...]\n/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:121 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.(*Client).GetComponentStates\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:37 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...].func1\n/go/src/github.com/milvus-io/milvus/pkg/util/retry/retry.go:44 github.com/milvus-io/milvus/pkg/util/retry.Do\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:61 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...]\n/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:82 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentHealthy[...]\n/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:261 github.com/milvus-io/milvus/internal/distributed/datanode.(*Server).init: attempt #0: err: rpc error: code = Unimplemented desc = grpc: Decompressor is not installed for grpc-encoding \"zstd\": service unimplemented: unrecoverable error"] [stack="github.com/milvus-io/milvus/internal/distributed/datanode.(*Server).init\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:262\ngithub.com/milvus-io/milvus/internal/distributed/datanode.(*Server).Run\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:187\ngithub.com/milvus-io/milvus/cmd/components.(*DataNode).Run\n\t/go/src/github.com/milvus-io/milvus/cmd/components/data_node.go:53\ngithub.com/milvus-io/milvus/cmd/roles.runComponent[...].func1\n\t/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:113"]
panic: attempt #0: stack trace: /go/src/github.com/milvus-io/milvus/pkg/tracer/stack_trace.go:51 github.com/milvus-io/milvus/pkg/tracer.StackTrace
/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:491 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).Call
/go/src/github.com/milvus-io/milvus/internal/util/grpcclient/client.go:505 github.com/milvus-io/milvus/internal/util/grpcclient.(*ClientBase[...]).ReCall
/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:107 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.wrapGrpcCall[...]
/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/client/client.go:121 github.com/milvus-io/milvus/internal/distributed/rootcoord/client.(*Client).GetComponentStates
/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:37 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...].func1
/go/src/github.com/milvus-io/milvus/pkg/util/retry/retry.go:44 github.com/milvus-io/milvus/pkg/util/retry.Do
/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:61 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentStates[...]
/go/src/github.com/milvus-io/milvus/internal/util/componentutil/componentutil.go:82 github.com/milvus-io/milvus/internal/util/componentutil.WaitForComponentHealthy[...]
/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:261 github.com/milvus-io/milvus/internal/distributed/datanode.(*Server).init: attempt #0: err: rpc error: code = Unimplemented desc = grpc: Decompressor is not installed for grpc-encoding "zstd": service unimplemented: unrecoverable error

goroutine 252 [running]:
panic({0x4347a20, 0xc00150b398})
	/usr/local/go/src/runtime/panic.go:987 +0x3bb fp=0xc000fb18c8 sp=0xc000fb1808 pc=0x1781c1b
github.com/milvus-io/milvus/internal/distributed/datanode.(*Server).init(0xc001179180)
	/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:263 +0xfeb fp=0xc000fb1e48 sp=0xc000fb18c8 pc=0x379474b
github.com/milvus-io/milvus/internal/distributed/datanode.(*Server).Run(0xc001125b10?)
	/go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:187 +0x25 fp=0xc000fb1e88 sp=0xc000fb1e48 pc=0x37933c5
github.com/milvus-io/milvus/cmd/components.(*DataNode).Run(0xc000198fc0?)
	/go/src/github.com/milvus-io/milvus/cmd/components/data_node.go:53 +0x2f fp=0xc000fb1f58 sp=0xc000fb1e88 pc=0x3d373af
github.com/milvus-io/milvus/cmd/roles.runComponent[...].func1()
	/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:113 +0x15a fp=0xc000fb1fe0 sp=0xc000fb1f58 pc=0x3d3d8fa
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1598 +0x1 fp=0xc000fb1fe8 sp=0xc000fb1fe0 pc=0x17bb9a1
created by github.com/milvus-io/milvus/cmd/roles.runComponent[...]
	/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:100 +0x158

failed job: https://qa-jenkins.milvus.io/blue/organizations/jenkins/rolling_update_for_operator_test_simple/detail/rolling_update_for_operator_test_simple/145/pipeline
log:
artifacts-kafka-mixcoord-145-server-logs (1).tar.gz
artifacts-kafka-mixcoord-145-pytest-logs.tar.gz