Skip to content

Commit

Permalink
fix: grant ManualCompact api doesn't work (#38096) (#38168)
Browse files Browse the repository at this point in the history 
issue: #38086
pr: #38096
cause ManualCompact api pass collection id in request, but RBAC requires
to check collection name, so grant ManualCompact api doesn't work.

This PR refine the ManualCompact api to accpet collection name in
request.

Signed-off-by: Wei Liu <[email protected]>
  • Loading branch information
@weiliu1031
weiliu1031 authored Dec 5, 2024
1 parent d4ef89f commit 8646bba
Show file tree
Hide file tree
Showing 10 changed files with 82 additions and 15 deletions.
2 changes: 1 addition & 1 deletion client/go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ require (
github.com/blang/semver/v4 v4.0.0
github.com/cockroachdb/errors v1.9.1
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.17
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241129073142-d8e5b9597772
github.com/milvus-io/milvus/pkg v0.0.2-0.20240317152703-17b4938985f3
github.com/quasilyte/go-ruleguard/dsl v0.3.22
github.com/samber/lo v1.27.0
Expand Down
4 changes: 2 additions & 2 deletions client/go.sum
View file
Original file line number Diff line number Diff line change
Expand Up @@ -400,8 +400,8 @@ github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfr
github.com/mediocregopher/radix/v3 v3.4.2/go.mod h1:8FL3F6UQRXHXIBSPUs5h0RybMF8i4n7wVopoX3x7Bv8=
github.com/microcosm-cc/bluemonday v1.0.2/go.mod h1:iVP4YcDBq+n/5fb23BhYFvIMq/leAFZyRl6bYmGDlGc=
github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.17 h1:ANkXdUKKpIPPQkw9pkV9ku9AEtSaPyua9XzdMTUxjCs=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.17/go.mod h1:/6UT4zZl6awVeXLeE7UGDWZvXj3IWkRsh3mqsn0DiAs=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241129073142-d8e5b9597772 h1:HN66VyfkOxDp4AZXMTPxX8c3LkV4RV7srsffZXk3DMs=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241129073142-d8e5b9597772/go.mod h1:/6UT4zZl6awVeXLeE7UGDWZvXj3IWkRsh3mqsn0DiAs=
github.com/milvus-io/milvus/pkg v0.0.2-0.20240317152703-17b4938985f3 h1:ZBpRWhBa7FTFxW4YYVv9AUESoW1Xyb3KNXTzTqfkZmw=
github.com/milvus-io/milvus/pkg v0.0.2-0.20240317152703-17b4938985f3/go.mod h1:jQ2BUZny1COsgv1Qbcv8dmbppW+V9J/c4YQZNb3EOm8=
github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=
Expand Down
55 changes: 55 additions & 0 deletions client/mock_milvus_server_test.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ require (
github.com/go-playground/validator/v10 v10.14.0
github.com/gofrs/flock v0.8.1
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.4
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/btree v1.1.2
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0
github.com/klauspost/compress v1.17.9
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241120092224-a1c2ac2fd2c1
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241129073142-d8e5b9597772
github.com/minio/minio-go/v7 v7.0.73
github.com/pingcap/log v1.1.1-0.20221015072633-39906604fb81
github.com/prometheus/client_golang v1.14.0
Expand Down
8 changes: 2 additions & 6 deletions go.sum
View file
Original file line number Diff line number Diff line change
Expand Up @@ -162,10 +162,6 @@ github.com/cloudwego/base64x v0.1.4 h1:jwCgWpFanWmN8xoIUHa2rtzmkd5J2plF/dnLS6Xd/
github.com/cloudwego/base64x v0.1.4/go.mod h1:0zlkT4Wn5C6NdauXdJRhSKRlJvmclQ1hhJgA0rcu/8w=
github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg=
github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
github.com/cloudwego/base64x v0.1.4 h1:jwCgWpFanWmN8xoIUHa2rtzmkd5J2plF/dnLS6Xd/0Y=
github.com/cloudwego/base64x v0.1.4/go.mod h1:0zlkT4Wn5C6NdauXdJRhSKRlJvmclQ1hhJgA0rcu/8w=
github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg=
github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
Expand Down Expand Up @@ -618,8 +614,8 @@ github.com/milvus-io/cgosymbolizer v0.0.0-20240722103217-b7dee0e50119 h1:9VXijWu
github.com/milvus-io/cgosymbolizer v0.0.0-20240722103217-b7dee0e50119/go.mod h1:DvXTE/K/RtHehxU8/GtDs4vFtfw64jJ3PaCnFri8CRg=
github.com/milvus-io/gorocksdb v0.0.0-20220624081344-8c5f4212846b h1:TfeY0NxYxZzUfIfYe5qYDBzt4ZYRqzUjTR6CvUzjat8=
github.com/milvus-io/gorocksdb v0.0.0-20220624081344-8c5f4212846b/go.mod h1:iwW+9cWfIzzDseEBCCeDSN5SD16Tidvy8cwQ7ZY8Qj4=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241120092224-a1c2ac2fd2c1 h1:Xp4zOR85XFFtM7Eif945BeSmDf30hbdijbeNSuy92Bg=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241120092224-a1c2ac2fd2c1/go.mod h1:/6UT4zZl6awVeXLeE7UGDWZvXj3IWkRsh3mqsn0DiAs=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241129073142-d8e5b9597772 h1:HN66VyfkOxDp4AZXMTPxX8c3LkV4RV7srsffZXk3DMs=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241129073142-d8e5b9597772/go.mod h1:/6UT4zZl6awVeXLeE7UGDWZvXj3IWkRsh3mqsn0DiAs=
github.com/milvus-io/milvus-storage/go v0.0.0-20231227072638-ebd0b8e56d70 h1:Z+sp64fmAOxAG7mU0dfVOXvAXlwRB0c8a96rIM5HevI=
github.com/milvus-io/milvus-storage/go v0.0.0-20231227072638-ebd0b8e56d70/go.mod h1:GPETMcTZq1gLY1WA6Na5kiNAKnq8SEMMiVKUZrM3sho=
github.com/milvus-io/pulsar-client-go v0.6.10 h1:eqpJjU+/QX0iIhEo3nhOqMNXL+TyInAs1IAHZCrCM/A=
Expand Down
5 changes: 5 additions & 0 deletions internal/proxy/database_interceptor.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,11 @@ func fillDatabase(ctx context.Context, req interface{}) (context.Context, interf
r.DbName = GetCurDBNameFromContextOrDefault(ctx)
}
return ctx, r
case *milvuspb.ManualCompactionRequest:
if r.DbName == "" {
r.DbName = GetCurDBNameFromContextOrDefault(ctx)
}
return ctx, r
default:
}
return ctx, req
Expand Down
2 changes: 1 addition & 1 deletion internal/proxy/database_interceptor_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ func TestDatabaseInterceptor(t *testing.T) {
&milvuspb.ListImportTasksRequest{},
&milvuspb.OperatePrivilegeRequest{Entity: &milvuspb.GrantEntity{}},
&milvuspb.SelectGrantRequest{Entity: &milvuspb.GrantEntity{}},
&milvuspb.ManualCompactionRequest{},
}

md := metadata.Pairs(util.HeaderDBName, "db")
Expand All @@ -117,7 +118,6 @@ func TestDatabaseInterceptor(t *testing.T) {
&milvuspb.CalcDistanceRequest{},
&milvuspb.FlushAllRequest{},
&milvuspb.GetCompactionStateRequest{},
&milvuspb.ManualCompactionRequest{},
&milvuspb.GetCompactionPlansRequest{},
&milvuspb.GetFlushAllStateRequest{},
&milvuspb.GetImportStateRequest{},
Expand Down
11 changes: 11 additions & 0 deletions internal/proxy/impl.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4542,6 +4542,7 @@ func (node *Proxy) ManualCompaction(ctx context.Context, req *milvuspb.ManualCom
defer sp.End()

log := log.Ctx(ctx).With(
zap.String("collectionName", req.GetCollectionName()),
zap.Int64("collectionID", req.GetCollectionID()))

log.Info("received ManualCompaction request")
Expand All @@ -4551,6 +4552,16 @@ func (node *Proxy) ManualCompaction(ctx context.Context, req *milvuspb.ManualCom
return resp, nil
}

// before v2.4.18, manual compact request only pass collectionID, should correct sdk's behavior to pass collectionName
if req.GetCollectionName() != "" {
var err error
req.CollectionID, err = globalMetaCache.GetCollectionID(ctx, req.GetDbName(), req.GetCollectionName())
if err != nil {
resp.Status = merr.Status(err)
return resp, nil
}
}

resp, err := node.dataCoord.ManualCompaction(ctx, req)
log.Info("received ManualCompaction response",
zap.Any("resp", resp),
Expand Down
2 changes: 1 addition & 1 deletion pkg/go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ require (
github.com/expr-lang/expr v1.15.7
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0
github.com/klauspost/compress v1.17.7
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241120092224-a1c2ac2fd2c1
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241129073142-d8e5b9597772
github.com/nats-io/nats-server/v2 v2.10.12
github.com/nats-io/nats.go v1.34.1
github.com/panjf2000/ants/v2 v2.7.2
Expand Down
4 changes: 2 additions & 2 deletions pkg/go.sum
View file
Original file line number Diff line number Diff line change
Expand Up @@ -503,8 +503,8 @@ github.com/milvus-io/cgosymbolizer v0.0.0-20240722103217-b7dee0e50119 h1:9VXijWu
github.com/milvus-io/cgosymbolizer v0.0.0-20240722103217-b7dee0e50119/go.mod h1:DvXTE/K/RtHehxU8/GtDs4vFtfw64jJ3PaCnFri8CRg=
github.com/milvus-io/gorocksdb v0.0.0-20220624081344-8c5f4212846b h1:TfeY0NxYxZzUfIfYe5qYDBzt4ZYRqzUjTR6CvUzjat8=
github.com/milvus-io/gorocksdb v0.0.0-20220624081344-8c5f4212846b/go.mod h1:iwW+9cWfIzzDseEBCCeDSN5SD16Tidvy8cwQ7ZY8Qj4=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241120092224-a1c2ac2fd2c1 h1:Xp4zOR85XFFtM7Eif945BeSmDf30hbdijbeNSuy92Bg=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241120092224-a1c2ac2fd2c1/go.mod h1:/6UT4zZl6awVeXLeE7UGDWZvXj3IWkRsh3mqsn0DiAs=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241129073142-d8e5b9597772 h1:HN66VyfkOxDp4AZXMTPxX8c3LkV4RV7srsffZXk3DMs=
github.com/milvus-io/milvus-proto/go-api/v2 v2.4.18-0.20241129073142-d8e5b9597772/go.mod h1:/6UT4zZl6awVeXLeE7UGDWZvXj3IWkRsh3mqsn0DiAs=
github.com/milvus-io/pulsar-client-go v0.6.10 h1:eqpJjU+/QX0iIhEo3nhOqMNXL+TyInAs1IAHZCrCM/A=
github.com/milvus-io/pulsar-client-go v0.6.10/go.mod h1:lQqCkgwDF8YFYjKA+zOheTk1tev2B+bKj5j7+nm8M1w=
github.com/minio/highwayhash v1.0.2 h1:Aak5U0nElisjDCfPSG79Tgzkn2gl66NxOMspRrKnA/g=
Expand Down

0 comments on commit 8646bba

Please sign in to comment.