Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance: no need database check in restful sdk #38078

Merged
merged 1 commit into from
Dec 2, 2024

Conversation

smellthemoon
Copy link
Contributor

@smellthemoon smellthemoon commented Nov 28, 2024

#38077
remove the check for two reason

  1. server will do the same to make sure use the correct database;
  2. each req has an additional overhead of calling the proxy to check database.

@smellthemoon smellthemoon changed the title enhance: remove no need database check in restful sdk enhance: no need database check in restful sdk Nov 28, 2024
@sre-ci-robot sre-ci-robot added the size/L Denotes a PR that changes 100-499 lines. label Nov 28, 2024
@mergify mergify bot added dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement labels Nov 28, 2024
Copy link
Contributor

mergify bot commented Nov 28, 2024

@smellthemoon Please associate the related issue to the body of your Pull Request. (eg. “issue: #”)

Copy link
Contributor

mergify bot commented Nov 28, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link

codecov bot commented Nov 28, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.06%. Comparing base (23dc313) to head (a171b7d).
Report is 5 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #38078      +/-   ##
==========================================
- Coverage   81.06%   81.06%   -0.01%     
==========================================
  Files        1369     1369              
  Lines      190978   190957      -21     
==========================================
- Hits       154819   154801      -18     
- Misses      30659    30667       +8     
+ Partials     5500     5489      -11     
Components Coverage Δ
Client 73.67% <ø> (ø)
Core 68.87% <ø> (ø)
Go 83.20% <100.00%> (-0.01%) ⬇️
Files with missing lines Coverage Δ
...nternal/distributed/proxy/httpserver/handler_v2.go 92.52% <100.00%> (-0.08%) ⬇️
internal/rootcoord/drop_collection_task.go 100.00% <100.00%> (ø)
internal/rootcoord/meta_table.go 83.10% <100.00%> (+0.24%) ⬆️

... and 24 files with indirect coverage changes

@smellthemoon
Copy link
Contributor Author

/run-cpu-e2e

Copy link
Contributor

mergify bot commented Nov 28, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@czs007
Copy link
Collaborator

czs007 commented Nov 29, 2024

pytest : test] [gw5] [ 37%] FAILED testcases/test_vector_operations.py::TestInsertVectorNegative::test_insert_vector_with_invalid_database_name

[pytest : test]

[pytest : test] =================================== FAILURES ===================================

[pytest : test] ____ TestInsertVectorNegative.test_insert_vector_with_invalid_database_name ____

[pytest : test] [gw5] linux -- Python 3.10.12 /usr/local/bin/python3

[pytest : test] [gw5] linux -- Python 3.10.12 /usr/local/bin/python3[gw5] linux -- Python 3.10.12 /usr/local/bin/python3

[pytest : test]

[pytest : test] self = <test_vector_operations.TestInsertVectorNegative object at 0x7f3f03cd50c0>

[pytest : test]

[pytest : test] def test_insert_vector_with_invalid_database_name(self):

[pytest : test] """

[pytest : test] Insert a vector with an invalid database name

[pytest : test] """

[pytest : test] # create a collection

[pytest : test] name = gen_collection_name()

[pytest : test] dim = 128

[pytest : test] payload = {

[pytest : test] "collectionName": name,

[pytest : test] "dimension": dim,

[pytest : test] }

[pytest : test] rsp = self.collection_client.collection_create(payload)

[pytest : test] assert rsp['code'] == 0

[pytest : test] rsp = self.collection_client.collection_describe(name)

[pytest : test] assert rsp['code'] == 0

[pytest : test] # insert data

[pytest : test] nb = 10

[pytest : test] data = get_data_by_payload(payload, nb)

[pytest : test] payload = {

[pytest : test] "collectionName": name,

[pytest : test] "data": data,

[pytest : test] }

[pytest : test] body_size = sys.getsizeof(json.dumps(payload))

[pytest : test] logger.info(f"body size: {body_size / 1024 / 1024} MB")

[pytest : test] success = False

[pytest : test] rsp = self.vector_client.vector_insert(payload, db_name="invalid_database")

[pytest : test] > assert rsp['code'] == 800

[pytest : test] E assert 100 == 800

[pytest : test]

[pytest : test] testcases/test_vector_operations.py:829: AssertionError

Copy link
Contributor

mergify bot commented Nov 29, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link
Contributor

mergify bot commented Nov 29, 2024

@smellthemoon go-sdk check failed, comment rerun go-sdk can trigger the job again.

Copy link
Contributor

mergify bot commented Nov 29, 2024

@smellthemoon go-sdk check failed, comment rerun go-sdk can trigger the job again.

Copy link
Contributor

mergify bot commented Nov 29, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@smellthemoon
Copy link
Contributor Author

pytest : test] [gw5] [ 37%] FAILED testcases/test_vector_operations.py::TestInsertVectorNegative::test_insert_vector_with_invalid_database_name

[pytest : test]

[pytest : test] =================================== FAILURES ===================================

[pytest : test] ____ TestInsertVectorNegative.test_insert_vector_with_invalid_database_name ____

[pytest : test] [gw5] linux -- Python 3.10.12 /usr/local/bin/python3

[pytest : test] [gw5] linux -- Python 3.10.12 /usr/local/bin/python3[gw5] linux -- Python 3.10.12 /usr/local/bin/python3

[pytest : test]

[pytest : test] self = <test_vector_operations.TestInsertVectorNegative object at 0x7f3f03cd50c0>

[pytest : test]

[pytest : test] def test_insert_vector_with_invalid_database_name(self):

[pytest : test] """

[pytest : test] Insert a vector with an invalid database name

[pytest : test] """

[pytest : test] # create a collection

[pytest : test] name = gen_collection_name()

[pytest : test] dim = 128

[pytest : test] payload = {

[pytest : test] "collectionName": name,

[pytest : test] "dimension": dim,

[pytest : test] }

[pytest : test] rsp = self.collection_client.collection_create(payload)

[pytest : test] assert rsp['code'] == 0

[pytest : test] rsp = self.collection_client.collection_describe(name)

[pytest : test] assert rsp['code'] == 0

[pytest : test] # insert data

[pytest : test] nb = 10

[pytest : test] data = get_data_by_payload(payload, nb)

[pytest : test] payload = {

[pytest : test] "collectionName": name,

[pytest : test] "data": data,

[pytest : test] }

[pytest : test] body_size = sys.getsizeof(json.dumps(payload))

[pytest : test] logger.info(f"body size: {body_size / 1024 / 1024} MB")

[pytest : test] success = False

[pytest : test] rsp = self.vector_client.vector_insert(payload, db_name="invalid_database")

[pytest : test] > assert rsp['code'] == 800

[pytest : test] E assert 100 == 800

[pytest : test]

[pytest : test] testcases/test_vector_operations.py:829: AssertionError

because in rootcoord, first see whether the collection exist and then see the db exist, has already change it logic in rootcoord.

@smellthemoon
Copy link
Contributor Author

smellthemoon commented Nov 29, 2024

@czs007 But I am not sure whether I should change the code or the test case here, because the errors reported here are actually correct. The root cause is that, under a non-existent db, restful would report an error of db not exist, but pymilvus would report an error of collection not found. Any comment about this?

Copy link
Contributor

mergify bot commented Nov 29, 2024

@smellthemoon Thanks for your contribution. Please submit with DCO, see the contributing guide https://github.com/milvus-io/milvus/blob/master/CONTRIBUTING.md#developer-certificate-of-origin-dco.

@mergify mergify bot added needs-dco DCO is missing in this pull request. dco-passed DCO check passed. and removed dco-passed DCO check passed. do-not-merge/missing-related-issue needs-dco DCO is missing in this pull request. labels Nov 29, 2024
Copy link
Contributor

mergify bot commented Nov 29, 2024

@smellthemoon go-sdk check failed, comment rerun go-sdk can trigger the job again.

Copy link
Contributor

mergify bot commented Nov 29, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@smellthemoon
Copy link
Contributor Author

/run-cpu-e2e

@smellthemoon
Copy link
Contributor Author

rerun go-sdk

Copy link
Contributor

mergify bot commented Nov 29, 2024

@smellthemoon go-sdk check failed, comment rerun go-sdk can trigger the job again.

Copy link
Contributor

mergify bot commented Nov 29, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link
Contributor

mergify bot commented Nov 29, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

Copy link
Contributor

mergify bot commented Nov 29, 2024

@smellthemoon go-sdk check failed, comment rerun go-sdk can trigger the job again.

Copy link
Contributor

mergify bot commented Dec 2, 2024

@smellthemoon E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@smellthemoon
Copy link
Contributor Author

/run-cpu-e2e

@mergify mergify bot added the ci-passed label Dec 2, 2024
@czs007
Copy link
Collaborator

czs007 commented Dec 2, 2024

/approve
/lgtm

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: czs007, smellthemoon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot sre-ci-robot merged commit db453c0 into milvus-io:master Dec 2, 2024
20 checks passed
sre-ci-robot pushed a commit that referenced this pull request Dec 4, 2024
pr: #38078
pr: #38140
issue: #38077

---------

Signed-off-by: lixinguo <[email protected]>
Co-authored-by: lixinguo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved ci-passed dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement lgtm size/L Denotes a PR that changes 100-499 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants