Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Unable to delete entities #37725

Open
1 task done
qiulingdong opened this issue Nov 15, 2024 · 14 comments
Open
1 task done

[Bug]: Unable to delete entities #37725

qiulingdong opened this issue Nov 15, 2024 · 14 comments
Assignees
Labels
kind/bug Issues or changes related a bug triage/needs-information Indicates an issue needs more information in order to work on it.

Comments

@qiulingdong
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: 2.4.6
- Deployment mode(standalone or cluster): standalone 
- MQ type(rocksmq, pulsar or kafka):  None 
- SDK version(e.g. pymilvus v2.0.0rc2):  pymilvus  v.2.4.9
- OS(Ubuntu or CentOS): Ubuntu 22.04
- CPU/Memory: AMD 64C / 256G
- GPU: 96G
- Others:

Current Behavior

I created a strongly consistent collection and inserted multiple records. Frequently, when deleting data, I encounter situations where the data cannot be deleted properly. There are several phenomena:

  1. Data can be deleted normally.
  2. Data cannot be deleted.
  3. Some of the data that meets the conditions is deleted, but the remaining part cannot be deleted.

Using code

#setting
MILVUS_FIELD = [
    FieldSchema(name='doc_id', dtype=DataType.VARCHAR, max_length=64, is_primary=True, auto_id=False),
    FieldSchema(name='kb_id', dtype=DataType.VARCHAR, max_length=64, is_partition_key=True),
    FieldSchema(name='file_id', dtype=DataType.VARCHAR, max_length=64),
    FieldSchema(name="file_name", dtype=DataType.VARCHAR, max_length=256),
    FieldSchema(name='content', dtype=DataType.VARCHAR, max_length=65535),  
    FieldSchema(name='embedding', dtype=DataType.FLOAT_VECTOR, dim=1024),
    FieldSchema(name="source", dtype=DataType.VARCHAR, max_length=1024), 
    FieldSchema(name="keywords", dtype=DataType.VARCHAR, max_length=2048), 
    FieldSchema(name="metadata", dtype=DataType.VARCHAR, max_length=16144),
    FieldSchema(name="dtype", dtype=DataType.VARCHAR, max_length=32),
]

MILVUS_INDEX_PARAMS = {
    'metric_type': 'COSINE',
    'index_type': 'IVF_FLAT',
    'params': {'nlist': 1024}
}


#create connections
connections.connect(db_name=database,host=host, port=port, user=user, password=password)
collection = Collection(name=collection_name,
                                    consistency_level=0,
                                    schema=CollectionSchema(fields=MILVUS_FIELD, enable_dynamic_field=True))
collection.create_index(field_name='embedding', index_params=MILVUS_INDEX_PARAMS)

#insert data ...
#...

#delete data
collection.load()
result = collection.query(expr="file_id=='FILE593c68a689aa4797bf59f44a3a2e8c9a'",output_fields=["doc_id","file_id"])
print(f"before delete:{result}")

result=collection.delete(expr="file_id=='FILE593c68a689aa4797bf59f44a3a2e8c9a'")
print(f"delete result:{result}")

result = collection.query(expr="file_id=='FILE593c68a689aa4797bf59f44a3a2e8c9a'",output_fields=["doc_id","file_id"],consistency_level="Strong")
print(f"end delete:{result}")

print log

before delete:data: ["{'doc_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_1', 'file_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a'}", "{'doc_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_2', 'file_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a'}", "{'doc_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_4', 'file_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a'}", "{'doc_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_6', 'file_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a'}"]

delete result:(insert count: 0, delete count: 4, upsert count: 0, timestamp: 0, success count: 0, err count: 0

end delete:data: ["{'doc_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_1', 'file_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a'}", "{'doc_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_2', 'file_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a'}", "{'doc_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_4', 'file_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a'}", "{'doc_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_6', 'file_id': 'FILE593c68a689aa4797bf59f44a3a2e8c9a'}"]

Expected Behavior

I expect the data in the Collection to be successfully deleted as it .

Steps To Reproduce

1.Deploy milvus using the docker compose file I provided.
2.Created a strongly consistent collection and inserted multiple records.
3.Run Python Code

Milvus Log

No response

Anything else?

No response

@qiulingdong qiulingdong added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Nov 15, 2024
@yanliang567
Copy link
Contributor

@qiulingdong I guess there are duplicated doc_id when you insert into the collection. you can check it by
collection.query(expr="doc_id=='xxxxx'", output_fields=["count(*)"]

/assign @qiulingdong
/unassign

@yanliang567 yanliang567 added triage/needs-information Indicates an issue needs more information in order to work on it. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Nov 15, 2024
@qiulingdong
Copy link
Author

qiulingdong commented Nov 15, 2024

@qiulingdong I guess there are duplicated doc_id when you insert into the collection. you can check it by collection.query(expr="doc_id=='xxxxx'", output_fields=["count(*)"]

/assign @qiulingdong /unassign

I have tried to look up, and there is no duplicate doc_id appearing.
Additionally, I attempted to use attu to randomly generate 1000 data entries, and the same issue persists.

result = collection.query(expr="doc_id=='FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_1'", output_fields=["count(*)"])
print(f"1.count:{result}")
result = collection.query(expr="doc_id=='FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_2'", output_fields=["count(*)"])
print(f"2.count:{result}")
result = collection.query(expr="doc_id=='FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_4'", output_fields=["count(*)"])
print(f"3.count:{result}")
result = collection.query(expr="doc_id=='FILE593c68a689aa4797bf59f44a3a2e8c9a_CHUNK_6'", output_fields=["count(*)"])
print(f"4.count:{result}")

1.count:data: ["{'count()': 1}"]
2.count:data: ["{'count(
)': 1}"]
3.count:data: ["{'count()': 1}"]
4.count:data: ["{'count(
)': 1}"]

1

@xiaofan-luan
Copy link
Collaborator

could you try 2.4.15?

@rabbor
Copy link

rabbor commented Nov 19, 2024

  • Milvus version: 2.4.15
  • Deployment mode(standalone or cluster): standalone
  • MQ type(rocksmq, pulsar or kafka): None
  • SDK version(e.g. pymilvus v2.0.0rc2): pymilvus v.2.4.9

I found a similar issue on version v2.4.15, but the phenomenon is not exactly the same

When the collection is loaded, I delete the entity and release the collection after the deletion returns, but when I reconnect the data to query, the entity reappears

The steps are as follows:

1. load_collection  
2.  delete  
3.  release_collection  
4.  reconnect milvus db 
5.  get the deleted entity when query

@yanliang567
Copy link
Contributor

I believe this is caused by the default consistency level is bound. So if you add consistency_level=ConsistencyLevel.Strong in the query() after delete, you will won't see the deleted entities immediately. @qiulingdong @rabbor
/assign @qiulingdong
/unassign

@xiaofan-luan
Copy link
Collaborator

I believe this is caused by the default consistency level is bound. So if you add consistency_level=ConsistencyLevel.Strong in the query() after delete, you will won't see the deleted entities immediately. @qiulingdong @rabbor /assign @qiulingdong /unassign

I thought there is a bug on remote loading.

@yanliang567 for some reason, L0 segment is not processed as expect.

that's why I talked about the test on delete and right now the test itself is not very solid

@xiaofan-luan
Copy link
Collaborator

especially under different failure recovery test

@yanliang567
Copy link
Contributor

remote load is only avaliable on 2.4.16 or later, I don't think that is the issue here. @qiulingdong @rabbor please try as my comments above and keep us posted. thx.

@rabbor
Copy link

rabbor commented Nov 21, 2024

I was not previously aware of the "consistency_level" parameter in the documentation. Following your suggestion, I added the parameter ”consistency_level=ConsistencyLevel.Strong“ to the interface "create_collection", "delete", "search", but the same issue still occurs after testing. The steps to reproduce are as follows:

- create collection
- insert 3 user data 
- delete 1 user
- search (The immediate retrieval results are as expected, but when reconnecting to the database for retrieval, deleted data reappears.)

@yanliang567

@yanliang567
Copy link
Contributor

I cannot reproduce the issue, could you please share your code sample to reproduce the issue?
here is my code:

import time

from pymilvus import (
    utility,
    FieldSchema, CollectionSchema, DataType, MilvusClient,
    Collection, AnnSearchRequest, RRFRanker, connections,
)
from pymilvus.client.constants import ConsistencyLevel
import random
import json


def create_collection(collection_name, dimension):
    schema = MilvusClient.create_schema(
        auto_id=False,
        enable_dynamic_field=False,
    )
    # 2. Add fields to schema
    schema.add_field(field_name="id", datatype=DataType.VARCHAR, is_primary=True, max_length=256)
    schema.add_field(field_name="embeddings", datatype=DataType.FLOAT_VECTOR, dim=dimension)
    schema.add_field(field_name="title", datatype=DataType.VARCHAR, max_length=256)
    schema.add_field(field_name="doc_type", datatype=DataType.VARCHAR, max_length=256)
    schema.add_field(field_name="uri", datatype=DataType.VARCHAR, max_length=256)

    index_params = {"index_type": "HNSW", "metric_type": "COSINE", "params":{"M": 32, "efConstruction": 360}}
    collection = Collection(name=collection_name, schema=schema, auto_id=False)
    collection.create_index(field_name="embeddings", index_params=index_params)
    collection.load()
    return collection


# 1. Set up a Milvus client
host = '10.xx.xx.xx'
connections.connect(host=host)

# insert
c_name = "issue_37725_7"
dim = 128
if not utility.has_collection(c_name):
    # prepare data
    data = []
    colors = ["green", "blue", "yellow", "red", "black", "white", "purple", "pink", "orange", "brown", "grey"]
    for i in range(4000):
        current_color = random.choice(colors)
        doc = { "id": str(i),
                "embeddings": [random.uniform(-1, 1) for _ in range(dim)],
                "title": current_color,
                "doc_type": "type_1"}
        doc["uri"] = "uri" + str(int(i / 1000))
        data.append(doc)

    c = create_collection(c_name, dim)
    for i in range(0, len(data), 1000):
        print(f"indexing docs from {i} to {i+1000}")
        c.insert(data=data[i:i+1000])

c=Collection(c_name)
c.flush()
c.load()
query_results = c.query(
    expr=f"id in ['11','12','13','14','15','16','17','18','19','20']",
    output_fields=["id", "title"])
print(f"{c_name} query result before delete: {query_results}")

c.delete(expr=f"id in ['11','13','15','17','19']")
query_results = c.query(
    expr=f"id in ['11','12','13','14','15','16','17','18','19','20']",
    output_fields=["id", "title"],
    consistency_level=ConsistencyLevel.Strong
)
print(f"{c_name} query result after delete: {query_results}")
c.release()

connections.disconnect("default")
connections.connect(host=host)
c=Collection(c_name)
c.load()
query_results = c.query(
    expr=f"id in ['11','12','13','14','15','16','17','18','19','20']",
    output_fields=["id", "title"])
print(f"{c_name} query result after reconnect: {query_results}")

print('completed')

@rabbor
Copy link

rabbor commented Nov 21, 2024

The code is relatively complex, and I extracted the core process for testing but couldn't reproduce it. Nevertheless, I really appreciate your help, thank you!

code :

import json
import time

from pymilvus import MilvusClient, DataType
from pymilvus.client.types import LoadState
from pymilvus.grpc_gen import common_pb2
ConsistencyLevel = common_pb2.ConsistencyLevel

#change this 
milvus_ip_port = "http://172.16.4.34:19530"

client = MilvusClient(milvus_ip_port)


collection_name = "ccc"
# create collection
schema = client.create_schema(auto_id=False,enable_dynamic_field=True)

# Add fields to schema
schema.add_field(field_name="user_id", datatype=DataType.VARCHAR, max_length=256, is_primary=True)
schema.add_field(field_name="face_feature", datatype=DataType.FLOAT_VECTOR, dim=512)
schema.add_field(field_name="user_attr", datatype=DataType.JSON)
        
client.create_collection(collection_name=collection_name, schema=schema)
        
# create index for collection
index_params = client.prepare_index_params()
index_params.add_index(
    field_name="user_id",
    index_type="Trie", 
    index_name="user_id_index"
    )
index_params.add_index(
    field_name="face_feature", 
    index_type="IVF_FLAT",
    metric_type="COSINE",
    params={ "nlist": 128 }
    )    
client.create_index(collection_name=collection_name,index_params=index_params)


# insert data
feature_1 = [0.0062522911466658115,
                    0.0369855880737305,
                    0.005088048055768013,
                    0.031010277569293976,
                    0.03752732276916504,
                    -0.17541715875267982,
                    0.041872020810842514,
                    0.03986259922385216,
                    -0.02916378155350685,
                    -0.001541009871289134,
                    -0.052109215408563614,
                    0.01816626638174057,
                    0.01826130598783493,
                    -0.018437810242176056,
                    -0.031444747000932693,
                    0.0297611765563488,
                    0.010060690343379974,
                    0.01699862815439701,
                    -0.06126023456454277,
                    -0.04887784644961357,
                    -0.0504799522459507,
                    0.08184323459863663,
                    0.0033128317445516586,
                    -0.003214397234842181,
                    -0.04265949875116348,
                    0.04393575340509415,
                    0.04447884112596512,
                    0.02204933948814869,
                    0.05455310642719269,
                    0.005071076564490795,
                    -0.00794264953583479,
                    -0.025701601058244705,
                    0.007128019351512194,
                    0.0053392257541418076,
                    0.005631135310977697,
                    -0.040324222296476364,
                    0.036223914474248886,
                    0.08852320909500122,
                    0.016292614862322807,
                    0.013271693140268326,
                    0.05458026006817818,
                    0.005872129928320646,
                    -0.0003394294762983918,
                    -0.06495323032140732,
                    0.05935942754149437,
                    -0.020392922684550285,
                    0.01260641124099493,
                    0.07271937280893326,
                    0.027968989685177803,
                    -0.0757063552737236,
                    0.05604659765958786,
                    -0.04217071831226349,
                    -0.00870297197252512,
                    0.0692979246377945,
                    -0.014717662706971169,
                    -0.04073153808712959,
                    0.05702415481209755,
                    -0.026556963101029396,
                    -0.008906629867851734,
                    -0.011907186359167099,
                    0.03486619517207146,
                    0.041491858661174774,
                    0.06994962692260742,
                    -0.010495159775018692,
                    -0.1047615185379982,
                    0.0049217273481190205,
                    -0.014147421345114708,
                    -0.003102385438978672,
                    0.06028267741203308,
                    0.023827949538826942,
                    -0.052326448261737823,
                    0.015206441283226013,
                    -0.05582936108112335,
                    0.051131658256053925,
                    0.04510338976979256,
                    -0.019075937569141388,
                    -0.007175539154559374,
                    0.039889752864837646,
                    0.04021560400724411,
                    -0.0792907252907753,
                    -0.008798012509942055,
                    -0.009184962138533592,
                    -0.0022707832977175713,
                    -0.08645948022603989,
                    0.021031051874160767,
                    -0.05800171196460724,
                    -0.01775895059108734,
                    -0.0017310903640463948,
                    -0.07831317186355591,
                    -0.01983626000583172,
                    -0.029218090698122978,
                    0.020542273297905922,
                    -0.037608787417411804,
                    -0.055367738008499146,
                    -0.08656809478998184,
                    -0.0214247889816761,
                    0.062129173427820206,
                    -0.022673889994621277,
                    0.03242230415344238,
                    0.004755407106131315,
                    -0.02995125763118267,
                    -0.07722699642181396,
                    0.03896650671958923,
                    0.015220018103718758,
                    0.11893609166145325,
                    0.017338057979941368,
                    0.018274882808327675,
                    -0.06435582786798477,
                    -0.029082318767905235,
                    0.031444747000932693,
                    0.037988949567079544,
                    0.008614720776677132,
                    0.035273510962724686,
                    -0.0588163398206234,
                    0.013509294018149376,
                    0.051104504615068436,
                    -0.025104204192757607,
                    0.0087504917755723,
                    -0.009599066339433193,
                    -0.041654787957668304,
                    0.01652342826128006,
                    -0.002454075263813138,
                    0.027032164856791496,
                    0.0440443716943264,
                    0.012518159113824368,
                    -0.06359551101922989,
                    0.015233594924211502,
                    -0.02031145989894867,
                    0.008078421466052532,
                    0.02356998436152935,
                    -0.03459465503692627,
                    -0.05376563221216202,
                    -0.07787869870662689,
                    -0.07353400439023972,
                    -0.05175620689988136,
                    0.019876990467309952,
                    0.008214193396270275,
                    0.013536447659134865,
                    0.050126947462558746,
                    -0.0024608636740595102,
                    -0.022348036989569664,
                    0.005946804769337177,
                    -0.005403717514127493,
                    -0.02805045247077942,
                    0.012158364057540894,
                    -0.020650889724493027,
                    0.030222801491618156,
                    -0.050941579043865204,
                    0.04969247803092003,
                    -0.06321534514427185,
                    0.023135513067245483,
                    0.00558361504226923,
                    -0.042985349893569946,
                    0.019849836826324463,
                    0.046080946922302246,
                    0.10394688695669174,
                    -0.028919393196702003,
                    0.041274625808000565,
                    -0.036278221756219864,
                    -0.010461216792464256,
                    0.06109730899333954,
                    0.003261917270720005,
                    0.03318262845277786,
                    0.011486293748021126,
                    0.03728293627500534,
                    0.021465521305799484,
                    0.04236080124974251,
                    -0.07228490710258484,
                    -0.035110585391521454,
                    0.11116994917392731,
                    -0.02183210477232933,
                    -0.0012703149113804102,
                    0.0557207427918911,
                    0.03345416858792305,
                    -0.12436696141958237,
                    0.02840345911681652,
                    0.007148385047912598,
                    -0.03646830469369888,
                    0.053249698132276535,
                    -0.06908068805932999,
                    0.004141039680689573,
                    -0.03845057263970375,
                    -0.10057974606752396,
                    -0.037038546055555344,
                    0.040269915014505386,
                    0.09623505175113678,
                    -0.05528627336025238,
                    0.03752732276916504,
                    0.0683203637599945,
                    -0.03304685652256012,
                    -0.017161555588245392,
                    0.06935223191976547,
                    0.022456655278801918,
                    0.018777240067720413,
                    -0.010277925059199333,
                    -0.029734022915363312,
                    0.02767029218375683,
                    -0.013251326978206635,
                    -0.0683203637599945,
                    -0.09916771948337555,
                    -0.031064586713910103,
                    -0.07896487414836884,
                    -0.022606004029512405,
                    -0.02337990328669548,
                    0.02767029218375683,
                    0.03771740570664406,
                    -0.02992410399019718,
                    -0.03839626535773277,
                    0.011934340931475163,
                    -0.04013414308428764,
                    -0.01914382353425026,
                    -0.0347575806081295,
                    -0.039373818784952164,
                    0.009252848103642464,
                    -0.061205923557281494,
                    -0.03358994051814079,
                    0.002213080180808902,
                    0.10508736968040466,
                    -0.07195904850959778,
                    -0.040297068655490875,
                    -0.07923641800880432,
                    -0.04374567046761513,
                    0.034513190388679504,
                    -0.08803442865610123,
                    0.023488519713282585,
                    -0.015274327248334885,
                    0.025660868734121323,
                    -0.032938238233327866,
                    0.0681031346321106,
                    0.03418733924627304,
                    -0.02126186341047287,
                    0.029435325413942337,
                    0.059033576399087906,
                    -0.005403717514127493,
                    -0.05805601924657822,
                    -0.039102278649806976,
                    0.048008907586336136,
                    -0.005865341518074274,
                    0.03945528343319893,
                    0.017324481159448624,
                    0.06707126647233963,
                    0.0911843404173851,
                    0.05069718882441521,
                    -0.009673740714788437,
                    0.024153802543878555,
                    0.0037778501864522696,
                    -0.07223059237003326,
                    0.07815024256706238,
                    -0.0022572060115635395,
                    -0.0035063065588474274,
                    0.008044478483498096,
                    0.03263954073190689,
                    -0.03223222494125366,
                    -0.034295953810214996,
                    0.03147190064191818,
                    0.0361967608332634,
                    -0.005312071647495031,
                    0.018967319279909134,
                    -0.03888504207134247,
                    0.004208925645798445,
                    0.07771577686071396,
                    -0.02318982221186161,
                    0.07065564393997192,
                    0.009028824046254158,
                    0.02657053992152214,
                    -0.056318141520023346,
                    -0.013305636122822762,
                    0.016917165368795395,
                    -0.06288949400186539,
                    0.013726528733968735,
                    -0.0383419543504715,
                    0.014269615523517132,
                    0.015274327248334885,
                    -0.02997841313481331,
                    -0.08222340047359467,
                    -0.03948243707418442,
                    -0.017677487805485725,
                    -0.03399725630879402,
                    0.015477984212338924,
                    0.0557207427918911,
                    -0.06348688900470734,
                    -0.0007535334443673491,
                    -0.0399983711540699,
                    0.0011998832924291492,
                    -0.06506184488534927,
                    0.04909507930278778,
                    -0.010325444862246513,
                    0.0004633212520275265,
                    -0.017338057979941368,
                    0.0849931463599205,
                    -0.049203697592020035,
                    -0.012572468258440495,
                    -0.09933064132928848,
                    -0.02716793678700924,
                    0.0738055482506752,
                    -0.031173205003142357,
                    -0.029109472408890724,
                    0.015532293356955051,
                    -0.04007983207702637,
                    -0.04627102613449097,
                    -0.041057389229536057,
                    -0.041872020810842514,
                    -0.04388144239783287,
                    0.04450599476695061,
                    -0.04040568694472313,
                    0.017052937299013138,
                    -0.0371471643447876,
                    0.046678341925144196,
                    -0.04711281135678291,
                    -0.004018845036625862,
                    0.06815744191408157,
                    -0.02881077490746975,
                    -0.08488452434539795,
                    -0.027195090427994728,
                    0.03983544558286667,
                    -0.006788589525967836,
                    -0.06527908146381378,
                    -0.013875877484679222,
                    0.06712557375431061,
                    -0.055992286652326584,
                    -0.06131454184651375,
                    0.012816857546567917,
                    0.013115555047988892,
                    0.11176734417676926,
                    -0.015287904068827629,
                    -0.03095596842467785,
                    -0.013957340270280838,
                    0.07494603097438812,
                    -0.01693074218928814,
                    -0.015396521426737309,
                    -0.01441896427422762,
                    0.020623736083507538,
                    0.05930512025952339,
                    0.021777795627713203,
                    0.003876284696161747,
                    0.020610159263014793,
                    0.018206996843218803,
                    0.027086472138762474,
                    -0.07347969710826874,
                    -0.05243506655097008,
                    -0.07787869870662689,
                    -0.01842423342168331,
                    -0.010827800258994102,
                    0.01213120948523283,
                    -0.013509294018149376,
                    -0.0294081699103117,
                    -0.008967727422714233,
                    0.004310754593461752,
                    0.07445725053548813,
                    -0.03774455934762955,
                    0.06913499534130096,
                    -0.020433655008673668,
                    0.010271136648952961,
                    -0.03462180867791176,
                    0.015097823925316334,
                    -0.0031634827610105276,
                    -0.01961902342736721,
                    0.06348688900470734,
                    0.04662403464317322,
                    0.026909969747066498,
                    0.07342538982629776,
                    0.0043616690672934055,
                    0.05876203253865242,
                    0.025769487023353577,
                    0.1274082511663437,
                    -0.008526468649506569,
                    0.0523807592689991,
                    -0.021058205515146255,
                    0.0638127401471138,
                    0.012579256668686867,
                    0.0010216827504336834,
                    0.0017412732122465968,
                    0.03155336529016495,
                    0.07130734622478485,
                    0.020949587225914,
                    0.011852878145873547,
                    -0.017650334164500237,
                    -0.04665118828415871,
                    0.02606818452477455,
                    0.05219067633152008,
                    0.015138555318117142,
                    0.0638127401471138,
                    0.11947917938232422,
                    0.03486619517207146,
                    0.010732760652899742,
                    0.048443377017974854,
                    0.08135446161031723,
                    0.027317285537719727,
                    -0.03242230415344238,
                    -0.05610090494155884,
                    -0.024941278621554375,
                    -0.07315384596586227,
                    -0.015749529004096985,
                    0.08265786617994308,
                    -0.00710086477920413,
                    0.007827243767678738,
                    -0.014242460951209068,
                    -0.10014527291059494,
                    -0.08037690073251724,
                    0.005037133581936359,
                    0.04700419679284096,
                    0.06821174919605255,
                    0.012233038432896137,
                    -0.060391295701265335,
                    0.0380975641310215,
                    -0.02600029855966568,
                    -0.02997841313481331,
                    0.007481025997549295,
                    0.015600179322063923,
                    -0.047438666224479675,
                    -0.0062726568430662155,
                    -0.009001670405268669,
                    0.011364099569618702,
                    0.023108359426259995,
                    -0.042442262172698975,
                    -0.004847052972763777,
                    0.025986721739172935,
                    -0.025375748053193092,
                    -0.12892889976501465,
                    -0.015559447929263115,
                    -0.042442262172698975,
                    -0.06473599374294281,
                    0.0039238049648702145,
                    0.06223779171705246,
                    -0.0316619835793972,
                    0.046678341925144196,
                    0.0022198688238859177,
                    -0.05197344347834587,
                    -0.030114183202385902,
                    0.022090071812272072,
                    -0.03220507130026817,
                    -0.025579405948519707,
                    0.02843061462044716,
                    -0.00403581652790308,
                    -0.012993360869586468,
                    -0.009402196854352951,
                    0.010420485399663448,
                    -0.03807041049003601,
                    0.03152621164917946,
                    0.036658383905887604,
                    0.041247472167015076,
                    -0.0626179501414299,
                    0.011805357411503792,
                    -0.004208925645798445,
                    -0.0409487746655941,
                    0.030032720416784286,
                    0.040894463658332825,
                    -0.054824650287628174,
                    0.013441408053040504,
                    -0.019686909392476082,
                    0.06685402989387512,
                    0.0020297884475439787,
                    -0.006713915150612593,
                    0.005695626605302095,
                    0.010705606080591679,
                    0.01421530731022358,
                    -0.04018845036625862,
                    -0.05297815427184105,
                    -0.0029937680810689926,
                    0.05387424677610397,
                    -0.010074267163872719,
                    -0.07646667212247849,
                    0.050941579043865204,
                    0.0440443716943264,
                    -0.006944727152585983,
                    0.0005159327993169427,
                    0.012219461612403393,
                    0.05800171196460724,
                    0.010529102757573128,
                    -0.017107246443629265,
                    0.04208925738930702,
                    0.018492117524147034,
                    -0.0916731134057045,
                    0.0020840971264988184,
                    0.0007569277659058571,
                    0.09330237656831741,
                    0.020908856764435768,
                    -0.08819735795259476,
                    0.12491004914045334,
                    -0.006978670135140419,
                    0.023827949538826942,
                    0.07782439142465591,
                    0.06913499534130096,
                    -0.035680826753377914,
                    0.011241904459893703,
                    0.004181771073490381,
                    0.0184513870626688,
                    -0.0031702714040875435,
                    0.02742590196430683,
                    -0.013332790695130825,
                    -0.06082576513290405,
                    0.04192633181810379,
                    -0.05669829994440079,
                    0.023637870326638222,
                    0.0745115578174591,
                    0.06506184488534927,
                    -0.003384111914783716,
                    -0.05740431323647499,
                    -0.020297883078455925,
                    -0.04073153808712959,
                    0.06044560298323631,
                    -0.017161555588245392,
                    -0.03421449288725853,
                    -0.04233364388346672,
                    -0.029272399842739105,
                    -0.043772827833890915,
                    -0.004704492632299662,
                    0.011757837608456612,
                    -0.05330400541424751,
                    -0.0490407720208168,
                    0.005895890295505524,
                    0.02302689664065838]

feature_2 = [-0.01052358653396368,
                    0.02793102152645588,
                    0.06894399225711823,
                    0.05285530537366867,
                    -0.017407435923814774,
                    -0.029460765421390533,
                    -0.04067010059952736,
                    0.0133588882163167,
                    0.02806289680302143,
                    0.03502587229013443,
                    0.006877255626022816,
                    0.027825521305203438,
                    -0.02740352414548397,
                    0.02740352414548397,
                    0.07806970924139023,
                    0.024647345766425133,
                    0.0478704497218132,
                    0.06213926896452904,
                    -0.030805885791778564,
                    0.06873299926519394,
                    -0.06182277202606201,
                    0.02319672703742981,
                    0.010754366405308247,
                    -0.015402942895889282,
                    0.02504296973347664,
                    -0.03434012457728386,
                    0.12860402464866638,
                    0.06978799402713776,
                    0.04372958838939667,
                    -0.06092602387070656,
                    0.10344237089157104,
                    0.07448272407054901,
                    0.01997898891568184,
                    0.0024281395599246025,
                    0.01711731217801571,
                    0.024146223440766335,
                    -0.0035144558642059565,
                    -0.05298718065023422,
                    -0.006817912217229605,
                    -0.020150424912571907,
                    0.010009275749325752,
                    0.006672850344330072,
                    -0.060134779661893845,
                    0.04190972074866295,
                    0.010906022042036057,
                    0.029091518372297287,
                    0.07353322952985764,
                    0.02256372943520546,
                    0.0015239748172461987,
                    0.007919064722955227,
                    0.003247409826144576,
                    -0.009389465674757957,
                    0.07743671536445618,
                    0.032863128930330276,
                    -0.02795739658176899,
                    -0.048081446439027786,
                    -0.07933570444583893,
                    -0.05865778401494026,
                    0.04019535332918167,
                    -0.04333396628499031,
                    0.00998949445784092,
                    -0.01967567764222622,
                    -0.023328600451350212,
                    0.07590696960687637,
                    0.11583857238292694,
                    -0.05211680755019188,
                    -0.11721006780862808,
                    0.06746700406074524,
                    0.04578683152794838,
                    -0.030225638300180435,
                    -0.018304182216525078,
                    0.06440751254558563,
                    -0.029566265642642975,
                    -0.0026490292511880398,
                    -0.06066227704286575,
                    0.05602029338479042,
                    -0.07247823476791382,
                    0.023842912167310715,
                    0.032177381217479706,
                    -0.004150750115513802,
                    0.0026869431603699923,
                    -0.014545759186148643,
                    0.05011231452226639,
                    -0.00013640770339407027,
                    -0.024001160636544228,
                    0.04125034809112549,
                    0.048714444041252136,
                    0.05586204305291176,
                    -0.0032012537121772766,
                    -0.02677052654325962,
                    -0.027746398001909256,
                    -0.013068764470517635,
                    0.011730237863957882,
                    -0.000713770801667124,
                    -0.016695313155651093,
                    -0.033496126532554626,
                    -0.07965220510959625,
                    -0.06435476243495941,
                    0.031518008559942245,
                    -0.00713441101834178,
                    0.035869866609573364,
                    0.03766335919499397,
                    -0.03861285746097565,
                    0.004625498317182064,
                    0.0012890733778476715,
                    0.003857329487800598,
                    -0.0023308820091187954,
                    0.014994132332503796,
                    0.027588147670030594,
                    0.016418376937508583,
                    0.06451301276683807,
                    -0.019913051277399063,
                    -0.06493501365184784,
                    -0.020097676664590836,
                    0.01871299371123314,
                    -0.027218898758292198,
                    0.028326645493507385,
                    -0.039113979786634445,
                    0.04251634329557419,
                    0.09626839309930801,
                    -0.01934599131345749,
                    0.004404608625918627,
                    0.0074377222917973995,
                    0.02170654572546482,
                    -0.0017473372863605618,
                    -0.07073748856782913,
                    -0.011182958260178566,
                    -0.004282624926418066,
                    -0.0021825232543051243,
                    -0.026269402354955673,
                    -0.017024999484419823,
                    0.025623217225074768,
                    0.04146134853363037,
                    0.08007420599460602,
                    0.05897428095340729,
                    -0.09964438527822495,
                    0.038480982184410095,
                    -0.07965220510959625,
                    0.011097240261733532,
                    -0.09974987804889679,
                    0.06150627136230469,
                    0.02795739658176899,
                    -0.08123470097780228,
                    0.03766335919499397,
                    -0.013016014359891415,
                    -0.006257445551455021,
                    -0.007444316055625677,
                    0.023948412388563156,
                    0.00207702349871397,
                    -0.015666691586375237,
                    0.005795884877443314,
                    0.039061229676008224,
                    0.09189016371965408,
                    0.001744040404446423,
                    0.10196537524461746,
                    0.047975946217775345,
                    -0.020651549100875854,
                    0.032652128487825394,
                    0.035210493952035904,
                    -0.021205421537160873,
                    0.017987683415412903,
                    0.04649895429611206,
                    0.03863923251628876,
                    0.03064763732254505,
                    -0.02426491118967533,
                    -0.00245286594145,
                    0.09373640269041061,
                    -0.005848634522408247,
                    0.022827478125691414,
                    0.05454329773783684,
                    0.017961308360099792,
                    -0.0031155352480709553,
                    -0.003468299750238657,
                    -0.038375481963157654,
                    -0.012554453685879707,
                    -0.02745627425611019,
                    -0.03249387815594673,
                    0.008650967851281166,
                    0.008169625885784626,
                    0.06314151734113693,
                    -0.05549279600381851,
                    -0.02553090639412403,
                    0.10001362860202789,
                    0.022801103070378304,
                    -0.04813419654965401,
                    0.021864794194698334,
                    0.025491343811154366,
                    -0.021640608087182045,
                    -0.04251634329557419,
                    -0.05786653608083725,
                    0.011756612919270992,
                    0.07110673934221268,
                    -0.05739178881049156,
                    -0.000454967055702582,
                    -0.03420824930071831,
                    -0.02471328340470791,
                    0.0047738575376570225,
                    0.008796029724180698,
                    -0.009165278635919094,
                    0.020071301609277725,
                    0.08846142143011093,
                    0.07580146938562393,
                    -0.02633533999323845,
                    -0.024924281984567642,
                    -0.08039070665836334,
                    0.011328021064400673,
                    -0.03104325942695141,
                    -0.027060650289058685,
                    0.04639345407485962,
                    -0.022497791796922684,
                    -0.02281429059803486,
                    -0.03732048720121384,
                    -0.005400261376053095,
                    -0.05665329098701477,
                    -0.03262575343251228,
                    0.012013767845928669,
                    -0.03172900900244713,
                    -0.018422869965434074,
                    -0.03122788481414318,
                    0.013860011473298073,
                    -0.030041014775633812,
                    -0.009415839798748493,
                    -0.011756612919270992,
                    -0.03642373904585838,
                    0.0425427183508873,
                    -0.01556119229644537,
                    0.01898992992937565,
                    0.1137285828590393,
                    0.010048838332295418,
                    -0.006415694952011108,
                    -0.024370409548282623,
                    -0.045549456030130386,
                    -0.02801014669239521,
                    -0.05544004589319229,
                    0.0336807519197464,
                    -0.01618100330233574,
                    -0.007846533320844173,
                    -0.03863923251628876,
                    -0.05172118544578552,
                    -0.015099631622433662,
                    0.048608943819999695,
                    0.044863708317279816,
                    -0.07574871927499771,
                    0.029065143316984177,
                    -0.04251634329557419,
                    -0.007688283920288086,
                    0.005077168811112642,
                    0.05617854371666908,
                    0.026045216247439384,
                    0.0800214558839798,
                    -0.08376669138669968,
                    0.02408028580248356,
                    0.1379934847354889,
                    -0.009264184162020683,
                    -0.05412130057811737,
                    -0.10006637871265411,
                    -0.005954134277999401,
                    -0.024502284824848175,
                    -0.04705282673239708,
                    0.027588147670030594,
                    -0.06192827224731445,
                    -0.034762121737003326,
                    0.06398551166057587,
                    0.07870271056890488,
                    0.05907978117465973,
                    0.05019143968820572,
                    -0.030885010957717896,
                    0.03375987708568573,
                    -0.007015724200755358,
                    0.06182277202606201,
                    0.05074531212449074,
                    -0.0863514319062233,
                    0.00024067098274827003,
                    -0.01995261386036873,
                    0.07479922473430634,
                    -0.1182650625705719,
                    0.017209623008966446,
                    0.008439969271421432,
                    -0.02605840377509594,
                    -0.03486762195825577,
                    0.010477430187165737,
                    0.0336807519197464,
                    -0.07627622038125992,
                    -0.04312296584248543,
                    -0.03054213710129261,
                    0.04515383392572403,
                    0.010325774550437927,
                    0.013609449379146099,
                    0.03310050442814827,
                    -0.03431374952197075,
                    -0.06730875372886658,
                    -0.011604957282543182,
                    -0.022023044526576996,
                    0.06820549815893173,
                    0.052512429654598236,
                    0.016959061846137047,
                    -0.04839794710278511,
                    -0.048081446439027786,
                    0.03958872705698013,
                    0.015073256567120552,
                    0.04014260321855545,
                    -0.014216072857379913,
                    -0.05997652933001518,
                    -0.04512745887041092,
                    -0.07954670488834381,
                    -0.035315994173288345,
                    0.07812245935201645,
                    -0.018699806183576584,
                    0.007141004782170057,
                    0.027588147670030594,
                    0.0024792407639324665,
                    0.024950657039880753,
                    -0.044810958206653595,
                    0.10497210919857025,
                    0.026045216247439384,
                    -0.0004714513779617846,
                    0.03447199612855911,
                    -0.008664155378937721,
                    -0.010807116515934467,
                    0.0009231215808540583,
                    0.04080197587609291,
                    -0.05966002866625786,
                    -0.0068047246895730495,
                    0.013438012450933456,
                    -0.02983001433312893,
                    0.05346192792057991,
                    0.03940410539507866,
                    -0.02555728144943714,
                    -0.03507861867547035,
                    -0.01661618798971176,
                    -0.022708790376782417,
                    -0.022036230191588402,
                    0.01618100330233574,
                    -0.05303993076086044,
                    -0.06018752604722977,
                    0.05485979840159416,
                    -0.0002476768277119845,
                    0.05517629534006119,
                    -0.037900734692811966,
                    0.040379974991083145,
                    -0.017381060868501663,
                    -0.03573799133300781,
                    -0.06324701756238937,
                    0.03610724210739136,
                    -0.04773857444524765,
                    -0.05232780799269676,
                    -0.06620100885629654,
                    -0.07806970924139023,
                    0.031438883394002914,
                    0.07416622340679169,
                    -0.06883849203586578,
                    -0.0015083147445693612,
                    -0.0004945294349454343,
                    0.05765553563833237,
                    -0.0009560902253724635,
                    -0.014216072857379913,
                    0.01267314050346613,
                    -0.11974205821752548,
                    0.012831389904022217,
                    -0.024950657039880753,
                    0.06915499269962311,
                    -0.01320723257958889,
                    0.023354975506663322,
                    -0.030911386013031006,
                    0.003672705264762044,
                    0.06087327376008034,
                    0.10048837959766388,
                    0.00749047240242362,
                    0.031016886234283447,
                    -0.00445076497271657,
                    0.01691949926316738,
                    -0.11393957585096359,
                    0.017512936145067215,
                    0.04876719415187836,
                    -0.034762121737003326,
                    0.02917064167559147,
                    0.00761575298383832,
                    0.07073748856782913,
                    -0.04715832695364952,
                    -0.029698140919208527,
                    0.0622975192964077,
                    -0.034656621515750885,
                    0.03257300332188606,
                    -0.000599204795435071,
                    0.07464097440242767,
                    -0.03172900900244713,
                    0.035948991775512695,
                    0.001338526257313788,
                    -0.014387509785592556,
                    0.027825521305203438,
                    -0.06730875372886658,
                    0.04074922576546669,
                    0.015033694915473461,
                    0.014862257987260818,
                    0.034735746681690216,
                    -0.003224331885576248,
                    0.020163612440228462,
                    -0.019609740003943443,
                    -0.03956235200166702,
                    -0.032230131328105927,
                    0.01691949926316738,
                    0.042885590344667435,
                    0.019833926111459732,
                    -0.03513136878609657,
                    -0.016748063266277313,
                    0.03415549919009209,
                    0.04441533610224724,
                    0.035184118896722794,
                    -0.010635679587721825,
                    -0.03742598742246628,
                    -0.016220565885305405,
                    0.02542540617287159,
                    -0.004341968335211277,
                    -0.022735165432095528,
                    0.09099341183900833,
                    0.009613651782274246,
                    -0.014519384130835533,
                    0.05224868282675743,
                    0.019266866147518158,
                    0.008070720359683037,
                    0.0939474031329155,
                    0.0058881971053779125,
                    -0.022656042128801346,
                    0.04454721137881279,
                    0.00356720550917089,
                    -0.020783422514796257,
                    -0.0012528079096227884,
                    -0.009046591818332672,
                    0.03935135528445244,
                    0.09305065870285034,
                    -0.03046301193535328,
                    0.023262664675712585,
                    0.01717006228864193,
                    0.09484414756298065,
                    -0.043624088168144226,
                    -0.0031699335668236017,
                    -0.01898992992937565,
                    -0.07400797307491302,
                    -0.035817116498947144,
                    0.021640608087182045,
                    0.009778494946658611,
                    0.028221145272254944,
                    0.008710311725735664,
                    0.03122788481414318,
                    -0.001094558509066701,
                    0.02977726422250271,
                    0.025768280029296875,
                    -0.06461851298809052,
                    -0.0244363471865654,
                    -0.09273415803909302,
                    -0.10091038048267365,
                    -0.027720022946596146,
                    -0.007503659930080175,
                    0.057550039142370224,
                    0.012178611010313034,
                    -0.021033985540270805,
                    0.028801392763853073,
                    0.07458822429180145,
                    -0.0622975192964077,
                    -0.06324701756238937,
                    0.013286356814205647,
                    -0.006722303107380867,
                    -0.0181063711643219,
                    -0.017657997086644173,
                    -0.007332223001867533,
                    -0.0828171968460083,
                    -0.03381262347102165,
                    0.039113979786634445,
                    -0.03315325081348419,
                    -0.06045127660036087,
                    0.028405770659446716,
                    0.023671474307775497,
                    -0.02074385993182659,
                    -0.040907472372055054,
                    -0.0007286066538654268,
                    0.03834910690784454,
                    0.01465125847607851,
                    0.014440258964896202,
                    -0.02539903111755848,
                    0.013292950578033924,
                    -0.012653359211981297,
                    0.027113400399684906,
                    -0.0860876813530922,
                    -0.05222230777144432,
                    0.02917064167559147,
                    -0.04900456964969635,
                    0.07226723432540894,
                    0.03998435288667679,
                    0.03312687575817108,
                    0.021970294415950775,
                    0.039799727499485016,
                    0.06815274804830551,
                    -0.02360553853213787,
                    -0.05280255526304245,
                    -0.08930542320013046,
                    -0.07253098487854004,
                    -0.008472937159240246,
                    0.0183569323271513,
                    0.04256909340620041,
                    -0.003797986079007387,
                    0.005096950102597475,
                    0.04322846606373787,
                    -0.014994132332503796,
                    0.02094167284667492,
                    -0.02157467044889927,
                    -0.02162742055952549,
                    -0.008433375507593155,
                    -0.03354887664318085,
                    0.04459996148943901,
                    -0.01747337356209755,
                    0.045364830642938614,
                    0.05235418304800987]

insert_data_list = []
insert_data_list.append({"user_id" : "63",
                         "face_feature" : feature_1,
                         "user_attr" : {"age": 28, "name": "mery"}})

insert_data_list.append({"user_id" : "92",
                         "face_feature" : feature_2,
                         "user_attr" : {"age": 25, "name": "jck"}})

insert_result = client.insert(collection_name=collection_name, data=insert_data_list)

print(insert_result)


# delete one user
delete_result = client.delete(collection_name=collection_name, ids=["63"])
print(delete_result)

# search 
# load before search
state_dict = client.get_load_state(collection_name=collection_name)
state = state_dict["state"]
if state != LoadState.Loaded :
    client.load_collection(collection_name=collection_name)

search_params = {
    "metric_type": "COSINE",
    "params": {
        "radius": 0.6, 
        "range_filter": 1.0 
        }
    }

search_vector = [ feature_1, feature_2]

search_result = client.search(collection_name=collection_name,
                              data=search_vector,
                              limit=1,
                              search_params=search_params,
                              output_fields=["user_attr"],
                              consistency_level=ConsistencyLevel.Strong)

print(search_result)

# release collection
client.release_collection(collection_name=collection_name)

# load again
client.load_collection(collection_name=collection_name)

# search again

search_result = client.search(collection_name=collection_name,
                              data=search_vector,
                              limit=1,
                              search_params=search_params,
                              output_fields=["user_attr"],
                              consistency_level=ConsistencyLevel.Strong)

print(search_result)

Copy link

stale bot commented Dec 21, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

@stale stale bot added the stale indicates no udpates for 30 days label Dec 21, 2024
@Harleyzheng
Copy link

Harleyzheng commented Dec 25, 2024

Encountered this issue in version: "@zilliz/milvus2-sdk-node": "^2.4.9". Fixed when downgrading to v2.3.15

@stale stale bot removed the stale indicates no udpates for 30 days label Dec 25, 2024
@xiaofan-luan
Copy link
Collaborator

Encountered this issue in version: "@zilliz/milvus2-sdk-node": "^2.4.9". Fixed when downgrading to v2.3.15

please try latest 2.4.19 and all the known deleted issue has been fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Issues or changes related a bug triage/needs-information Indicates an issue needs more information in order to work on it.
Projects
None yet
Development

No branches or pull requests

5 participants