[Bug]: [null & default]Query with expression "nullableFid == 0" on nullable field should not filter "None"

[qixuan0212]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version: master-20240909-73a34a5a-amd64
- Deployment mode(standalone or cluster):standalone
- MQ type(rocksmq, pulsar or kafka):  all  
- SDK version(e.g. pymilvus v2.0.0rc2): 2.5.0rc78
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Querywith expression "nullableFid == 0" on nullable field can filter "None"
data: ["{'int32': 10, 'int64': 1, 'nullableFid': None}", "{'int32': 10, 'int64': 2, 'nullableFid': None}"]

Expected Behavior

No "None" data is queried using expression "nullableFid == 0"

Steps To Reproduce

from pymilvus import CollectionSchema, FieldSchema
from pymilvus import Collection
from pymilvus import connections
from pymilvus import DataType
from pymilvus import Partition
from pymilvus import utility
import json
import random

connections.connect()

dim = 128
int64_field = FieldSchema(name="int64", dtype=DataType.INT64, is_primary=True)
double_field = FieldSchema(name="nullableFid", dtype=DataType.DOUBLE, nullable=True)
int32_field = FieldSchema(name="int32", dtype=DataType.INT64, default_value=10)
float_vector = FieldSchema(name="float_vector", dtype=DataType.FLOAT_VECTOR, dim=dim)
schema = CollectionSchema(fields=[int64_field, double_field, int32_field, float_vector])
utility.drop_collection("test")
collection = Collection("test", schema=schema)
res = collection.schema
print(res)

index = {"index_type": "DISKANN", "metric_type": "L2", "params": {}}

nb = 2
vectors = [[random.random() for _ in range(dim)] for _ in range(nb)]
data = [[1, 2], [3.0, None], [4, None], vectors]
#  equals to data1 = [[1,2], [None,None],[None,None], vectors]
data1 = [[1, 2], [], [], vectors]

collection.insert(data=data1)
# collection.upsert(data=data1)
collection.create_index("float_vector", index, index_name="index_name_1")
collection.load()
collection.flush()
res = collection.num_entities
print(res)
default_search_params = {"metric_type": "L2", "params": {}}
limit = 10
nq = 1
import time
#
# start = time.time()
# res1 = collection.search(vectors[:nq], "float_vector", default_search_params, limit, "int64 >= 0",
#                          output_fields=["nullableFid", "int32"])
# end = time.time() - start
# print(res1)
# print("search successfully in %f s" % end)
start = time.time()
res = collection.query("nullableFid == 0", output_fields=["nullableFid", "int32"])
end = time.time() - start
print(res)
print("query successfully in %f s" % end)

Milvus Log

https://grafana-4am.zilliz.cc/explore?orgId=1&panes=%7B%22rKe%22:%7B%22datasource%22:%22vhI6Vw67k%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bcluster%3D%5C%22devops%5C%22,namespace%3D%5C%22chaos-testing%5C%22,pod%3D~%5C%22milvus-for-qxtest-gkbhi.%2A%5C%22%7D%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22vhI6Vw67k%22%7D%7D%5D,%22range%22:%7B%22from%22:%221725890150730%22,%22to%22:%221725890160429%22%7D%7D%7D&schemaVersion=1

Anything else?

No response

[yanliang567]

/assign @smellthemoon
/unassign

[smellthemoon]

Expr in null value, waiting #35527

[binbinlv]

/assign @qixuan0212 please help to verify

[qixuan0212]

Verified and fixed:

milvus: master-20241019-ef923958
pymilvus: 2.5.0rc89

results:

res = collection.query("nullableFid == 0", output_fields=["nullableFid", "int32"])
print(res)
data: []