[Bug]: If there are duplicate primary keys in the backed-up collection, after restoring the collection, the query results will show earlier entities instead of the latest ones.

[zhuwenxing]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version:2.3-20240801-923278b7-amd64
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

FAILED testcases/test_restore_backup.py::TestRestoreBackup::test_milvus_restore_back_with_dup_pk - AssertionError: src: {'int_array': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 'float_vector': [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.0, 29.0, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.0, 56.0, 57.0, 58.0, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0, 101.0, 102.0, 103.0, 104.0, 105.0, 106.0, 107.0, 108.0, 109.0, 110.0, 111.0, 112.0, 113.0, 114.0, 115.0, 116.0, 117.0, 118.0, 119.0, 120.0, 121.0, 122.0, 123.0, 124.0, 125.0, 126.0, 127.0], 'int64': 0, 'key': 0, 'json': {'key_3000': 3000}, 'var_array': ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']}, dist: {'key': 0, 'json': {'key_0': 0}, 'var_array': ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'], 'int_array': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 'float_vector': [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.0, 29.0, 30.0, 31.0, 32.0, 33.0, 34.0, 35.0, 36.0, 37.0, 38.0, 39.0, 40.0, 41.0, 42.0, 43.0, 44.0, 45.0, 46.0, 47.0, 48.0, 49.0, 50.0, 51.0, 52.0, 53.0, 54.0, 55.0, 56.0, 57.0, 58.0, 59.0, 60.0, 61.0, 62.0, 63.0, 64.0, 65.0, 66.0, 67.0, 68.0, 69.0, 70.0, 71.0, 72.0, 73.0, 74.0, 75.0, 76.0, 77.0, 78.0, 79.0, 80.0, 81.0, 82.0, 83.0, 84.0, 85.0, 86.0, 87.0, 88.0, 89.0, 90.0, 91.0, 92.0, 93.0, 94.0, 95.0, 96.0, 97.0, 98.0, 99.0, 100.0, 101.0, 102.0, 103.0, 104.0, 105.0, 106.0, 107.0, 108.0, 109.0, 110.0, 111.0, 112.0, 113.0, 114.0, 115.0, 116.0, 117.0, 118.0, 119.0, 120.0, 121.0, 122.0, 123.0, 124.0, 125.0, 126.0, 127.0], 'int64': 0}

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

failed job: https://qa-jenkins.milvus.io/blue/organizations/jenkins/milvus_backup_api_test/detail/milvus_backup_api_test/62/pipeline

cluster:4am
ns: chaos-testing
pod info

[2024-08-01T08:37:21.971Z] + kubectl get pods -o wide

[2024-08-01T08:37:21.972Z] + grep backup-test-62

[2024-08-01T08:37:22.228Z] backup-test-62-etcd-0                                             1/1     Running       0              3m2s    10.104.24.101   4am-node29   <none>           <none>

[2024-08-01T08:37:22.228Z] backup-test-62-milvus-standalone-767579df79-p65px                 1/1     Running       0              3m2s    10.104.24.100   4am-node29   <none>           <none>

[2024-08-01T08:37:22.228Z] backup-test-62-minio-598788b55-m8xt7                              1/1     Running       0              3m2s    10.104.24.99    4am-node29   <none>           <none>

Anything else?

No response

[zhuwenxing]

/assign @bigsheeper

PTAL

[xiaofan-luan]

Let's skip any duplicated Pk issue for now.
This is not something we can handle right now

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.