[Bug]: [benchmark][retrieve] After load fails, the data on disk is not released

[wangting0128]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version:master-20241223-f499ca47-amd64
- Deployment mode(standalone or cluster):standalone
- MQ type(rocksmq, pulsar or kafka):rocksmq    
- SDK version(e.g. pymilvus v2.0.0rc2):2.5.0rc124
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

AWS EC2

server:

NAME                                               READY   STATUS      RESTARTS   AGE     IP             NODE                                         NOMINATED NODE   READINESS GATES
retrieve-perf-etcd-0                               1/1     Running     0          25h     10.15.43.23    ip-10-15-33-8.us-west-2.compute.internal     <none>           <none>
retrieve-perf-etcd-1                               1/1     Running     0          25h     10.15.30.59    ip-10-15-25-170.us-west-2.compute.internal   <none>           <none>
retrieve-perf-etcd-2                               1/1     Running     0          25h     10.15.33.203   ip-10-15-43-90.us-west-2.compute.internal    <none>           <none>
retrieve-perf-milvus-standalone-7c66958b6b-ssbfd   1/1     Running     0          25h     10.15.32.53    ip-10-15-40-67.us-west-2.compute.internal    <none>           <none>

After the load fails, the 39G data on the disk is not released

Expected Behavior

No response

Steps To Reproduce

1. create a collection with fields: 'id'(INT64, primary key), 'sparse_float_vector'
2. build SPARSE_INVERTED_INDEX index on field sparse_float_vector
3. insert 40m data
4. flush collection
5. rebuild index
6. load collection <- `raises error, but data on disk is not released`

Milvus Log

No response

Anything else?

server config:

cluster:
  enabled: false
etcd:
  image:
    registry: harbor-us-vdc.zilliz.cc
    repository: milvus/etcd
    tag: 3.5.16-r1
  metrics:
    enabled: true
    podMonitor:
      enabled: true
  replicaCount: 3
externalS3:
  accessKey: ***
  bucketName: ***
  enabled: true
  host: ***
  port: "443"
  secretKey: ***
  useSSL: true
extraConfigFiles:
  user.yaml: |
    localStorage:
      path: /milvus-data
    dataCoord:
      segment:
        sealProportion: 1
    indexCoord:
      scheduler:
        interval: 1
    queryNode:
      cache:
        warmup: sync
      mmap:
        vectorField: true
        vectorIndex: true
        scalarField: true
        scalarIndex: true
image:
  all:
    repository: milvusdb/milvus
    tag: master-20241223-f499ca47-amd64
log:
  level: info
metrics:
  serviceMonitor:
    enabled: true
minio:
  enabled: false
nodeSelector:
  kubernetes.io/hostname: ***
  wt: "true"
pulsarv3:
  enabled: false
standalone:
  resources:
    limits:
      cpu: "2.0"
      memory: 16Gi
    requests:
      cpu: "1.0"
      memory: 10Gi
tolerations:
- effect: NoSchedule
  key: wt
  operator: Equal
  value: "true"
volumeMounts:
- mountPath: /milvus-data
  name: milvus-data-volume
volumes:
- hostPath:
    path: /mnt/data
    type: Directory
  name: milvus-data-volume

client config:

{
     "dataset_params": {
          "metric_type": "IP",
          "vector_field_name": "sparse_float_vector",
          "dim": 30000,
          "sparse_range": [
               100,
               150
          ],
          "dataset_name": "local",
          "dataset_size": "40m",
          "ni_per": 20000
     },
     "collection_params": {
          "shards_num": 1,
          "collection_name": "scene_sparse_float_vector"
     },
     "index_params": {
          "index_type": "SPARSE_INVERTED_INDEX",
          "index_param": {
               "drop_ratio_build": 0.2
          }
     },
     "concurrent_params": {
          "concurrent_number": [
               10,
               100
          ],
          "during_time": "30m",
          "interval": 20
     },
     "concurrent_tasks": [
          {
               "type": "query",
               "weight": 1,
               "params": {
                    "expr": "",
                    "output_fields": [
                         "sparse_float_vector"
                    ],
                    "limit": 10,
                    "timeout": 600,
                    "random_data": true,
                    "random_count": 10,
                    "random_range": [
                         0,
                         40000000
                    ],
                    "field_name": "id",
                    "field_type": "int64"
               }
          }
     ]
}

[zhengbuqian]

zilliztech/knowhere#1005