Skip to content

Commit

Permalink
Release new docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Milvus-doc-bot authored and Milvus-doc-bot committed Sep 22, 2023
1 parent 76f7740 commit cc50228
Show file tree
Hide file tree
Showing 6 changed files with 33 additions and 20 deletions.
4 changes: 1 addition & 3 deletions v2.3.x/site/en/faq/product_faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,7 @@ No. Milvus has varied behavior. Data must be loaded to memory before searching.

#### Can indexes be created after inserting vectors?

Yes. If `create_index()` is called, Milvus builds an index for subsequently inserted vectors. However, Milvus does not build an index until the newly inserted vectors fill an entire segment and the newly created index file is separate from the previous one.


Yes. If an index has been built for a collection by `create_index()` before, Milvus will automatically build an index for subsequently inserted vectors. However, Milvus does not build an index until the newly inserted vectors fill an entire segment and the newly created index file is separate from the previous one.

#### How are the FLAT and IVF_FLAT indexes different?

Expand Down
2 changes: 2 additions & 0 deletions v2.3.x/site/en/userGuide/bulk_insert.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,8 @@ In this method, you need to set the name of the target collection as **collectio

</div>

After inserting entities into a collection that has previously been indexed, you do not need to re-index the collection, as Milvus will automatically create an index for the newly inserted data. For more information, refer to [Can indexes be created after inserting vectors?](product_faq.md#Can-indexes-be-created-after-inserting-vectors)

## List tasks

### Check task state
Expand Down
10 changes: 8 additions & 2 deletions v2.3.x/site/en/userGuide/insert_data.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This topic describes how to insert data in Milvus via client.

You can also migrate data to Milvus with [MilvusDM](migrate_overview.md), an open-source tool designed specifically for importing and exporting data with Milvus.

Milvus 2.1 supports VARCHAR data type on scalar field. When building indexes for VARCHAR-type scalar fields, the default index type is dictionary tree.
Milvus 2.1 supports the `VARCHAR` data type on scalar fields. When building indexes for VARCHAR-type scalar fields, the default index type is dictionary tree.

The following example inserts 2,000 rows of randomly generated data as the example data (Milvus CLI example uses a pre-built, remote CSV file containing similar data). Real applications will likely use much higher dimensional vectors than the example. You can prepare your own data to replace the example.

Expand Down Expand Up @@ -373,9 +373,15 @@ Output:
</tbody>
</table>

<div class="alert note">

After inserting entities into a collection that has previously been indexed, you do not need to re-index the collection, as Milvus will automatically create an index for the newly inserted data. For more information, refer to [Can indexes be created after inserting vectors?](product_faq.md#Can-indexes-be-created-after-inserting-vectors)

</div>

## Flush the Data in Milvus

When data is inserted into Milvus it is inserted into segments. Segments have to reach a certain size to be sealed and indexed. Unsealed segments will be searched brute force. In order to avoid this with any remainder data, it is best to call flush(). The flush call will seal any remaining segments and send them for indexing. It is important to only call this at the end of an insert session, as calling this too much will cause fragmented data that will need to be cleaned later on.
When data is inserted into Milvus it is inserted into segments. Segments have to reach a certain size to be sealed and indexed. Unsealed segments will be searched brute force. In order to avoid this with any remainder data, it is best to call `flush()`. The `flush()` call will seal any remaining segments and send them for indexing. It is important to only call this method at the end of an insert session. Calling it too often will cause fragmented data that will need to be cleaned later on.


## Limits
Expand Down
10 changes: 5 additions & 5 deletions v2.3.x/site/en/userGuide/search/query.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ Output:
</tr>
<tr>
<td><code>offset</code></td>
<td>Number of results to skip in the returned set. This parameter is available only when <code>limit</code> is specified, and the sum of this value and <code>limit</code> should be less than 16384.</td>
<td>Number of results to skip in the returned set. This parameter is available only when <code>limit</code> is specified, and the sum of this value and <code>limit</code> should be less than 16384. For example, if you want the 9th and 10th nearest neighbors to the query vector, set <code>limit</code> to <code>2</code> and <code>offset</code> to <code>8</code>.</td>
</tr>
<tr>
<td><code>output_fields</code> (optional)</td>
Expand Down Expand Up @@ -246,7 +246,7 @@ Output:
</tr>
<tr>
<td><code>offset</code> (optional)</td>
<td>Number of results to skip in the returned set. This parameter is available only when <code>limit</code> is specified, and the sum of this value and <code>limit</code> should be less than 16384.</td>
<td>Number of results to skip in the returned set. This parameter is available only when <code>limit</code> is specified, and the sum of this value and <code>limit</code> should be less than 16384. For example, if you want the 9th and 10th nearest neighbors to the query vector, set <code>limit</code> to <code>2</code> and <code>offset</code> to <code>8</code>.</td>
</tr>
</tbody>
</table>
Expand Down Expand Up @@ -328,7 +328,7 @@ Output:
</tr>
<tr>
<td><code>Offset</code> (optional)</td>
<td>Number of results to skip in the returned set. This parameter is available only when <code>limit</code> is specified, and the sum of this value and <code>limit</code> in <code>WithLimit()</code> should be less than 16384.</td>
<td>Number of results to skip in the returned set. This parameter is available only when <code>limit</code> is specified, and the sum of this value and <code>limit</code> in <code>WithLimit()</code> should be less than 16384. For example, if you want the 9th and 10th nearest neighbors to the query vector, set <code>limit</code> to <code>2</code> and <code>offset</code> to <code>8</code>.</td>
</tr>
<tr>
<td><code>ConsistencyLevel</code></td>
Expand Down Expand Up @@ -373,11 +373,11 @@ Output:
</tr>
<tr>
<td><code>limit</code></td>
<td>The maximum number of entities to return.<br>The sum of this value of that of `offset` should be less than **1024**.<br>The value defaults to <code>100</code>.<br>The value ranges from <code>1</code> to <code>100</code></td>
<td>The maximum number of entities to return.<br>The sum of this value of that of <code>offset</code> should be less than <code>1024</code>.<br>The value defaults to <code>100</code>.<br>The value ranges from <code>1</code> to <code>100</code></td>
</tr>
<tr>
<td><code>offset</code></td>
<td>The number of entities to skip in the search results.<br>The sum of this value and that of `limit` should not be greater than <code>1024</code>.<br>The maximum value is <code>1024</code>.</td>
<td>The number of entities to skip in the search results.<br>The sum of this value and that of <code>limit</code> should not be greater than <code>1024</code>.<br>The maximum value is <code>1024</code>. For example, if you want the 9th and 10th nearest neighbors to the query vector, set <code>limit</code> to <code>2</code> and <code>offset</code> to <code>8</code>.</td>
</tr>
<tr>
<td><code>outputFields</code></td>
Expand Down
19 changes: 10 additions & 9 deletions v2.3.x/site/en/userGuide/search/search.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ Output:
</tr>
<tr>
<td><code>offset</code></td>
<td>Number of entities to skip during the search. The sum of this parameter and <code>limit</code> of the <code>search</code> method should be less than <code>16384</code>.</td>
<td>Number of entities to skip during the search. The sum of this value and <code>limit</code> of the <code>search</code> method should be less than <code>16384</code>. For example, if you want the 9th and 10th nearest neighbors to the query vector, set <code>limit</code> to <code>2</code> and <code>offset</code> to <code>8</code>.</td>
</tr>
<tr>
<td><code>ignore_growing</code></td>
Expand Down Expand Up @@ -277,7 +277,7 @@ Output:
<td>Options for ANN searches.</td>
<td><ul>
<li><code>Limit</code> Indicates the number of entities to return.</li>
<li><code>Offset</code> Indicates the number of entities to skip during the search. The sum of this parameter and <code>Limit</code> should be less than <code>16384</code>.</li>
<li><code>Offset</code> Indicates the number of entities to skip during the search. The sum of this parameter and <code>Limit</code> should be less than <code>16384</code>. For example, if you want the 9th and 10th nearest neighbors to the query vector, set <code>limit</code> to <code>2</code> and <code>offset</code> to <code>8</code>.</li>
<li><code>ConsistencyLevel</code> Indicates the consistency level applied during the search.</li>
<li><code>Ignore Growing</code> Indicates whether to ignore growing segments during similarity searches. The value defaults to <code>False</code>, indicating that searches involve growing segments. </li>
</ul></td>
Expand Down Expand Up @@ -306,7 +306,8 @@ Output:
<li><code>nprobe</code> Indicates the number of cluster units to search. This parameter is available only when <code>index_type</code> is set to <code>IVF_FLAT</code>, <code>IVF_SQ8</code>, or <code>IVF_PQ</code>. The value should be less than <code>nlist</code> specified for the index-building process.</li>
<li><code>ef</code> Indicates the search scope. This parameter is available only when <code>index_type</code> is set to <code>HNSW</code>. The value should be within the range from <code>top_k</code> to <code>32768</code>.</li>
<li><code>metric_type</code> Indicates the metric type used in the search. It should be the same as the one specified when you index the collection.</li>
<li><code>offset</code> Indicates the number of entities to skip during the search. The sum of this parameter and <code>topK</code> of the <code>withTopK()</code> method should be less than <code>16384</code>.</li>
<li><code>limit</code> Indicates the number of entities to return starting from the last skippped entity.</li>
<li><code>offset</code> Indicates the number of entities to skip during the search. The sum of this value and <code>topK</code> of the <code>withTopK()</code> method should be less than <code>16384</code>. For example, if you want the 9th and 10th nearest neighbors to the query vector, set <code>topK</code> to <code>2</code> and <code>offset</code> to <code>8</code>.</li>
</ul></td>
</tr>
</tbody>
Expand Down Expand Up @@ -348,11 +349,11 @@ Output:
</tr>
<tr>
<td><code>limit</code></td>
<td>The maximum number of entities to return.<br>The sum of this value of that of `offset` should be less than **1024**.<br>The value defaults to <code>100</code>.<br>The value ranges from <code>1</code> to <code>100</code></td>
<td>The maximum number of entities to return.<br>The sum of this parameter value and <code>offset</code> should be less than <code>1024</code>.<br>The value defaults to <code>100</code>.<br>The value ranges from <code>1</code> to <code>100</code></td>
</tr>
<tr>
<td><code>offset</code></td>
<td>The number of entities to skip in the search results.<br>The sum of this value and that of `limit` should not be greater than <code>1024</code>.<br>The maximum value is <code>1024</code>.</td>
<td>The number of entities to skip in the search results.<br>The sum of this parameter value and <code>limit</code> should not be greater than <code>1024</code>.<br>The maximum value is <code>1024</code>. For example, if you want the 9th and 10th nearest neighbors to the query vector, set <code>limit</code> to <code>2</code> and <code>offset</code> to <code>8</code>.</td>
</tr>
<tr>
<td><code>outputFields</code></td>
Expand Down Expand Up @@ -485,7 +486,7 @@ R<SearchResults> respSearch = milvusClient.search(searchParam);
</tr>
<tr>
<td><code>limit</code></td>
<td>Number of the most similar results to return. The sum of this value and <code>offset</code> in <code>param</code> should be less than 16384.</td>
<td>Number of the results to return. The sum of this value and <code>offset</code> in <code>param</code> should be less than 16384.</td>
</tr>
<tr>
<td><code>expr</code></td>
Expand Down Expand Up @@ -528,11 +529,11 @@ R<SearchResults> respSearch = milvusClient.search(searchParam);
</tr>
<tr>
<td><code>limit</code> (optional)</td>
<td>Number of the most similar results to return. The sum of this value and <code>offset</code> should be less than 16384.</td>
<td>Number of the results to return. The sum of this value and <code>offset</code> should be less than 16384.</td>
</tr>
<tr>
<td><code>offset</code> (optional)</td>
<td>Number of entities to skip. The sum of this value of <code>limit</code> should be less than 16384.</td>
<td>Number of entities to skip. The sum of this value and <code>limit</code> should be less than 16384. For example, if you want the 9th and 10th nearest neighbors to the query vector, set <code>limit</code> to <code>2</code> and <code>offset</code> to <code>8</code>.</td>
</tr>
<tr>
<td><code>filter</code> (optional)</td>
Expand Down Expand Up @@ -596,7 +597,7 @@ R<SearchResults> respSearch = milvusClient.search(searchParam);
</tr>
<tr>
<td><code>topK</code></td>
<td>Number of the most similar results to return. The sum of this value and that of <code>offset</code> in <code>WithOffset</code> of <code>opts</code> should be less than 16384.</td>
<td>Number of the results to return. The sum of this value and that of <code>offset</code> in <code>WithOffset</code> of <code>opts</code> should be less than 16384.</td>
<td>N/A</td>
</tr>
<tr>
Expand Down
8 changes: 7 additions & 1 deletion v2.3.x/site/en/userGuide/upsert_entities.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,9 +138,15 @@ err != nil {
</tbody>
</table>

<div class="alert note">

After upserting entities into a collection that has previously been indexed, you do not need to re-index the collection, as Milvus will automatically create an index for the newly upserted data. For more information, refer to [Can indexes be created after inserting vectors?](product_faq.md#Can-indexes-be-created-after-inserting-vectors)

</div>

## Flush data

When data is upserted into Milvus it is updated and inserted into segments. Segments have to reach a certain size to be sealed and indexed. Unsealed segments will be searched brute force. In order to avoid this with any remainder data, it is best to call `flush()`. The `flush()` call will seal any remaining segments and send them for indexing. It is important to only call this at the end of an upsert session, as calling this too much will cause fragmented data that will need to be cleaned later on.
When data is upserted into Milvus it is updated and inserted into segments. Segments have to reach a certain size to be sealed and indexed. Unsealed segments will be searched brute force. In order to avoid this with any remainder data, it is best to call `flush()`. The `flush()` call will seal any remaining segments and send them for indexing. It is important to only call this method at the end of an upsert session. Calling it too often will cause fragmented data that will need to be cleaned later on.

## Limits

Expand Down

0 comments on commit cc50228

Please sign in to comment.