You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: v3.0.x/site/en/userGuide/insert-and-delete/upsert-entities.md
+366Lines changed: 366 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -34,6 +34,8 @@ To perform a merge, set `partial_update` to `True` in the `upsert` request along
34
34
35
35
Upon receiving such a request, Milvus performs a query with strong consistency to retrieve the entity, updates the field values based on the data in the request, inserts the modified data, and then deletes the existing entity with the original primary key carried in the request.
36
36
37
+
For `ARRAY` fields, merge mode supports two operators in Milvus v2.6.17 and later: `ARRAY_APPEND` and `ARRAY_REMOVE`. These operators let you append elements to or remove matching elements from an existing `ARRAY` field, without first querying the entity to retrieve its current value. For details, see [Upsert ARRAY fields in merge mode](upsert-entities.md#Upsert-ARRAY-fields-in-merge-mode).
38
+
37
39
### Upsert behaviors: special notes
38
40
39
41
There are several special notes you should consider before using the merge feature. The following cases assume that you have a collection with two scalar fields named `title` and `issue`, along with a primary key `id` and a vector field called `vector`.
@@ -66,6 +68,22 @@ There are several special notes you should consider before using the merge featu
66
68
67
69
When you upsert the `extras` field of an entity with modified JSON data, note that the JSON field is treated as a whole, and you cannot update individual keys selectively. In other words, the JSON field **DOES NOT** support upsert in **merge** mode.
68
70
71
+
-**Upsert an**`ARRAY`**field.**
72
+
73
+
By default, an `ARRAY` field in merge mode follows **REPLACE** semantics: the value carried in the request overwrites the existing array. For finer-grained updates, Milvus v2.6.17 and later also supports two operators:
74
+
75
+
-`ARRAY_APPEND` appends the elements in the request payload to the existing array.
76
+
77
+
-`ARRAY_REMOVE` removes every element from the existing array that matches a value in the request payload.
78
+
79
+
For operator syntax, supported element types, and other constraints, see [Upsert ARRAY fields in merge mode](upsert-entities.md#Upsert-ARRAY-fields-in-merge-mode).
80
+
81
+
-**Upsert a StructArray field.**
82
+
83
+
Upserting a StructArray field in an entity overwrites the field value. To do so, you need to provide a list of dictionaries, each of which contains all subfields defined in the struct schema, even when you perform the upsert in merge mode.
84
+
85
+
For details, refer to [Upsert StructArray field in merge mode](upsert-entities.md#Upsert-StructArray-field-in-merge-mode).
86
+
69
87
### Limits & Restrictions
70
88
71
89
Based on the above content, there are several limits and restrictions to follow:
@@ -566,3 +584,351 @@ curl -X POST "http://localhost:19530/v2/vectordb/entities/upsert" \
566
584
# }
567
585
```
568
586
587
+
## Upsert ARRAY fields in merge mode | Milvus 2.6.17+
588
+
589
+
Before Milvus v2.6.17, updating part of an `ARRAY` field required a client-side read-modify-write flow: query the existing array, change it in application code, and upsert the full replacement value. Partial-update operators (`ARRAY_APPEND` and `ARRAY_REMOVE`) let you send only the elements to append or remove, which reduces client-side logic and avoids the extra read before the upsert.
590
+
591
+
Suppose the entity with primary key `1` already has `tags = ["new", "trial"]`. Before partial-update operators, adding element `"premium"` to an array required upserting the full replacement array:
Attaching either operator to a field via `field_ops` implicitly enables partial-update semantics. Therefore, you do **not** need to pass `partial_update=True` alongside `field_ops`.
742
+
743
+
</div>
744
+
745
+
### Limits
746
+
747
+
- The payload values must match the `element_type` of the target `ARRAY` field. For example, if the target field is `ARRAY<VARCHAR>`, the payload must contain string values.
748
+
749
+
- In Milvus v2.6.17 and later, `ARRAY_APPEND` and `ARRAY_REMOVE` support `ARRAY` fields whose `element_type` is `BOOL`, `INT8`, `INT16`, `INT32`, `INT64`, `FLOAT`, `DOUBLE`, or `VARCHAR`.
750
+
751
+
- After an `ARRAY_APPEND` operation, the resulting array length must not exceed the field's `max_capacity`.
752
+
753
+
- Concurrent upserts to the same entity are not atomic across requests. If two requests update the same `ARRAY` field at the same time, the later write can overwrite the earlier one. Use application-level coordination if you need to preserve all concurrent changes.
754
+
755
+
### Example
756
+
757
+
The following example uses a small `users` collection with a primary key `pk`, a `tags` field of type `ARRAY<VARCHAR>`, and an `embedding` vector field. It first inserts two entities with initial `tags` values, then uses `ARRAY_APPEND` and `ARRAY_REMOVE` to show how each operator changes the stored array.
758
+
759
+
<divclass="multipleCode">
760
+
<a href="#python">Python</a>
761
+
<a href="#java">Java</a>
762
+
<a href="#javascript">NodeJS</a>
763
+
<a href="#go">Go</a>
764
+
<a href="#bash">cURL</a>
765
+
</div>
766
+
767
+
```python
768
+
from pymilvus import DataType, FieldOp, MilvusClient
769
+
770
+
client = MilvusClient(
771
+
uri="http://localhost:19530",
772
+
token="root:Milvus"
773
+
)
774
+
775
+
# 1. Create a collection with an ARRAY<VARCHAR> field
# 4. Remove matching tags without replacing the full ARRAY field
835
+
client.upsert(
836
+
collection_name="users",
837
+
# highlight-start
838
+
data=[
839
+
{"pk": 1, "tags": ["new"]},
840
+
{"pk": 2, "tags": ["trial"]},
841
+
],
842
+
field_ops={"tags": FieldOp.array_remove()},
843
+
# highlight-end
844
+
)
845
+
846
+
res = client.query(
847
+
collection_name="users",
848
+
filter="pk in [1, 2]",
849
+
output_fields=["pk", "tags"],
850
+
)
851
+
print(res)
852
+
853
+
# Example output:
854
+
# data: [
855
+
# "{'pk': 1, 'tags': ['premium', 'vip']}",
856
+
# "{'pk': 2, 'tags': ['new', 'premium']}"
857
+
# ]
858
+
```
859
+
860
+
```java
861
+
// java
862
+
```
863
+
864
+
```javascript
865
+
// nodejs
866
+
```
867
+
868
+
```go
869
+
// go
870
+
```
871
+
872
+
```bash
873
+
# restful
874
+
```
875
+
876
+
## Upsert StructArray field in merge mode
877
+
878
+
Upserting a StructArray field in an entity overwrites the field value. That means you need to include all subfields defined in the struct schema when you upsert a StructArray field.
879
+
880
+
The following example demonstrates how to upsert the `chunks` field in merge mode, a StructArray field with 6 subfields. When the operation completes, the `chunks` field of the entity with id 1 is set to the array with the two-element structs provided in the request.
881
+
882
+
<divclass="multipleCode">
883
+
<a href="#python">Python</a>
884
+
<a href="#java">Java</a>
885
+
<a href="#javascript">NodeJS</a>
886
+
<a href="#go">Go</a>
887
+
<a href="#bash">cURL</a>
888
+
</div>
889
+
890
+
```python
891
+
client.upsert(
892
+
collection_name="books",
893
+
# highlight-start
894
+
data=[{
895
+
"id": 1,
896
+
"chunks": [
897
+
{
898
+
"text": "Use HNSW efSearch to trade recall for latency.",
899
+
"section": "index",
900
+
"page": 1,
901
+
"quality_score": 0.92,
902
+
"has_code": True,
903
+
"emb_list_vector": [0.11, 0.21, 0.31, 0.41]
904
+
},
905
+
{
906
+
"text": "Range search returns vectors within a distance boundary.",
0 commit comments