Skip to content

Commit ba361a0

Browse files
Milvus-doc-botMilvus-doc-bot
authored andcommitted
Release new docs to preview
1 parent 8bcb2c3 commit ba361a0

3 files changed

Lines changed: 437 additions & 10 deletions

File tree

v3.0.x/site/en/userGuide/insert-and-delete/upsert-entities.md

Lines changed: 366 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ To perform a merge, set `partial_update` to `True` in the `upsert` request along
3434

3535
Upon receiving such a request, Milvus performs a query with strong consistency to retrieve the entity, updates the field values based on the data in the request, inserts the modified data, and then deletes the existing entity with the original primary key carried in the request.
3636

37+
For `ARRAY` fields, merge mode supports two operators in Milvus v2.6.17 and later: `ARRAY_APPEND` and `ARRAY_REMOVE`. These operators let you append elements to or remove matching elements from an existing `ARRAY` field, without first querying the entity to retrieve its current value. For details, see [Upsert ARRAY fields in merge mode](upsert-entities.md#Upsert-ARRAY-fields-in-merge-mode).
38+
3739
### Upsert behaviors: special notes
3840

3941
There are several special notes you should consider before using the merge feature. The following cases assume that you have a collection with two scalar fields named `title` and `issue`, along with a primary key `id` and a vector field called `vector`.
@@ -66,6 +68,22 @@ There are several special notes you should consider before using the merge featu
6668

6769
When you upsert the `extras` field of an entity with modified JSON data, note that the JSON field is treated as a whole, and you cannot update individual keys selectively. In other words, the JSON field **DOES NOT** support upsert in **merge** mode.
6870

71+
- **Upsert an** `ARRAY` **field.**
72+
73+
By default, an `ARRAY` field in merge mode follows **REPLACE** semantics: the value carried in the request overwrites the existing array. For finer-grained updates, Milvus v2.6.17 and later also supports two operators:
74+
75+
- `ARRAY_APPEND` appends the elements in the request payload to the existing array.
76+
77+
- `ARRAY_REMOVE` removes every element from the existing array that matches a value in the request payload.
78+
79+
For operator syntax, supported element types, and other constraints, see [Upsert ARRAY fields in merge mode](upsert-entities.md#Upsert-ARRAY-fields-in-merge-mode).
80+
81+
- **Upsert a StructArray field.**
82+
83+
Upserting a StructArray field in an entity overwrites the field value. To do so, you need to provide a list of dictionaries, each of which contains all subfields defined in the struct schema, even when you perform the upsert in merge mode.
84+
85+
For details, refer to [Upsert StructArray field in merge mode](upsert-entities.md#Upsert-StructArray-field-in-merge-mode).
86+
6987
### Limits & Restrictions
7088

7189
Based on the above content, there are several limits and restrictions to follow:
@@ -566,3 +584,351 @@ curl -X POST "http://localhost:19530/v2/vectordb/entities/upsert" \
566584
# }
567585
```
568586

587+
## Upsert ARRAY fields in merge mode | Milvus 2.6.17+
588+
589+
Before Milvus v2.6.17, updating part of an `ARRAY` field required a client-side read-modify-write flow: query the existing array, change it in application code, and upsert the full replacement value. Partial-update operators (`ARRAY_APPEND` and `ARRAY_REMOVE`) let you send only the elements to append or remove, which reduces client-side logic and avoids the extra read before the upsert.
590+
591+
Suppose the entity with primary key `1` already has `tags = ["new", "trial"]`. Before partial-update operators, adding element `"premium"` to an array required upserting the full replacement array:
592+
593+
<div class="multipleCode">
594+
<a href="#python">Python</a>
595+
<a href="#java">Java</a>
596+
<a href="#javascript">NodeJS</a>
597+
<a href="#go">Go</a>
598+
<a href="#bash">cURL</a>
599+
</div>
600+
601+
```python
602+
client.upsert(
603+
collection_name="users",
604+
# highlight-start
605+
data=[{"pk": 1, "tags": ["new", "trial", "premium"]}],
606+
partial_update=True,
607+
# highlight-end
608+
)
609+
```
610+
611+
```java
612+
List<JsonObject> replacementData = Collections.singletonList(
613+
gson.fromJson("{\"pk\": 1, \"tags\": [\"new\", \"trial\", \"premium\"]}", JsonObject.class)
614+
);
615+
616+
client.upsert(UpsertReq.builder()
617+
.collectionName("users")
618+
// highlight-start
619+
.partialUpdate(true)
620+
.data(replacementData)
621+
// highlight-end
622+
.build());
623+
```
624+
625+
```javascript
626+
// nodejs
627+
```
628+
629+
```go
630+
// go
631+
```
632+
633+
```bash
634+
# restful
635+
```
636+
637+
With `ARRAY_APPEND`, send only the element to add:
638+
639+
<div class="multipleCode">
640+
<a href="#python">Python</a>
641+
<a href="#java">Java</a>
642+
<a href="#javascript">NodeJS</a>
643+
<a href="#go">Go</a>
644+
<a href="#bash">cURL</a>
645+
</div>
646+
647+
```python
648+
client.upsert(
649+
collection_name="users",
650+
# highlight-start
651+
data=[{"pk": 1, "tags": ["premium"]}],
652+
field_ops={"tags": FieldOp.array_append()},
653+
# highlight-end
654+
)
655+
```
656+
657+
```java
658+
List<JsonObject> appendData = Collections.singletonList(
659+
gson.fromJson("{\"pk\": 1, \"tags\": [\"premium\"]}", JsonObject.class)
660+
);
661+
662+
UpsertReq.FieldPartialUpdateOp appendTags = UpsertReq.FieldPartialUpdateOp.builder()
663+
.fieldName("tags")
664+
.opType(UpsertReq.FieldPartialUpdateOp.OpType.ARRAY_APPEND)
665+
.build();
666+
667+
client.upsert(UpsertReq.builder()
668+
.collectionName("users")
669+
// highlight-start
670+
.data(appendData)
671+
.fieldOps(Collections.singletonList(appendTags))
672+
// highlight-end
673+
.build());
674+
```
675+
676+
```javascript
677+
// nodejs
678+
```
679+
680+
```go
681+
// go
682+
```
683+
684+
```bash
685+
# restful
686+
```
687+
688+
With `ARRAY_REMOVE`, send only the matching element to remove:
689+
690+
<div class="multipleCode">
691+
<a href="#python">Python</a>
692+
<a href="#java">Java</a>
693+
<a href="#javascript">NodeJS</a>
694+
<a href="#go">Go</a>
695+
<a href="#bash">cURL</a>
696+
</div>
697+
698+
```python
699+
client.upsert(
700+
collection_name="users",
701+
# highlight-start
702+
data=[{"pk": 1, "tags": ["trial"]}],
703+
field_ops={"tags": FieldOp.array_remove()},
704+
# highlight-end
705+
)
706+
```
707+
708+
```java
709+
List<JsonObject> removeData = Collections.singletonList(
710+
gson.fromJson("{\"pk\": 1, \"tags\": [\"trial\"]}", JsonObject.class)
711+
);
712+
713+
UpsertReq.FieldPartialUpdateOp removeTags = UpsertReq.FieldPartialUpdateOp.builder()
714+
.fieldName("tags")
715+
.opType(UpsertReq.FieldPartialUpdateOp.OpType.ARRAY_REMOVE)
716+
.build();
717+
718+
client.upsert(UpsertReq.builder()
719+
.collectionName("users")
720+
// highlight-start
721+
.data(removeData)
722+
.fieldOps(Collections.singletonList(removeTags))
723+
// highlight-end
724+
.build());
725+
```
726+
727+
```javascript
728+
// nodejs
729+
```
730+
731+
```go
732+
// go
733+
```
734+
735+
```bash
736+
# restful
737+
```
738+
739+
<div class="alert note">
740+
741+
Attaching either operator to a field via `field_ops` implicitly enables partial-update semantics. Therefore, you do **not** need to pass `partial_update=True` alongside `field_ops`.
742+
743+
</div>
744+
745+
### Limits
746+
747+
- The payload values must match the `element_type` of the target `ARRAY` field. For example, if the target field is `ARRAY<VARCHAR>`, the payload must contain string values.
748+
749+
- In Milvus v2.6.17 and later, `ARRAY_APPEND` and `ARRAY_REMOVE` support `ARRAY` fields whose `element_type` is `BOOL`, `INT8`, `INT16`, `INT32`, `INT64`, `FLOAT`, `DOUBLE`, or `VARCHAR`.
750+
751+
- After an `ARRAY_APPEND` operation, the resulting array length must not exceed the field's `max_capacity`.
752+
753+
- Concurrent upserts to the same entity are not atomic across requests. If two requests update the same `ARRAY` field at the same time, the later write can overwrite the earlier one. Use application-level coordination if you need to preserve all concurrent changes.
754+
755+
### Example
756+
757+
The following example uses a small `users` collection with a primary key `pk`, a `tags` field of type `ARRAY<VARCHAR>`, and an `embedding` vector field. It first inserts two entities with initial `tags` values, then uses `ARRAY_APPEND` and `ARRAY_REMOVE` to show how each operator changes the stored array.
758+
759+
<div class="multipleCode">
760+
<a href="#python">Python</a>
761+
<a href="#java">Java</a>
762+
<a href="#javascript">NodeJS</a>
763+
<a href="#go">Go</a>
764+
<a href="#bash">cURL</a>
765+
</div>
766+
767+
```python
768+
from pymilvus import DataType, FieldOp, MilvusClient
769+
770+
client = MilvusClient(
771+
uri="http://localhost:19530",
772+
token="root:Milvus"
773+
)
774+
775+
# 1. Create a collection with an ARRAY<VARCHAR> field
776+
schema = client.create_schema(enable_dynamic_field=False)
777+
schema.add_field("pk", DataType.INT64, is_primary=True)
778+
schema.add_field("embedding", DataType.FLOAT_VECTOR, dim=5)
779+
schema.add_field(
780+
"tags",
781+
DataType.ARRAY,
782+
element_type=DataType.VARCHAR,
783+
max_capacity=8,
784+
max_length=32,
785+
)
786+
787+
index_params = client.prepare_index_params()
788+
index_params.add_index(
789+
field_name="embedding",
790+
index_type="AUTOINDEX",
791+
metric_type="L2",
792+
)
793+
794+
client.create_collection(
795+
collection_name="users",
796+
schema=schema,
797+
index_params=index_params
798+
)
799+
800+
# 2. Seed two entities
801+
client.insert(
802+
collection_name="users",
803+
data=[
804+
{"pk": 1, "embedding": [0.1, 0.2, 0.3, 0.4, 0.5], "tags": ["new"]},
805+
{"pk": 2, "embedding": [0.6, 0.7, 0.8, 0.9, 1.0], "tags": ["new", "trial"]},
806+
],
807+
)
808+
809+
# 3. Append tags without reading the existing ARRAY values
810+
client.upsert(
811+
collection_name="users",
812+
# highlight-start
813+
data=[
814+
{"pk": 1, "tags": ["premium", "vip"]},
815+
{"pk": 2, "tags": ["premium"]},
816+
],
817+
field_ops={"tags": FieldOp.array_append()},
818+
# highlight-end
819+
)
820+
821+
res = client.query(
822+
collection_name="users",
823+
filter="pk in [1, 2]",
824+
output_fields=["pk", "tags"],
825+
)
826+
print(res)
827+
828+
# Example output:
829+
# data: [
830+
# "{'pk': 1, 'tags': ['new', 'premium', 'vip']}",
831+
# "{'pk': 2, 'tags': ['new', 'trial', 'premium']}"
832+
# ]
833+
834+
# 4. Remove matching tags without replacing the full ARRAY field
835+
client.upsert(
836+
collection_name="users",
837+
# highlight-start
838+
data=[
839+
{"pk": 1, "tags": ["new"]},
840+
{"pk": 2, "tags": ["trial"]},
841+
],
842+
field_ops={"tags": FieldOp.array_remove()},
843+
# highlight-end
844+
)
845+
846+
res = client.query(
847+
collection_name="users",
848+
filter="pk in [1, 2]",
849+
output_fields=["pk", "tags"],
850+
)
851+
print(res)
852+
853+
# Example output:
854+
# data: [
855+
# "{'pk': 1, 'tags': ['premium', 'vip']}",
856+
# "{'pk': 2, 'tags': ['new', 'premium']}"
857+
# ]
858+
```
859+
860+
```java
861+
// java
862+
```
863+
864+
```javascript
865+
// nodejs
866+
```
867+
868+
```go
869+
// go
870+
```
871+
872+
```bash
873+
# restful
874+
```
875+
876+
## Upsert StructArray field in merge mode
877+
878+
Upserting a StructArray field in an entity overwrites the field value. That means you need to include all subfields defined in the struct schema when you upsert a StructArray field.
879+
880+
The following example demonstrates how to upsert the `chunks` field in merge mode, a StructArray field with 6 subfields. When the operation completes, the `chunks` field of the entity with id 1 is set to the array with the two-element structs provided in the request.
881+
882+
<div class="multipleCode">
883+
<a href="#python">Python</a>
884+
<a href="#java">Java</a>
885+
<a href="#javascript">NodeJS</a>
886+
<a href="#go">Go</a>
887+
<a href="#bash">cURL</a>
888+
</div>
889+
890+
```python
891+
client.upsert(
892+
collection_name="books",
893+
# highlight-start
894+
data=[{
895+
"id": 1,
896+
"chunks": [
897+
{
898+
"text": "Use HNSW efSearch to trade recall for latency.",
899+
"section": "index",
900+
"page": 1,
901+
"quality_score": 0.92,
902+
"has_code": True,
903+
"emb_list_vector": [0.11, 0.21, 0.31, 0.41]
904+
},
905+
{
906+
"text": "Range search returns vectors within a distance boundary.",
907+
"section": "search",
908+
"page": 2,
909+
"quality_score": 0.86,
910+
"has_code": False,
911+
"emb_list_vector": [0.18, 0.23, 0.29, 0.36]
912+
}
913+
]
914+
}],
915+
# highlight-end
916+
partial_update=True
917+
)
918+
```
919+
920+
```java
921+
// java
922+
```
923+
924+
```javascript
925+
// nodejs
926+
```
927+
928+
```go
929+
// go
930+
```
931+
932+
```bash
933+
# restful
934+
```

0 commit comments

Comments
 (0)