diff --git a/site/en/adminGuide/data-infra-integration-overview.md b/site/en/adminGuide/data-infra-integration-overview.md new file mode 100644 index 000000000..86a2be0cc --- /dev/null +++ b/site/en/adminGuide/data-infra-integration-overview.md @@ -0,0 +1,15 @@ +--- +id: data-infra-integration-overview.md +title: Data Infrastructure & Integration +summary: Overview of the third-party infrastructure Milvus integrates with — metadata, object storage, and message queues. +--- + +# Data Infrastructure & Integration + +Milvus builds on open data infrastructure for its core dependencies. This chapter covers the components you can plug in and configure: + +- **[Metadata](etcd.md)** — Milvus stores metadata (collection schemas, node status, consumption checkpoints) in etcd. +- **[Object Storage](object-storage.md)** — Milvus stores index files and binary logs in MinIO, AWS S3, or other S3-compatible / cloud object storage. +- **[Message Queue](mqtype-overview.md)** — Milvus uses a write-ahead log (WAL): Woodpecker (default), Pulsar, Kafka, or RocksMQ. + +By default, a new Milvus 3.x deployment runs with **Woodpecker** as the message queue, **etcd** for metadata, and **MinIO** for object storage — no extra messaging infrastructure required. diff --git a/site/en/adminGuide/deploy_pulsar.md b/site/en/adminGuide/deploy_pulsar.md index 4d7dd952e..5016d1399 100644 --- a/site/en/adminGuide/deploy_pulsar.md +++ b/site/en/adminGuide/deploy_pulsar.md @@ -7,7 +7,7 @@ summary: Learn how to configure message storage with Docker Compose or Helm. # Configure Message Storage with Docker Compose or Helm -Milvus uses Pulsar or Kafka for managing logs of recent changes, outputting stream logs, and providing log subscriptions. Pulsar is the default message storage system. This topic introduces how to configure message storage with Docker Compose or Helm. +Milvus uses a message queue for managing logs of recent changes, outputting stream logs, and providing log subscriptions. Starting from Milvus 3.0.x, Woodpecker is the default recommended message queue; this topic introduces how to configure Pulsar or Kafka with Docker Compose or Helm. You can configure Pulsar with [Docker Compose](https://docs.docker.com/get-started/overview/) or on K8s and configure Kafka on K8s. @@ -75,7 +75,7 @@ helm install milvus/milvus -f values.yaml For Milvus clusters on K8s, you can configure Woodpecker in the same command that starts Milvus. Alternatively, you can configure Woodpecker using the values.yml file on the /charts/milvus path in the [milvus-helm](https://github.com/milvus-io/milvus-helm) repository before you start Milvus. -For details on how to configure Milvus using Helm, refer to [Configure Milvus with Helm Charts](configure-helm.md). For details on Woodpecker-related configuration items, refer to [woodpecker-related configurations](use-woodpecker.md). +For details on how to configure Milvus using Helm, refer to [Configure Milvus with Helm Charts](configure-helm.md). For details on Woodpecker-related configuration items, refer to [woodpecker-related configurations](woodpecker.md). | ### Using the YAML file @@ -104,7 +104,7 @@ extraConfigFiles: maxIntervalForLocalStorage: 10ms # Maximum interval between two sync operations local storage backend, default is 10 milliseconds. maxBytes: 256M # Maximum size of write buffer in bytes. maxEntries: 10000 # Maximum entries number of write buffer. - maxFlushRetries: 5 # Maximum size of write buffer in bytes. + maxFlushRetries: 5 # Maximum number of flush retries. retryInterval: 1000ms # Maximum interval between two retries. default is 1000 milliseconds. maxFlushSize: 2M # Maximum size of a fragment in bytes to flush. maxFlushThreads: 32 # Maximum number of threads to flush data @@ -156,7 +156,7 @@ helm install milvus/milvus -f values.yaml ## Configure RocksMQ with Helm -Milvus standalone uses RocksMQ as the default message storage. For detailed steps on how to configure Milvus with Helm, refer to [Configure Milvus with Helm Charts](configure-helm.md). For details on RocksMQ-related configuration items, refer to [RocksMQ-related configurations](configure_rocksmq.md). +RocksMQ was the default message storage in Milvus standalone up to 2.5.x; from 2.6.x the default is Woodpecker. For detailed steps on how to configure Milvus with Helm, refer to [Configure Milvus with Helm Charts](configure-helm.md). For details on RocksMQ-related configuration items, refer to [RocksMQ-related configurations](configure_rocksmq.md). - If you start Milvus with RocksMQ and want to change its settings, you can run `helm upgrade -f ` with the changed settings in the following YAML file. diff --git a/site/en/adminGuide/etcd.md b/site/en/adminGuide/etcd.md new file mode 100644 index 000000000..027eba05a --- /dev/null +++ b/site/en/adminGuide/etcd.md @@ -0,0 +1,24 @@ +--- +id: etcd.md +title: etcd +--- + +# etcd (Metadata) + +Milvus uses **etcd** to store metadata such as collection schemas, node status, and message-consumption checkpoints. + +## Version + +Milvus is validated against **etcd 3.5.x**. When you install Milvus with Helm, the bundled etcd image is `milvusdb/etcd:3.5.25-r1` by default. + +## Change the etcd image with Helm + +To pin or replace the etcd image version, override `etcd.image.tag` (and, if needed, `etcd.image.repository`) when installing or upgrading with Helm: + +```bash +helm install my-release zilliztech/milvus \ + --set etcd.image.repository=milvusdb/etcd \ + --set etcd.image.tag=3.5.25-r1 +``` + +For external etcd, or for detailed Docker Compose / Helm / Milvus Operator configuration, see [Configure Meta Storage with Docker Compose or Helm](deploy_etcd.md) and [Configure Meta Storage with Milvus Operator](meta_storage_operator.md). diff --git a/site/en/adminGuide/message_storage_operator.md b/site/en/adminGuide/message_storage_operator.md index a2b6a43fd..c2542af55 100644 --- a/site/en/adminGuide/message_storage_operator.md +++ b/site/en/adminGuide/message_storage_operator.md @@ -7,7 +7,7 @@ summary: Learn how to configure message storage with Milvus Operator. # Configure Message Storage with Milvus Operator -Milvus uses RocksMQ, Pulsar or Kafka for managing logs of recent changes, outputting stream logs, and providing log subscriptions. This topic introduces how to configure message storage dependencies when you install Milvus with Milvus Operator. For more details, refer to [Configure Message Storage with Milvus Operator](https://github.com/zilliztech/milvus-operator/blob/main/docs/administration/manage-dependencies/message-storage.md) in the Milvus Operator repository. +In Milvus 3.x, Woodpecker is the default message queue (see [Woodpecker](woodpecker.md)). With Milvus Operator, you can also configure RocksMQ, Pulsar, or Kafka for managing logs of recent changes, outputting stream logs, and providing log subscriptions. This topic introduces how to configure message storage dependencies when you install Milvus with Milvus Operator. For more details, refer to [Configure Message Storage with Milvus Operator](https://github.com/zilliztech/milvus-operator/blob/main/docs/administration/manage-dependencies/message-storage.md) in the Milvus Operator repository. This topic assumes that you have deployed Milvus Operator. @@ -31,14 +31,14 @@ The table below shows whether RocksMQ, Pulsar, Kafka, and Woodpecker are support There are also other limitations for specifying the message storage: - Only one message storage for one Milvus instance is supported. However we still have backward compatibility with multiple message storages set for one instance. The priority is as follows: - - standalone mode: RocksMQ (default) > Pulsar > Kafka - - cluster mode: Pulsar (default) > Kafka + - standalone mode: Woodpecker (default) > RocksMQ > Pulsar > Kafka + - cluster mode: Woodpecker (default) > Pulsar > Kafka - The message storage cannot be changed while the Milvus system is running. - Only Kafka 2.x or 3.x verison is supported. - **Upgrade limitations**: {{fragments/mq_upgrade_limitation.md}} ## Configure RocksMQ -RocksMQ is the default message storage in Milvus standalone. +RocksMQ was the default message storage in Milvus standalone up to 2.5.x (superseded by Woodpecker from 2.6.x).
@@ -85,7 +85,7 @@ spec: ## Configure Woodpecker -Woodpecker is a cloud-native Write-Ahead Log (WAL) designed for object storage. It offers high throughput, low operational overhead, and seamless scalability. For more details, see [Use Woodpecker](use-woodpecker.md). +Woodpecker is a cloud-native Write-Ahead Log (WAL) designed for object storage. It offers high throughput, low operational overhead, and seamless scalability. For more details, see [Woodpecker](woodpecker.md). ## Configure Pulsar @@ -185,7 +185,7 @@ kubectl apply -f milvuscluster.yaml ## Configure Kafka -Pulsar is the default message storage in a Milvus cluster. If you want to use Kafka, add the optional field `msgStreamType` to configure Kafka. +Pulsar was the default message storage in a Milvus cluster up to 2.5.x (superseded by Woodpecker from 2.6.x). If you want to use Kafka, add the optional field `msgStreamType` to configure Kafka. `kafka` supports `external` and `inCluster`. diff --git a/site/en/adminGuide/mq_kafka.md b/site/en/adminGuide/mq_kafka.md new file mode 100644 index 000000000..60b090fe2 --- /dev/null +++ b/site/en/adminGuide/mq_kafka.md @@ -0,0 +1,123 @@ +--- +id: mq_kafka.md +title: Kafka +--- + +# Use Kafka as the Milvus Message Queue + +Apache Kafka is one of the message-queue (WAL) backends Milvus supports. In Milvus 3.x, [Woodpecker](woodpecker.md) is the default message queue; Kafka remains fully supported for users who prefer it. Kafka is primarily used with Milvus Distributed (cluster); standalone deployments typically use embedded Woodpecker or [RocksMQ](mq_rocksmq.md). + +## Version compatibility + +- Milvus supports **Kafka 2.x and 3.x** only. +- Kafka is configured for Milvus Distributed (cluster) via Helm or Milvus Operator. + +## Deploy a Milvus cluster with Kafka using Helm + +### Install and configure + +To use an external Kafka service, disable the bundled Pulsar and enable `externalKafka` in a `values.yaml` override, then install Milvus with it: + +```yaml +pulsarv3: + enabled: false +externalKafka: + enabled: true + brokerList: : + securityProtocol: SASL_SSL + sasl: + mechanisms: PLAIN + username: "" + password: "" +``` + +```bash +helm install my-release zilliztech/milvus -f values.yaml +``` + +For SASL/SSL authentication details, see [Connect to Kafka with SASL/SSL](connect_kafka_ssl.md). + +### Uninstall + +```bash +helm uninstall my-release +``` + +## Deploy a Milvus cluster with Kafka using Milvus Operator + +With Milvus Operator, set `spec.dependencies.msgStreamType: "kafka"` and configure Kafka under `spec.dependencies.kafka` (cluster only). `kafka` supports `external` and `inCluster`. + +### External Kafka + +```yaml +apiVersion: milvus.io/v1alpha1 +kind: Milvus +metadata: + name: my-release + labels: + app: milvus +spec: + config: + kafka: + # securityProtocol supports: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL + securityProtocol: PLAINTEXT + # saslMechanisms supports: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512 + saslMechanisms: PLAIN + saslUsername: "" + saslPassword: "" + dependencies: + msgStreamType: "kafka" + kafka: + external: true + brokerList: + - "kafkaBrokerAddr1:9092" + - "kafkaBrokerAddr2:9092" +``` + +
+ +SASL configurations are supported in Milvus Operator v0.8.5 or later. + +
+ +### Internal (in-cluster) Kafka + +```yaml +apiVersion: milvus.io/v1alpha1 +kind: Milvus +metadata: + name: my-release + labels: + app: milvus +spec: + dependencies: + msgStreamType: "kafka" + kafka: + inCluster: + values: {} # see https://artifacthub.io/packages/helm/bitnami/kafka + components: {} + config: {} +``` + +Apply the configuration (assuming the file is `milvuscluster.yaml`): + +```bash +kubectl apply -f milvuscluster.yaml +``` + +### Uninstall + +```bash +kubectl delete milvus my-release +``` + +## Notes + +- **Upgrading from 2.5.x to 2.6.x:** {{fragments/mq_upgrade_limitation.md}} If you run Kafka and want to keep it, do not change the message queue during the upgrade. +- Only **Kafka 2.x and 3.x** are supported. +- For SASL/SSL connectivity, see [Connect to Kafka with SASL/SSL](connect_kafka_ssl.md). + +## What's next + +- [Woodpecker (default message queue)](woodpecker.md) +- [Switch between Kafka and Woodpecker](switch-kafka-woodpecker.md) diff --git a/site/en/adminGuide/mq_pulsar.md b/site/en/adminGuide/mq_pulsar.md new file mode 100644 index 000000000..bf8a1169a --- /dev/null +++ b/site/en/adminGuide/mq_pulsar.md @@ -0,0 +1,153 @@ +--- +id: mq_pulsar.md +title: Pulsar +--- + +# Use Pulsar as the Milvus Message Queue + +Apache Pulsar is one of the message-queue (WAL) backends Milvus supports. In Milvus 3.x, [Woodpecker](woodpecker.md) is the default message queue; Pulsar remains fully supported for users who prefer it. Pulsar is primarily used with Milvus Distributed (cluster); standalone deployments typically use embedded Woodpecker or [RocksMQ](mq_rocksmq.md). + +## Version compatibility + +| Milvus version | Supported Pulsar version | Default | +| --- | --- | --- | +| 2.5.x and later | Pulsar v3 (recommended) or Pulsar v2 | Pulsar v3 (via Helm / Milvus Operator) | +| 2.4.x and earlier | Pulsar v2 | Pulsar v2 | + +Since Milvus 2.5, the Milvus Helm chart and Milvus Operator deploy **Pulsar v3** by default; Pulsar v2 remains compatible. See [Upgrade Pulsar from v2 to v3](upgrade-pulsar-v3.md) and [Continue Using Pulsar v2](use-pulsar-v2.md). + +## Deploy a Milvus cluster with Pulsar using Helm + +### Install + +To deploy a Milvus cluster that uses the bundled Pulsar (instead of Woodpecker), install the Helm chart with the Streaming Node enabled: + +```bash +helm install my-release zilliztech/milvus \ + --set image.all.tag=v{{var.milvus_release_version}} \ + --set pulsarv3.enabled=true \ + --set woodpecker.enabled=false \ + --set streaming.enabled=true \ + --set indexNode.enabled=false +``` + +On Kubernetes v1.25 and later, if you hit PodDisruptionBudget API issues from the bundled Pulsar sub-chart, disable the Pulsar PDB policies: + +```bash +helm install my-release zilliztech/milvus \ + --set pulsar.bookkeeper.pdb.usePolicy=false \ + --set pulsar.broker.pdb.usePolicy=false \ + --set pulsar.proxy.pdb.usePolicy=false \ + --set pulsar.zookeeper.pdb.usePolicy=false +``` + +### Configure + +To connect Milvus to an **external** Pulsar service, disable the bundled Pulsar and enable `externalPulsar` in a `values.yaml` override: + +```yaml +pulsarv3: + enabled: false +externalPulsar: + enabled: true + host: + port: 6650 + maxMessageSize: "5242880" # 5 MB, maximum size of each message + tenant: public + namespace: default +``` + +```bash +helm install my-release zilliztech/milvus -f values.yaml +``` + +### Uninstall + +```bash +helm uninstall my-release +``` + +If you used the bundled Pulsar and want to remove its persisted data, delete the Pulsar PVCs (named `my-release-pulsarv3-*`): + +```bash +kubectl get pvc | grep my-release-pulsarv3 +kubectl delete pvc ... +``` + +## Deploy a Milvus cluster with Pulsar using Milvus Operator + +With Milvus Operator, configure Pulsar under `spec.dependencies.pulsar` (supported for Milvus cluster only). `pulsar` supports `external` and `inCluster`. + +### External Pulsar + +```yaml +apiVersion: milvus.io/v1alpha1 +kind: Milvus +metadata: + name: my-release + labels: + app: milvus +spec: + dependencies: + pulsar: + external: true + endpoints: + - 192.168.1.1:6650 + components: {} + config: {} +``` + +### Internal (in-cluster) Pulsar + +```yaml +apiVersion: milvus.io/v1alpha1 +kind: Milvus +metadata: + name: my-release + labels: + app: milvus +spec: + dependencies: + pulsar: + inCluster: + values: + components: + autorecovery: false + zookeeper: + replicaCount: 1 + bookkeeper: + replicaCount: 1 + broker: + replicaCount: 1 + configData: + autoSkipNonRecoverableData: "true" + managedLedgerDefaultEnsembleSize: "1" + managedLedgerDefaultWriteQuorum: "1" + managedLedgerDefaultAckQuorum: "1" + proxy: + replicaCount: 1 + components: {} + config: {} +``` + +Apply the configuration (assuming the file is `milvuscluster.yaml`): + +```bash +kubectl apply -f milvuscluster.yaml +``` + +### Uninstall + +```bash +kubectl delete milvus my-release +``` + +## Notes + +- **Upgrading from 2.5.x to 2.6.x:** {{fragments/mq_upgrade_limitation.md}} If you run Pulsar and want to keep it, do not change the message queue during the upgrade. +- **Pulsar v2 → v3:** see [Upgrade Pulsar from v2 to v3](upgrade-pulsar-v3.md); to stay on v2, see [Continue Using Pulsar v2](use-pulsar-v2.md). + +## What's next + +- [Woodpecker (default message queue)](woodpecker.md) +- [Switch between Pulsar and Woodpecker](switch-pulsar-woodpecker.md) diff --git a/site/en/adminGuide/mq_rocksmq.md b/site/en/adminGuide/mq_rocksmq.md new file mode 100644 index 000000000..d404ac44a --- /dev/null +++ b/site/en/adminGuide/mq_rocksmq.md @@ -0,0 +1,79 @@ +--- +id: mq_rocksmq.md +title: RocksMQ +--- + +# Use RocksMQ as the Milvus Message Queue + +RocksMQ is an embedded message queue (WAL) bundled with Milvus, available for **Milvus Standalone only**. It was the default standalone message queue in earlier Milvus versions; in Milvus 3.x, Milvus Standalone uses embedded [Woodpecker](woodpecker.md) by default. + +## Version compatibility + +- **Standalone only** — RocksMQ is **not** supported in Milvus Distributed (cluster). See the [message queue support matrix](mqtype-overview.md#Supported-message-queues). +- RocksMQ ships with Milvus, so there is no separate version to install. +- It was the default standalone message queue in earlier Milvus versions, and is superseded by embedded Woodpecker in Milvus 3.x. + +## Deploy Milvus Standalone with RocksMQ using Docker + +### Install + +Follow [Run Milvus in Docker](install_standalone-docker.md). In Milvus 3.x the standalone default is Woodpecker, so switch the message-queue type to RocksMQ explicitly. The bootstrap script writes a fresh `user.yaml` on the **first** `start`, so set the type **after** that first start and then `restart` to apply (a `restart` preserves `user.yaml`): + +```bash +mkdir milvus-rocksmq && cd milvus-rocksmq +curl -sfL https://raw.githubusercontent.com/milvus-io/milvus/master/scripts/standalone_embed.sh -o standalone_embed.sh + +# 1. First start — boots the container and writes a default user.yaml +bash standalone_embed.sh start + +# 2. Set the message queue to RocksMQ +cat > user.yaml <<'EOF' +mq: + type: rocksmq +EOF + +# 3. Restart to apply the change +bash standalone_embed.sh restart +``` + +
+Switching mq.type this way is meant for a brand-new instance (no collections yet). To change the message queue of an instance that already holds data, follow the switch procedure instead. +
+ +### Configure + +To tune RocksMQ, add a `rocksmq` section to `user.yaml` and restart the service: + +```yaml +mq: + type: rocksmq +rocksmq: + path: /var/lib/milvus/rdb_data # where messages are stored + lrucacheratio: 0.06 # rocksdb cache memory ratio + rocksmqPageSize: 67108864 # 64 MB, size of each message page + retentionTimeInMinutes: 4320 # 3 days + retentionSizeInMB: 8192 # 8 GB + compactionInterval: 86400 # 1 day, trigger rocksdb compaction + compressionTypes: [0, 0, 7, 7, 7] +``` + +```bash +bash standalone_embed.sh restart +``` + +### Uninstall + +```bash +bash standalone_embed.sh stop +bash standalone_embed.sh delete +``` + +## Notes + +- **Upgrading from 2.5.x to 2.6.x:** {{fragments/mq_upgrade_limitation.md}} Because 2.6.x changes the standalone default to Woodpecker, pin `mq.type: rocksmq` in your `user.yaml` **before** upgrading if you want to keep RocksMQ. +- To change the message queue of a running instance, see [Switch from RocksMQ to Woodpecker](switch-rocksmq-woodpecker.md). + +## What's next + +- [Woodpecker (default message queue)](woodpecker.md) +- [Switch from RocksMQ to Woodpecker](switch-rocksmq-woodpecker.md) diff --git a/site/en/adminGuide/mqtype-overview.md b/site/en/adminGuide/mqtype-overview.md new file mode 100644 index 000000000..0c6c03e7c --- /dev/null +++ b/site/en/adminGuide/mqtype-overview.md @@ -0,0 +1,32 @@ +--- +id: mqtype-overview.md +title: Message Queue Overview +summary: Overview of the message queue (mqType) options Milvus supports, and which one to use for standalone vs. distributed deployments. +--- + +# Message Queue Overview + +Milvus relies on a message queue (write-ahead log, WAL) to manage logs of recent changes, output stream logs, and provide log subscriptions. In Milvus 3.x, **Woodpecker** is the default message queue and requires no separate messaging infrastructure. Pulsar, Kafka, and RocksMQ remain supported for specific scenarios. + +## Supported message queues + +| Message queue | Milvus Standalone | Milvus Distributed (cluster) | Default in | Notes | +| --- | :---: | :---: | --- | --- | +| [Woodpecker](woodpecker.md) | ✔️ (embedded) | ✔️ (embedded or service) | **Milvus 3.x** (both modes) | Default and recommended. Cloud-native WAL on object storage; no external service required. | +| [Pulsar](mq_pulsar.md) | ✔️ | ✔️ | ≤ 2.5.x (cluster default) | Supported, external or bundled. | +| [Kafka](mq_kafka.md) | ✔️ | ✔️ | — | Supported. Only Kafka 2.x or 3.x. | +| [RocksMQ](mq_rocksmq.md) | ✔️ | ✖️ | ≤ 2.5.x (standalone default) | Supported for **standalone only**. | + +
+ +- Each Milvus instance uses exactly one message queue. +- {{fragments/mq_upgrade_limitation.md}} +- To change the message queue of a running instance, see [Switch MQ Type](switch-mq-type.md) (supported from v2.6.14). + +
+ +## Choosing a message queue + +- **New deployments (Milvus 3.x):** use **Woodpecker** (the default). Standalone runs it embedded; for distributed (cluster), the recommended default is a dedicated [service](woodpecker.md#Deployment-modes) deployed with Helm, and embedded is also supported. +- **Existing Pulsar or Kafka users:** Pulsar and Kafka remain fully supported. Keep them, or [switch to Woodpecker](switch-mq-type.md). +- **RocksMQ:** standalone only, and superseded by embedded Woodpecker in Milvus 3.x. diff --git a/site/en/adminGuide/object-storage.md b/site/en/adminGuide/object-storage.md new file mode 100644 index 000000000..33021d1c0 --- /dev/null +++ b/site/en/adminGuide/object-storage.md @@ -0,0 +1,29 @@ +--- +id: object-storage.md +title: Object Storage +--- + +# Object Storage + +Milvus stores index files and binary logs — the bulk of its data — in object storage. Milvus supports MinIO and a range of S3-compatible and cloud object stores. + +## Supported object storage + +| Provider / service | Supported as Milvus object storage | +| --- | :---: | +| MinIO | ✔️ (default for self-hosted deployments) | +| AWS S3 | ✔️ | +| Azure Blob Storage | ✔️ | +| Google Cloud Storage (GCS) | ✔️ | +| Aliyun OSS | ✔️ | +| Tencent COS | ✔️ | +| Huawei Cloud OBS | ✔️ | +| Other S3-compatible storage | ✔️ | + +For configuration details, see [Configure Object Storage with Docker Compose or Helm](deploy_s3.md) and [Configure Object Storage with Milvus Operator](object_storage_operator.md). + +## Additional requirements when using embedded Woodpecker + +When you run the default **Woodpecker** message queue with its object-storage backend (`storage.type=minio`), Woodpecker writes its write-ahead log to the same object storage and requires **strict S3 Conditional-Write semantics**. Not every object store qualifies — for example, Huawei Cloud OBS is currently **unsupported** as a Woodpecker backend even though it works as regular Milvus object storage. + +See the object storage compatibility matrix on the [Woodpecker](woodpecker.md) page for the exact per-provider requirements. diff --git a/site/en/adminGuide/switch-kafka-woodpecker.md b/site/en/adminGuide/switch-kafka-woodpecker.md new file mode 100644 index 000000000..ca4daabf0 --- /dev/null +++ b/site/en/adminGuide/switch-kafka-woodpecker.md @@ -0,0 +1,145 @@ +--- +id: switch-kafka-woodpecker.md +title: Switch between Kafka and Woodpecker +summary: Switch the message queue of a Milvus cluster between Kafka and Woodpecker, with Helm or Milvus Operator. +--- + +# Switch between Kafka and Woodpecker + +This page describes how to switch the message queue (MQ) of a **Milvus cluster** between **Kafka** (builtin or external) and **Woodpecker** (MinIO backend), in both directions, for Helm and Milvus Operator deployments. For the general workflow and prerequisites, see [Switch MQ Type](switch-mq-type.md). + +## Switch from Kafka to Woodpecker + +### Step 1: Verify the Milvus instance is running + +Before switching, ensure your Milvus cluster is running properly — for example, by creating a test collection, inserting data, and running a query. + +### Step 2: Execute the MQ switch + +```shell +curl -X POST http://:/management/wal/alter \ + -H "Content-Type: application/json" \ + -d '{"target_wal_name": "woodpecker"}' +``` + +
+ +Expose the MixCoord management interface to reach it: + +- **Helm:** use `kubectl port-forward --address 0.0.0.0 service/my-release-milvus-mixcoord 29091:9091`; then `mixcoord_addr` is the machine's IP and `mixcoord_port` is `29091`. +- **Milvus Operator:** the MixCoord service is not exposed — use `kubectl exec` into the MixCoord pod and run the `curl` command from inside the pod. + +
+ +### Step 3: Verify the switch is complete + +```shell +kubectl logs | grep "successfully updated mq.type configuration in etcd" +``` + +A successful switch logs `[mqTypeValue=woodpecker]`. + +### Step 4 (Milvus Operator only): Update the MQ type in the Operator + +Update the MQ type in the Operator-managed configuration. Create `change_configmap.yaml`: + +```yaml +apiVersion: milvus.io/v1beta1 +kind: Milvus +metadata: + name: my-release + labels: + app: milvus +spec: + dependencies: + msgStreamType: woodpecker +``` + +```shell +kubectl patch -f change_configmap.yaml --patch-file change_configmap.yaml --type merge +``` + +### Step 5: (Optional) Stop Kafka and clean up + +**For builtin Kafka:** after confirming the switch, remove the Kafka pods and their PVCs. + +**For external Kafka:** clean up the Milvus topics in the external Kafka instance. Milvus topics follow the format `-dml__`. + +
+ +If you plan to switch back to Kafka later, clean up the data/topics first to avoid conflicts. + +
+ +## Switch from Woodpecker to Kafka + +
+ +This direction is supported with **Helm** only. With **Milvus Operator**, switching from Woodpecker to Pulsar/Kafka is **not supported** — the Operator does not allow pre-configuring the Kafka connection address that the switch requires. + +
+ +### Step 1: Verify the Milvus instance is running + +Ensure your Milvus cluster is running properly. + +### Step 2: Configure the target Kafka connection + +Render the Kafka access configuration into Milvus and enable the target MQ. The `streaming.enabled=true` flag is required for the Switch MQ feature. Create or edit `values.yaml`: + +```yaml +extraConfigFiles: + user.yaml: |+ + kafka: + brokerList: + - : + saslUsername: + saslPassword: + saslMechanisms: PLAIN + securityProtocol: SASL_SSL +``` + +```shell +helm upgrade -i my-release zilliztech/milvus \ + --set kafka.enabled=true \ + --set woodpecker.enabled=false \ + --set streaming.enabled=true \ + -f values.yaml +``` + +Wait for all pods to be ready, then verify that the Kafka access configuration has been rendered into the Milvus configuration. For SASL/SSL details, see [Connect to Kafka with SASL/SSL](connect_kafka_ssl.md). + +### Step 3: Execute the MQ switch + +
+ +Ensure the target Kafka does not contain Milvus topics from a previous configuration. If this is your first switch to Kafka, skip this note; otherwise clean up residual Milvus topics with the same names first. + +
+ +```shell +curl -X POST http://:/management/wal/alter \ + -H "Content-Type: application/json" \ + -d '{"target_wal_name": "kafka"}' +``` + +### Step 4: Verify the switch is complete + +```shell +kubectl logs | grep "successfully updated mq.type configuration in etcd" +``` + +A successful switch logs `[mqTypeValue=kafka]`. + +### Step 5: (Optional) Clean up Woodpecker data + +Delete the Woodpecker data on MinIO/S3 (under `/wp/...`, typically `files/wp/...`) and the Woodpecker metadata in etcd (`etcdctl get woodpecker --prefix`). If you plan to switch back to Woodpecker later, clean up these files first. + +## Supported scenarios + +| Source MQ | Target MQ | Helm | Milvus Operator | +|-----------|-----------|------|-----------------| +| Builtin Kafka | Woodpecker (MinIO) | **Supported** | **Supported** | +| External Kafka | Woodpecker (MinIO) | **Supported** | **Supported** | +| Woodpecker (MinIO) | External Kafka | **Supported** | **Not supported** | +| Kafka | Woodpecker (local) | **Supported but not recommended** (all pods need a shared FS) | **Not supported** | diff --git a/site/en/adminGuide/switch-mq-type.md b/site/en/adminGuide/switch-mq-type.md new file mode 100644 index 000000000..432892678 --- /dev/null +++ b/site/en/adminGuide/switch-mq-type.md @@ -0,0 +1,60 @@ +--- +id: switch-mq-type.md +title: Switch MQ Type +summary: Switch the message queue of an existing Milvus deployment between Woodpecker and another message queue without downtime. +--- + +# Switch MQ Type + +This guide describes how to switch the message queue (MQ) of an existing Milvus deployment **between Woodpecker and another message queue**, online and without downtime. + +
+ +This feature is pending release and is subject to change. Please reach out to Milvus support if you want to try it out or have any questions. + +
+ +## Prerequisites + +- The Milvus instance has been upgraded to a version that supports the Switch MQ feature (**v2.6.14 or later**). +- The instance is running properly. + +## Scope + +This guide covers switching **between Woodpecker and another message queue** only. Switching directly between Pulsar and Kafka is out of scope. + +- [Switch between RocksMQ and Woodpecker](switch-rocksmq-woodpecker.md) — Milvus Standalone (Docker Compose) +- [Switch between Pulsar and Woodpecker](switch-pulsar-woodpecker.md) — Milvus cluster (Helm / Milvus Operator) +- [Switch between Kafka and Woodpecker](switch-kafka-woodpecker.md) — Milvus cluster (Helm / Milvus Operator) + +## General workflow + +1. Ensure the Milvus instance is running properly. +2. Confirm the source MQ type and the target MQ type. +3. Render the target MQ's access settings into the Milvus configuration **without** changing the `mqType` value. +4. Trigger the switch by calling the WAL alter API on MixCoord. +5. Monitor the logs to confirm the switch has completed. + +
+ +Before switching, ensure that the target MQ does not contain topics with the same names as those used by the current Milvus instance. This is especially important if the target MQ has been used by another Milvus instance, as conflicting topic names can lead to unexpected behavior. + +
+ +## Support matrix + +| Source MQ | Target MQ | Deployment | Status | +|-----------|-----------|------------|--------| +| RocksMQ | Woodpecker (local/MinIO) | Standalone (Docker Compose) | **Supported** | +| Woodpecker (local/MinIO) | RocksMQ | Standalone (Docker Compose) | **Supported** | +| Pulsar (builtin/external) | Woodpecker (MinIO) | Cluster (Helm / Operator) | **Supported** | +| Woodpecker (MinIO) | Pulsar (external) | Cluster (Helm) | **Supported** (Operator: **not supported**) | +| Kafka (builtin/external) | Woodpecker (MinIO) | Cluster (Helm / Operator) | **Supported** | +| Woodpecker (MinIO) | Kafka (external) | Cluster (Helm) | **Supported** (Operator: **not supported**) | +| Woodpecker MinIO | Woodpecker local (or vice versa) | any | **Not supported** | + +
+ +Avoid switching MQ types back and forth repeatedly. If you do need to switch, make sure to clean up the related data before each switch — residual data may cause unexpected behavior. + +
diff --git a/site/en/adminGuide/switch-pulsar-woodpecker.md b/site/en/adminGuide/switch-pulsar-woodpecker.md new file mode 100644 index 000000000..bf3f1cd48 --- /dev/null +++ b/site/en/adminGuide/switch-pulsar-woodpecker.md @@ -0,0 +1,157 @@ +--- +id: switch-pulsar-woodpecker.md +title: Switch between Pulsar and Woodpecker +summary: Switch the message queue of a Milvus cluster between Pulsar and Woodpecker, with Helm or Milvus Operator. +--- + +# Switch between Pulsar and Woodpecker + +This page describes how to switch the message queue (MQ) of a **Milvus cluster** between **Pulsar** (builtin or external) and **Woodpecker** (MinIO backend), in both directions, for Helm and Milvus Operator deployments. For the general workflow and prerequisites, see [Switch MQ Type](switch-mq-type.md). + +## Switch from Pulsar to Woodpecker + +### Step 1: Verify the Milvus instance is running + +Before switching, ensure your Milvus cluster is running properly — for example, by creating a test collection, inserting data, and running a query. + +### Step 2: Execute the MQ switch + +```shell +curl -X POST http://:/management/wal/alter \ + -H "Content-Type: application/json" \ + -d '{"target_wal_name": "woodpecker"}' +``` + +
+ +Expose the MixCoord management interface to reach it: + +- **Helm:** use `kubectl port-forward --address 0.0.0.0 service/my-release-milvus-mixcoord 29091:9091`; then `mixcoord_addr` is the machine's IP and `mixcoord_port` is `29091`. +- **Milvus Operator:** the MixCoord service is not exposed — use `kubectl exec` into the MixCoord pod and run the `curl` command from inside the pod. + +
+ +### Step 3: Verify the switch is complete + +```shell +kubectl logs | grep "successfully updated mq.type configuration in etcd" +``` + +A successful switch logs `[mqTypeValue=woodpecker]`. + +### Step 4 (Milvus Operator only): Update the MQ type in the Operator + +Update the MQ type in the Operator-managed configuration. Create `change_configmap.yaml`: + +```yaml +apiVersion: milvus.io/v1beta1 +kind: Milvus +metadata: + name: my-release + labels: + app: milvus +spec: + dependencies: + msgStreamType: woodpecker +``` + +```shell +kubectl patch -f change_configmap.yaml --patch-file change_configmap.yaml --type merge +``` + +### Step 5: (Optional) Stop Pulsar and clean up + +**For builtin Pulsar (Helm):** disable Pulsar and enable Woodpecker, then delete the Pulsar PVCs. + +```shell +helm upgrade my-release zilliztech/milvus \ + --set image.all.tag=v{{var.milvus_release_version}} \ + --set pulsarv3.enabled=false \ + --set woodpecker.enabled=true \ + --set streaming.enabled=true \ + --set indexNode.enabled=false +``` + +```shell +kubectl get pvc | grep my-release-pulsarv3 +kubectl delete pvc ... +``` + +**For builtin Pulsar (Operator):** `helm uninstall my-release-pulsar`, then delete the Pulsar PVCs. + +**For external Pulsar:** clean up the Milvus topics in the external Pulsar instance. Milvus topics follow the format `-dml__` (for example, `by-dev-rootcoord-dml_10_464633776992639586v0`). + +
+ +If you plan to switch back to Pulsar later, clean up the data/topics first to avoid conflicts. Due to Helm chart limitations, switching back to a **builtin** Pulsar instance is currently not possible. + +
+ +## Switch from Woodpecker to Pulsar + +
+ +This direction is supported with **Helm** only. With **Milvus Operator**, switching from Woodpecker to Pulsar/Kafka is **not supported** — the Operator does not allow pre-configuring the Pulsar connection address that the switch requires. + +
+ +### Step 1: Verify the Milvus instance is running + +Ensure your Milvus cluster is running properly. + +### Step 2: Configure the target Pulsar connection + +Render the Pulsar access configuration into Milvus and enable the target MQ. The `streaming.enabled=true` flag is required for the Switch MQ feature. Create or edit `values.yaml`: + +```yaml +extraConfigFiles: + user.yaml: |+ + pulsar: + address: + port: +``` + +```shell +helm upgrade -i my-release zilliztech/milvus \ + --set pulsarv3.enabled=true \ + --set woodpecker.enabled=false \ + --set streaming.enabled=true \ + -f values.yaml +``` + +Wait for all pods to be ready, then verify that the Pulsar access configuration has been rendered into the Milvus configuration. + +### Step 3: Execute the MQ switch + +
+ +Ensure the target Pulsar does not contain Milvus topics from a previous configuration. If this is your first switch to Pulsar, skip this note; otherwise clean up residual Milvus topics with the same names first. + +
+ +```shell +curl -X POST http://:/management/wal/alter \ + -H "Content-Type: application/json" \ + -d '{"target_wal_name": "pulsar"}' +``` + +### Step 4: Verify the switch is complete + +```shell +kubectl logs | grep "successfully updated mq.type configuration in etcd" +``` + +A successful switch logs `[mqTypeValue=pulsar]`. + +### Step 5: (Optional) Clean up Woodpecker data + +Delete the Woodpecker data on MinIO/S3 (under `/wp/...`, typically `files/wp/...`) and the Woodpecker metadata in etcd (`etcdctl get woodpecker --prefix`). If you plan to switch back to Woodpecker later, clean up these files first. + +## Supported scenarios + +| Source MQ | Target MQ | Helm | Milvus Operator | +|-----------|-----------|------|-----------------| +| Builtin Pulsar | Woodpecker (MinIO) | **Supported** | **Supported** | +| External Pulsar | Woodpecker (MinIO) | **Supported** | **Supported** | +| Woodpecker (MinIO) | External Pulsar | **Supported** | **Not supported** | +| Pulsar | Woodpecker (local) | **Supported but not recommended** (all pods need a shared FS) | **Not supported** | diff --git a/site/en/adminGuide/switch-rocksmq-woodpecker.md b/site/en/adminGuide/switch-rocksmq-woodpecker.md new file mode 100644 index 000000000..b8d88cf1f --- /dev/null +++ b/site/en/adminGuide/switch-rocksmq-woodpecker.md @@ -0,0 +1,110 @@ +--- +id: switch-rocksmq-woodpecker.md +title: Switch between RocksMQ and Woodpecker +summary: Switch the message queue of a Milvus Standalone (Docker Compose) deployment between RocksMQ and Woodpecker. +--- + +# Switch between RocksMQ and Woodpecker + +This page describes how to switch the message queue (MQ) of a **Milvus Standalone (Docker Compose)** deployment between **RocksMQ** and **Woodpecker** (local or MinIO backend), in both directions. For the general workflow and prerequisites, see [Switch MQ Type](switch-mq-type.md). + +
+ +MQ switching requires the Docker **Compose** deployment (which enables an etcd config source). The single-container Docker deployment does not support switching. + +
+ +## Switch from RocksMQ to Woodpecker + +### Step 1: Verify the Milvus instance is running + +Verify your Milvus Standalone Docker Compose instance is running properly — for example, by creating a test collection, inserting data, and running a query. + +### Step 2: Configure Woodpecker storage + +Add the Woodpecker settings to the Milvus configuration **without** changing the `mqType` value. Run `docker exec -it milvus-standalone bash` to enter the container, then edit `/milvus/configs/user.yaml`: + +```yaml +woodpecker: + storage: + type: minio # minio or local +``` + +Restart the Milvus instance to apply the configuration: + +```shell +docker compose restart +``` + +### Step 3: Execute the MQ switch + +
+ +If this is your first time switching to Woodpecker, skip this note. Otherwise, clean up residual Woodpecker meta and data before switching again — residual data may cause unexpected behavior. + +
+ +```shell +curl -X POST http://:/management/wal/alter \ + -H "Content-Type: application/json" \ + -d '{"target_wal_name": "woodpecker"}' +``` + +The MixCoord port is typically `9091`. + +### Step 4: Verify the switch is complete + +```shell +docker logs milvus-standalone | grep "successfully updated mq.type configuration in etcd" +``` + +A successful switch logs `[mqTypeValue=woodpecker]`. + +### Step 5: (Optional) Clean up RocksMQ data + +RocksMQ data is in the `volumes/milvus/rdb_data` and `volumes/milvus/rdb_data_meta_kv` directories defined in `docker-compose.yaml`. If you plan to switch back to RocksMQ later, clean up these files first to avoid conflicts. + +## Switch from Woodpecker to RocksMQ + +### Step 1: Verify the Milvus instance is running + +Ensure your Milvus Standalone Docker Compose instance is running properly. + +### Step 2: Execute the MQ switch + +
+ +Ensure the instance has no residual RocksMQ data from a previous run. If this is your first time switching to RocksMQ, skip this note; otherwise clean up the related RocksMQ meta and data first. + +
+ +```shell +curl -X POST http://:/management/wal/alter \ + -H "Content-Type: application/json" \ + -d '{"target_wal_name": "rocksmq"}' +``` + +### Step 3: Verify the switch is complete + +```shell +docker logs milvus-standalone | grep "successfully updated mq.type configuration in etcd" +``` + +A successful switch logs `[mqTypeValue=rocksmq]`. + +### Step 4: (Optional) Clean up Woodpecker data + +- **Metadata (etcd):** the Woodpecker key prefix is typically `woodpecker/...`. View it with `etcdctl get woodpecker --prefix`, then delete it. +- **Storage data:** in **MinIO mode**, delete the log data under `/wp/...` (typically `files/wp/...`) in the bucket; in **local mode**, the data is on local disk at `volumes/milvus/data/wp/...`. + +If you plan to switch back to Woodpecker later, clean up these files first to avoid conflicts. + +## Supported scenarios + +| Source MQ | Target MQ | Status | Notes | +|-----------|-----------|--------|-------| +| RocksMQ | Woodpecker (MinIO/local) | **Supported** | | +| Woodpecker (MinIO/local) | RocksMQ | **Supported** | | +| Woodpecker MinIO | Woodpecker local | **Not supported** | Switching between Woodpecker storage modes requires additional metadata handling, which is not yet supported. | +| Woodpecker local | Woodpecker MinIO | **Not supported** | Same as above. | +| RocksMQ / Woodpecker | External Pulsar / Kafka | **Supported but not recommended** | Keep standalone instances as simple as possible. | diff --git a/site/en/adminGuide/switch_milvus_cluster_mq_type-operator.md b/site/en/adminGuide/switch_milvus_cluster_mq_type-operator.md deleted file mode 100644 index 6b522aa19..000000000 --- a/site/en/adminGuide/switch_milvus_cluster_mq_type-operator.md +++ /dev/null @@ -1,161 +0,0 @@ ---- -id: switch_milvus_cluster_mq_type-operator.md -summary: Learn how to switch the message queue type for a Milvus cluster. -title: Switch MQ Type for Milvus Cluster ---- - -# Switch MQ Type for Milvus Cluster - -This topic describes how to switch the message queue (MQ) type for an existing Milvus cluster deployment. Milvus supports online MQ switching between Pulsar, Kafka, and Woodpecker without downtime. - -
- -This feature is pending release and is subject to change. Please reach out to Milvus support if you want to try it out or have any questions. - -
- -## Prerequisites - -- A running Milvus cluster instance installed via [Milvus Operator](install_cluster-milvusoperator.md) or [Helm](install_cluster-helm.md). -- The Milvus instance has been upgraded to the latest version that supports this Switch MQ feature. - -## Switch from Pulsar/Kafka to Woodpecker (MinIO) - -Follow these steps to switch the MQ type from Pulsar or Kafka to Woodpecker with MinIO storage. - -### Step 1: Verify the Milvus instance is running - -Before switching, ensure that your Milvus cluster instance is running properly. You can verify this by creating a test collection, inserting data, and running a query. - -### Step 2: (Optional) Verify Woodpecker configuration - -The default Milvus configuration already sets Woodpecker storage type to MinIO, so no additional configuration is required in most cases. - -However, if you have previously customized the Woodpecker configuration, you must ensure that `woodpecker.storage.type` is set to `minio`. Update the Milvus configuration **without** changing the `mqType` value: - -```yaml -woodpecker: - storage: - type: minio -``` - -- For **Helm**, refer to [Configure Milvus with Helm Charts](configure-helm.md) for instructions on updating configuration. -- For **Milvus Operator**, refer to [Configure Milvus with Milvus Operator](configure_operator.md) for instructions on updating configuration. - -### Step 3: Execute the MQ switch - -Run the following command to trigger the switch to Woodpecker: - -```shell -curl -X POST http://:9091/management/wal/alter \ - -H "Content-Type: application/json" \ - -d '{"target_wal_name": "woodpecker"}' -``` - -
- -Replace `` with the actual address of your MixCoord service. - -
- -### Step 4: Verify the switch is complete - -The switch process completes automatically. Monitor the Milvus logs for the following key messages to confirm the switch has finished: - -``` -WAL switch success: switch to finish, re-opening required -AlterWAL broadcast message acknowledged by all vchannels -successfully updated mq.type configuration in etcd -``` - -
- -In the log messages above, `` is the source MQ type (e.g., `pulsar` or `kafka`), and `` is the target MQ type (`woodpecker`). - -- The first message indicates that the WAL switch from the source to the target has completed. -- The second message indicates that all physical channels have been switched. -- The third message indicates that the `mq.type` configuration has been updated in etcd. - -
- -## Switch from Woodpecker (MinIO) to Pulsar or Kafka - -Follow these steps to switch the MQ type from Woodpecker back to Pulsar or Kafka. - -### Step 1: Verify the Milvus instance is running - -Before switching, ensure that your Milvus cluster instance is running properly. - -### Step 2: Configure the target MQ - -Before triggering the switch, you need to ensure the target MQ service (Pulsar or Kafka) is available and its access configuration is rendered into the Milvus configuration. - -
- -The exact steps in this section depend on whether you are using an internal (bundled) or external MQ service. - -
- -#### Option A: Internal Pulsar/Kafka (bundled with Helm) - -If you are using the bundled Pulsar or Kafka deployed by Helm, update your Helm release to enable the target MQ service and disable Woodpecker. The `streaming.enabled=true` flag is required to enable the Streaming Node, which is a prerequisite for the Switch MQ feature. For example, to switch to Pulsar: - -```shell -helm upgrade -i my-release milvus/milvus \ - --set pulsarv3.enabled=true \ - --set woodpecker.enabled=false \ - --set streaming.enabled=true \ - -f values.yaml -``` - -After the upgrade, verify that the target MQ access configuration has been rendered into the Milvus configuration. For example, for Pulsar: - -```yaml -pulsar: - address: - port: 6650 -``` - -#### Option B: Internal Pulsar/Kafka (managed by Milvus Operator) - -If you are using Milvus Operator, update the Milvus custom resource to include the target MQ access configuration. Refer to [Configure Milvus with Milvus Operator](configure_operator.md) for details on updating Milvus configuration. - -#### Option C: External Pulsar/Kafka - -If you are using an external Pulsar or Kafka service, you do not need to change the `mqType`. Simply add the external MQ access configuration to your `values.yaml` and restart the Milvus instance to render the configuration. - -### Step 3: Execute the MQ switch - -Run the following command to trigger the switch to Pulsar (replace `pulsar` with `kafka` if switching to Kafka): - -```shell -curl -X POST http://:9091/management/wal/alter \ - -H "Content-Type: application/json" \ - -d '{"target_wal_name": "pulsar"}' -``` - -
- -Replace `` with the actual address of your MixCoord service. - -
- -### Step 4: Verify the switch is complete - -The switch process completes automatically. Monitor the Milvus logs for the following key messages to confirm the switch has finished: - -``` -WAL switch success: switch to finish, re-opening required -AlterWAL broadcast message acknowledged by all vchannels -successfully updated mq.type configuration in etcd -``` - -
- -In the log messages above, `` is the source MQ type (`woodpecker`), and `` is the target MQ type (e.g., `pulsar` or `kafka`). - -- The first message indicates that the WAL switch from the source to the target has completed. -- The second message indicates that all physical channels have been switched. -- The third message indicates that the `mq.type` configuration has been updated in etcd. - -
diff --git a/site/en/adminGuide/switch_milvus_standalone_mq_type-operator.md b/site/en/adminGuide/switch_milvus_standalone_mq_type-operator.md deleted file mode 100644 index 8b6dcbeef..000000000 --- a/site/en/adminGuide/switch_milvus_standalone_mq_type-operator.md +++ /dev/null @@ -1,165 +0,0 @@ ---- -id: switch_milvus_standalone_mq_type-operator.md -summary: Learn how to switch the message queue type for Milvus standalone. -title: Switch MQ Type for Milvus Standalone ---- - -# Switch MQ Type for Milvus Standalone - -This topic describes how to switch the message queue (MQ) type for an existing Milvus standalone deployment. Milvus supports online MQ switching without downtime. - -
- -This feature is pending release and is subject to change. Please reach out to Milvus support if you want to try it out or have any questions. - -
- -## Prerequisites - -- A running Milvus Standalone instance installed via [Docker](install_standalone-docker.md) or [Docker Compose](install_standalone-docker-compose.md). -- The Milvus instance has been upgraded to the latest version that supports this Switch MQ feature. - -## General workflow - -The general workflow for switching the MQ type is as follows: - -1. Ensure the Milvus instance is running properly. -2. Confirm the source MQ type and the target MQ type. -3. Configure the target MQ's access settings into the Milvus configuration without changing the `mqType` value. -4. Trigger the switch by calling the WAL alter API. -5. Monitor the logs to verify the switch has completed successfully. - -
- -Before switching, ensure that the target MQ does not contain topics with the same names as those used by the current Milvus instance. This is especially important if the target MQ service has been previously used by another Milvus instance, as conflicting topic names can lead to unexpected behavior. - -
- -## Switch from RocksMQ to Woodpecker (Local Storage) - -This procedure applies to **Milvus Standalone Docker** deployments that use RocksMQ by default. - -### Step 1: Verify the Milvus instance is running - -Ensure your Milvus Standalone Docker instance is running properly. You can verify this by creating a test collection, inserting data, and running a query. - -### Step 2: Configure Woodpecker with local storage - -Update the Milvus configuration to add Woodpecker settings **without** changing the `mqType` value. Create or update the `user.yaml` file with the following content: - -```yaml -woodpecker: - storage: - type: local -``` - -Then restart the Milvus instance to apply the configuration: - -```shell -bash standalone_embed.sh restart -``` - -### Step 3: Execute the MQ switch - -Run the following command to trigger the switch to Woodpecker: - -```shell -curl -X POST http://:9091/management/wal/alter \ - -H "Content-Type: application/json" \ - -d '{"target_wal_name": "woodpecker"}' -``` - -
- -Replace `` with the actual address of your MixCoord service (by default, `localhost` for standalone deployments). - -
- -### Step 4: Verify the switch is complete - -The switch process completes automatically. Monitor the Milvus logs for the following key messages to confirm the switch has finished: - -``` -WAL switch success: switch to finish, re-opening required -AlterWAL broadcast message acknowledged by all vchannels -successfully updated mq.type configuration in etcd -``` - -
- -In the log messages above, `` is the source MQ type (`rocksmq`), and `` is the target MQ type (`woodpecker`). - -- The first message indicates that the WAL switch from the source to the target has completed. -- The second message indicates that all physical channels have been switched. -- The third message indicates that the `mq.type` configuration has been updated in etcd. - -
- -## Switch from RocksMQ to Woodpecker (MinIO Storage) - -This procedure applies to **Milvus Standalone Docker Compose** deployments. - -
- -Starting from Milvus v2.6, the default `docker-compose.yaml` already declares `mqType` as Woodpecker. Unless you have modified the default configuration or upgraded from v2.5, this procedure may not be necessary. - -
- -### Step 1: Verify the Milvus instance is running - -Ensure your Milvus Standalone Docker Compose instance is running properly. - -### Step 2: (Optional) Verify Woodpecker configuration - -The default Milvus configuration already sets Woodpecker storage type to MinIO, so no additional configuration is required in most cases. - -However, if you have previously customized the Woodpecker configuration, you must ensure that `woodpecker.storage.type` is set to `minio`. Create or update the `user.yaml` file with the following content: - -```yaml -woodpecker: - storage: - type: minio -``` - -Then restart the Milvus instance to apply the configuration: - -```shell -docker compose down -docker compose up -d -``` - -### Step 3: Execute the MQ switch - -Run the following command to trigger the switch to Woodpecker: - -```shell -curl -X POST http://:9091/management/wal/alter \ - -H "Content-Type: application/json" \ - -d '{"target_wal_name": "woodpecker"}' -``` - -
- -Replace `` with the actual address of your MixCoord service (by default, `localhost` for standalone deployments). - -
- -### Step 4: Verify the switch is complete - -The switch process completes automatically. Monitor the Milvus logs for the following key messages to confirm the switch has finished: - -``` -WAL switch success: switch to finish, re-opening required -AlterWAL broadcast message acknowledged by all vchannels -successfully updated mq.type configuration in etcd -``` - -
- -In the log messages above, `` is the source MQ type (`rocksmq`), and `` is the target MQ type (`woodpecker`). - -- The first message indicates that the WAL switch from the source to the target has completed. -- The second message indicates that all physical channels have been switched. -- The third message indicates that the `mq.type` configuration has been updated in etcd. - -
diff --git a/site/en/adminGuide/use-woodpecker.md b/site/en/adminGuide/woodpecker.md similarity index 74% rename from site/en/adminGuide/use-woodpecker.md rename to site/en/adminGuide/woodpecker.md index 49bb74bda..3386eb96b 100644 --- a/site/en/adminGuide/use-woodpecker.md +++ b/site/en/adminGuide/woodpecker.md @@ -1,20 +1,19 @@ --- -id: use-woodpecker.md -title: Use Woodpecker +id: woodpecker.md +title: Woodpecker related_key: Woodpecker -summary: Learn how to enable woodpecker as the WAL in milvus. -beta: Milvus 2.6.x +summary: Learn how Woodpecker works as the default message queue (WAL) in Milvus, and how to run it in embedded or service mode. --- -# Use Woodpecker +# Woodpecker -This guide explains how to enable and use Woodpecker as the Write-Ahead Log (WAL) in Milvus 2.6.x. Woodpecker is a cloud‑native WAL designed for object storage, offering high throughput, low operational overhead, and seamless scalability. For architecture and benchmark details, see [Woodpecker](woodpecker_architecture.md). +Woodpecker is the **default message queue (write-ahead log, WAL)** in Milvus 3.x. It is a cloud‑native WAL designed for object storage, offering high throughput, low operational overhead, and seamless scalability. For architecture and benchmark details, see [Woodpecker](woodpecker_architecture.md). ## Overview -- Starting from Milvus 2.6, Woodpecker is an optional WAL that provides ordered writes and recovery as the logging service. -- As a message queue choice, it behaves similarly to Pulsar/Kafka and can be enabled via configuration. -- Two storage backends are supported: local file system (`local`) and object storage (`minio`/S3-compatible). +- In Milvus 3.x, Woodpecker is the **default** WAL/message queue, providing ordered writes and recovery as the logging service. No external message-queue service (such as Pulsar or Kafka) is required. +- Woodpecker can run **embedded** in the Milvus/streaming node (default), or as a **dedicated service** with its own pods (distributed/cluster only). +- It supports three `storage.type` modes: object storage (`minio`, the default), local file system (`local`), and the dedicated `service`. See [Deployment modes](#Deployment-modes). ## Quick start @@ -53,7 +52,7 @@ woodpecker: maxIntervalForLocalStorage: 10ms # Maximum interval between two sync operations local storage backend, default is 10 milliseconds. maxBytes: 256M # Maximum size of write buffer in bytes. maxEntries: 10000 # Maximum entries number of write buffer. - maxFlushRetries: 5 # Maximum size of write buffer in bytes. + maxFlushRetries: 5 # Maximum number of flush retries. retryInterval: 1000ms # Maximum interval between two retries. default is 1000 milliseconds. maxFlushSize: 2M # Maximum size of a fragment in bytes to flush. maxFlushThreads: 32 # Maximum number of threads to flush data @@ -84,17 +83,19 @@ Key notes: ## Deployment modes -Milvus supports both Standalone and Cluster modes. Woodpecker storage backend support matrix: +Woodpecker supports three `storage.type` modes: -| | `storage.type=local` | `storage.type=minio` | -| ----------------- | ------------------------ | ------------------------ | -| Milvus Standalone | Supported | Supported | -| Milvus Cluster | Limited (needs shared FS) | Supported | +| `storage.type` | How Woodpecker runs | WAL backend | Milvus Standalone | Milvus Distributed (cluster) | +| -------------- | ------------------- | ----------- | ----------------- | ---------------------------- | +| `minio` (default) | Embedded in the Milvus/streaming node | Object storage (MinIO/S3‑compatible) | Supported | Supported | +| `local` | Embedded in the Milvus/streaming node | Local file system | Supported | Limited (all nodes need a shared FS, e.g. NFS) | +| `service` | **Dedicated Woodpecker service** (its own pods) | Object storage (MinIO/S3‑compatible) | **Not supported** | Supported | Notes: - With `minio`, Woodpecker shares the same object storage with Milvus (MinIO/S3/GCS/OSS, etc.). - With `local`, a single‑node local disk is only suitable for Standalone. If all pods can access a shared file system (e.g., NFS), Cluster mode can also use `local`. +- **`service` mode runs Woodpecker as a separate, independently scalable service and is only available for distributed/cluster deployments.** Standalone deployments use the embedded modes (`minio` or `local`). ## Object storage compatibility for `storage.type=minio` @@ -156,7 +157,7 @@ Run the following command to uninstall the Milvus cluster. kubectl delete milvus my-release ``` -If you need to adjust Woodpecker parameters, follow the settings described in [message storage config](deploy_pulsar.md). +If you need to adjust Woodpecker parameters, follow the settings described in [Configuration](#Configuration). ### Enable Woodpecker for a Milvus Cluster on Kubernetes (Helm Chart, storage=minio) @@ -168,7 +169,7 @@ Then deploy with one of the following examples: ```bash helm install my-release zilliztech/milvus \ - --set image.all.tag=v2.6.0 \ + --set image.all.tag=v{{var.milvus_release_version}} \ --set pulsarv3.enabled=false \ --set woodpecker.enabled=true \ --set streaming.enabled=true \ @@ -179,7 +180,7 @@ helm install my-release zilliztech/milvus \ ```bash helm install my-release zilliztech/milvus \ - --set image.all.tag=v2.6.0 \ + --set image.all.tag=v{{var.milvus_release_version}} \ --set cluster.enabled=false \ --set pulsarv3.enabled=false \ --set standalone.messageQueue=woodpecker \ @@ -187,30 +188,27 @@ helm install my-release zilliztech/milvus \ --set streaming.enabled=true ``` -After deployment, follow the docs to port‑forward and connect. To adjust Woodpecker parameters, follow the settings described in [message storage config](deploy_pulsar.md). +After deployment, follow the docs to port‑forward and connect. To adjust Woodpecker parameters, follow the settings described in [Configuration](#Configuration). ### Enable Woodpecker for Milvus Standalone in Docker (storage=local) -Follow [Run Milvus in Docker](install_standalone-docker.md). Example: +In Milvus 3.x, the Docker standalone deployment uses Woodpecker with the **local filesystem** as its WAL backend **by default** — no extra configuration is required. Follow [Run Milvus in Docker](install_standalone-docker.md): ```bash mkdir milvus-wp && cd milvus-wp curl -sfL https://raw.githubusercontent.com/milvus-io/milvus/master/scripts/standalone_embed.sh -o standalone_embed.sh - -# Create user.yaml to enable Woodpecker with local filesystem -cat > user.yaml <<'EOF' -mq: - type: woodpecker -woodpecker: - storage: - type: local - rootPath: /var/lib/milvus/woodpecker -EOF - bash standalone_embed.sh start ``` -To further change Woodpecker settings, update `user.yaml` and run `bash standalone_embed.sh restart`. +To tune Woodpecker, edit the generated `user.yaml` after the first start and run `bash standalone_embed.sh restart` to apply the changes (a fresh `start` regenerates `user.yaml`, so apply edits with `restart`): + +```yaml +# user.yaml +woodpecker: + logstore: + segmentSyncPolicy: + maxFlushThreads: 16 +``` ### Enable Woodpecker for Milvus Standalone with Docker Compose (storage=minio) @@ -218,7 +216,7 @@ Follow [Run Milvus with Docker Compose](install_standalone-docker-compose.md). E ```bash mkdir milvus-wp-compose && cd milvus-wp-compose -wget https://github.com/milvus-io/milvus/releases/download/v2.6.0/milvus-standalone-docker-compose.yml -O docker-compose.yml +wget https://github.com/milvus-io/milvus/releases/download/v{{var.milvus_release_version}}/milvus-standalone-docker-compose.yml -O docker-compose.yml # By default, the Docker Compose standalone uses Woodpecker sudo docker compose up -d # If you need to change Woodpecker parameters further, write an override: @@ -237,6 +235,35 @@ EOF' docker restart milvus-standalone ``` +### Enable Woodpecker service mode for a Milvus Cluster (Helm) + +For distributed/cluster deployments, you can run Woodpecker as a **dedicated service** (separate pods) instead of embedded in the streaming node by setting `streaming.woodpecker.embedded=false`: + +```bash +helm install my-release zilliztech/milvus \ + --set image.all.tag=v{{var.milvus_release_version}} \ + --set pulsarv3.enabled=false \ + --set woodpecker.enabled=true \ + --set streaming.enabled=true \ + --set streaming.woodpecker.embedded=false \ + --set indexNode.enabled=false +``` + +This deploys Woodpecker as a dedicated StatefulSet (`my-release-milvus-woodpecker`, 4 replicas by default) fronted by a headless service, gossip-clustered on ports `18080` (service), `17946` (gossip), and `9091` (metrics), with MinIO as its storage backend. The service needs a quorum of **3** nodes; the default of **4** replicas keeps the quorum while tolerating a single node failure, so do not set `woodpecker.replicaCount` below 3. The cluster then includes a separate `woodpecker` pod set: + +``` +my-release-milvus-woodpecker-0 +my-release-milvus-woodpecker-1 +my-release-milvus-woodpecker-2 +my-release-milvus-woodpecker-3 +``` + +
+ +Woodpecker `service` mode is for **distributed/cluster** deployments only — standalone deployments run Woodpecker embedded (`minio` or `local`). Milvus Operator does not yet support Woodpecker service mode. + +
+ ## Throughput tuning tips Based on the benchmarks and backend limits in [Woodpecker](woodpecker_architecture.md), optimize end‑to‑end write throughput from the following aspects: @@ -256,10 +283,11 @@ Batch Insert Demo ```python from pymilvus import MilvusClient import random +import time # 1. Set up a Milvus client client = MilvusClient( - uri="http://:27017", + uri="http://:19530", ) # 2. Create a collection diff --git a/site/en/getstarted/install-overview.md b/site/en/getstarted/install-overview.md index 3ab7e6bd9..3d575821d 100644 --- a/site/en/getstarted/install-overview.md +++ b/site/en/getstarted/install-overview.md @@ -16,11 +16,11 @@ To integrate Milvus Lite into your applications, run `pip install pymilvus` to i ## Milvus Standalone -Milvus Standalone is a single-machine server deployment. All components of Milvus Standalone are packed into a single [Docker image](https://milvus.io/docs/install_standalone-docker.md), making deployment convenient. If you have a production workload but prefer not to use Kubernetes, running Milvus Standalone on a single machine with sufficient memory is a good option. +Milvus Standalone is a single-machine server deployment. All components of Milvus Standalone are packed into a single [Docker image](https://milvus.io/docs/install_standalone-docker.md), making deployment convenient. If you have a production workload but prefer not to use Kubernetes, running Milvus Standalone on a single machine with sufficient memory is a good option. By default, Milvus Standalone runs **Woodpecker** (embedded) as its message queue, so there's no separate messaging service to operate. ## Milvus Distributed -Milvus Distributed can be deployed on [Kubernetes](https://milvus.io/docs/install_cluster-milvusoperator.md) clusters. This deployment features a cloud-native architecture, where ingestion load and search queries are separately handled by isolated nodes, allowing redundancy for critical components. It offers the highest scalability and availability, as well as the flexibility in customizing the allocated resources in each component. Milvus Distributed is the top choice for enterprise users running large-scale vector search systems in production. +Milvus Distributed can be deployed on [Kubernetes](https://milvus.io/docs/install_cluster-milvusoperator.md) clusters. This deployment features a cloud-native architecture, where ingestion load and search queries are separately handled by isolated nodes, allowing redundancy for critical components. It offers the highest scalability and availability, as well as the flexibility in customizing the allocated resources in each component. Milvus Distributed is the top choice for enterprise users running large-scale vector search systems in production. By default, Milvus Distributed runs **Woodpecker** as its message queue, deployed as a dedicated service alongside Milvus. ## Choose the Right Deployment for Your Use Case @@ -60,4 +60,5 @@ The choice of Milvus deployment mode depends on your project's stage and scale. | CRUD operations | ✔️ | ✔️ | ✔️ | | Advanced data management | N/A | Access Control
Partition
Partition Key | Access Control
Partition
Partition Key
Physical Resource Grouping | | Consistency Levels | Strong | Strong
Bounded Staleness
Session
Eventual | Strong
Bounded Staleness
Session
Eventual | +| Message queue | N/A | Woodpecker (embedded) | Woodpecker (service) | diff --git a/site/en/getstarted/run-milvus-docker/install_standalone-binary.md b/site/en/getstarted/run-milvus-docker/install_standalone-binary.md index 28849cf43..ac8fe3bb3 100644 --- a/site/en/getstarted/run-milvus-docker/install_standalone-binary.md +++ b/site/en/getstarted/run-milvus-docker/install_standalone-binary.md @@ -74,6 +74,12 @@ If Milvus is running successfully, you should see the following output: You can find the Milvus binary at `/usr/bin/milvus`, the systemd service file at `/lib/systemd/system/milvus.service`, and the dependencies at `/usr/lib/milvus/`. +
+ +By default, Milvus Standalone runs **Woodpecker** (local filesystem) as its message queue with embedded etcd, so no external messaging or metadata service is required. See [Woodpecker](woodpecker.md). + +
+ ## (Optional) Update Milvus configurations You can modify the Milvus configurations in the `/etc/milvus/configs/milvus.yaml` file. For example, to change the `proxy.healthCheckTimeout` to `1000` ms, you can search for the target parameter and modify accordingly. For applicable configuration items, refer to [System Configuration](system_configuration.md). diff --git a/site/en/getstarted/run-milvus-docker/install_standalone-docker-compose.md b/site/en/getstarted/run-milvus-docker/install_standalone-docker-compose.md index 700f97c3f..e70e0847c 100644 --- a/site/en/getstarted/run-milvus-docker/install_standalone-docker-compose.md +++ b/site/en/getstarted/run-milvus-docker/install_standalone-docker-compose.md @@ -33,10 +33,11 @@ Creating milvus-standalone ... done
-**What's new in v{{var.milvus_release_version}}:** -- **Enhanced Architecture**: Features the new Streaming Node and optimized components -- **Updated Dependencies**: Includes the latest MinIO and etcd versions -- **Improved Configuration**: Optimized settings for better performance +**Default deployment (v{{var.milvus_release_version}}):** `docker compose up -d` starts three containers — `milvus-etcd` (metadata), `milvus-minio` (object storage), and `milvus-standalone`. The message queue is **Woodpecker (embedded, with MinIO / object storage as its WAL backend)**, so no separate message-queue container is required. + +**Message-queue default by version:** +- **2.5.x** — default message queue is **RocksMQ**. +- **2.6.x and later** — default message queue is **Woodpecker (embedded)**. Always download the latest Docker Compose configuration to ensure compatibility with v{{var.milvus_release_version}} features. @@ -106,6 +107,28 @@ $ sudo docker compose down $ sudo rm -rf volumes ``` +## Upgrading from Milvus 2.5.x to 2.6.x + +{{fragments/mq_upgrade_limitation.md}} + +Because 2.6.x changes the default message queue to Woodpecker, an instance running **RocksMQ** on 2.5.x must **explicitly pin RocksMQ before upgrading** — otherwise the upgrade would attempt to change the message queue, which is not supported. After downloading the 2.6.x Docker Compose file, set the message-queue type back to `rocksmq` in your `user.yaml` override, then upgrade: + +```yaml +# user.yaml — keep RocksMQ across the 2.5.x → 2.6.x upgrade +mq: + type: rocksmq +``` + +To switch the message queue *after* upgrading, see [Switch MQ Type](switch-mq-type.md). + +## Optional dependencies + +This deployment runs **Woodpecker** (local-filesystem WAL) for messaging, **etcd** for metadata, and **MinIO** for object storage. To use a different message queue or connect external object storage / metadata, see: + +- Message queue: [Woodpecker](woodpecker.md) (default) · [Pulsar](mq_pulsar.md) · [Kafka](mq_kafka.md) · [RocksMQ](mq_rocksmq.md) +- Object storage: [MinIO](deploy_s3.md) (default) · [AWS S3](deploy_s3.md) · [Azure Blob](abs.md) · [GCP Cloud Storage](gcs.md) · [Aliyun OSS](deploy_s3.md) · [Tencent COS](deploy_s3.md) · [Huawei OBS](deploy_s3.md) · [S3-compatible](deploy_s3.md) +- Metadata: [etcd](deploy_etcd.md) + ## What's next Having installed Milvus in Docker, you can: diff --git a/site/en/getstarted/run-milvus-docker/install_standalone-docker.md b/site/en/getstarted/run-milvus-docker/install_standalone-docker.md index 81944c33c..65b14e7a5 100644 --- a/site/en/getstarted/run-milvus-docker/install_standalone-docker.md +++ b/site/en/getstarted/run-milvus-docker/install_standalone-docker.md @@ -33,7 +33,7 @@ $ bash standalone_embed.sh start **What's new in v{{var.milvus_release_version}}:** - **Streaming Node**: Enhanced data processing capabilities -- **Woodpecker MQ**: Improved message queue with reduced maintenance overhead, see [Use Woodpecker](use-woodpecker.md) for detail +- **Woodpecker MQ (default)**: This Docker deployment runs Woodpecker as the message queue with the **local filesystem** as its WAL backend, so no external message-queue service is required. See [Woodpecker](woodpecker.md). - **Optimized Architecture**: Consolidated components for better performance Always download the latest script to ensure you get the most recent configurations and architecture improvements. @@ -106,6 +106,14 @@ $ bash standalone_embed.sh stop $ bash standalone_embed.sh delete ``` +## Optional dependencies + +By default this deployment runs **Woodpecker** (local-filesystem WAL) as the message queue and an **embedded etcd** for metadata — nothing else to install. To use a different message queue or connect external object storage / metadata, see: + +- Message queue: [Woodpecker](woodpecker.md) (default) · [Pulsar](mq_pulsar.md) · [Kafka](mq_kafka.md) · [RocksMQ](mq_rocksmq.md) +- Object storage: [MinIO](deploy_s3.md) (default) · [AWS S3](deploy_s3.md) · [Azure Blob](abs.md) · [GCP Cloud Storage](gcs.md) · [Aliyun OSS](deploy_s3.md) · [Tencent COS](deploy_s3.md) · [Huawei OBS](deploy_s3.md) · [S3-compatible](deploy_s3.md) +- Metadata: [etcd](deploy_etcd.md) + ## What's next Having installed Milvus in Docker, you can: diff --git a/site/en/getstarted/run-milvus-docker/install_standalone-windows.md b/site/en/getstarted/run-milvus-docker/install_standalone-windows.md index d4405c6d0..61080780c 100644 --- a/site/en/getstarted/run-milvus-docker/install_standalone-windows.md +++ b/site/en/getstarted/run-milvus-docker/install_standalone-windows.md @@ -3,11 +3,13 @@ id: install_standalone-windows.md label: Docker related_key: Docker summary: Learn how to install Milvus standalone with Docker Desktop for Windows. -title: Run Milvus in Docker (Linux) +title: Run Milvus in Docker (Windows) --- # Run Milvus in Docker (Windows) +> By default, this deployment runs **Woodpecker** (local filesystem) as its message queue, so no external message-queue service is required. See [Woodpecker](woodpecker.md). + This page demonstrates how to run Milvus on Windows using Docker Desktop for Windows.​ ## Prerequisites​ diff --git a/site/en/getstarted/run-milvus-docker/prerequisite-docker.md b/site/en/getstarted/run-milvus-docker/prerequisite-docker.md index 8fab33b69..10d32be13 100644 --- a/site/en/getstarted/run-milvus-docker/prerequisite-docker.md +++ b/site/en/getstarted/run-milvus-docker/prerequisite-docker.md @@ -32,7 +32,8 @@ The following dependencies will be obtained and configured automatically when Mi | -------- | ----------------------------- | ---- | | etcd | 3.5.0 | See [additional disk requirements](#Additional-disk-requirements). | | MinIO | RELEASE.2024-12-18T13-15-44Z | | -| Pulsar | 2.8.2 | | +| Woodpecker | Bundled with Milvus | Default message queue (embedded); no separate service to install. | +| Pulsar | 2.8.2 | Optional — only if you switch the message queue to Pulsar; not installed by default. | ### Additional disk requirements diff --git a/site/en/getstarted/run-milvus-k8s/install_cluster-helm.md b/site/en/getstarted/run-milvus-k8s/install_cluster-helm.md index b80e3620b..c834e66cc 100644 --- a/site/en/getstarted/run-milvus-k8s/install_cluster-helm.md +++ b/site/en/getstarted/run-milvus-k8s/install_cluster-helm.md @@ -80,7 +80,7 @@ helm install my-release zilliztech/milvus \ --set streaming.enabled=true ``` -**Note**: Standalone mode uses Woodpecker as the default message queue and enables the Streaming Node component. For details, refer to the [Architecture Overview](architecture_overview.md) and [Use Woodpecker](use-woodpecker.md). +**Note**: Standalone mode uses Woodpecker as the default message queue and enables the Streaming Node component. Standalone deployments run Woodpecker **embedded** in the Milvus pod; the dedicated Woodpecker **service** (separate pods) is used for **distributed/cluster** deployments only. For details, refer to the [Architecture Overview](architecture_overview.md) and [Woodpecker](woodpecker.md).
@@ -94,11 +94,13 @@ helm install my-release zilliztech/milvus \ --set pulsarv3.enabled=false \ --set woodpecker.enabled=true \ --set streaming.enabled=true \ + --set streaming.woodpecker.embedded=false \ --set indexNode.enabled=false ``` **What this command does:** - Uses **Woodpecker** as the message queue (recommended for reduced maintenance) +- Runs **Woodpecker as a dedicated service** (a separate StatefulSet), not embedded in the streaming node - Enables the new **Streaming Node** component for improved performance - Disables the legacy **Index Node** (functionality is now handled by Data Node) - Disables Pulsar to use Woodpecker instead @@ -115,16 +117,7 @@ For complete architecture details, refer to the [Architecture Overview](architec
-**Alternative Message Queue Options:** - -If you prefer to use **Pulsar** (traditional choice) instead of Woodpecker: - -```bash -helm install my-release zilliztech/milvus \ - --set image.all.tag=v{{var.milvus_release_version}} \ - --set streaming.enabled=true \ - --set indexNode.enabled=false -``` +**Alternative message queues:** To deploy with Pulsar, Kafka, or RocksMQ instead of Woodpecker, see [Optional dependencies](#Optional-dependencies). **Next steps:** The command above deploys Milvus with recommended configurations. For production use: @@ -136,14 +129,6 @@ The command above deploys Milvus with recommended configurations. For production **Important notes:** - **Release naming**: Use only letters, numbers, and dashes (no dots allowed) -- **Kubernetes v1.25+**: If you encounter PodDisruptionBudget issues, use this workaround: - ```bash - helm install my-release zilliztech/milvus \ - --set pulsar.bookkeeper.pdb.usePolicy=false \ - --set pulsar.broker.pdb.usePolicy=false \ - --set pulsar.proxy.pdb.usePolicy=false \ - --set pulsar.zookeeper.pdb.usePolicy=false - ``` For more information, see [Milvus Helm Chart](https://artifacthub.io/packages/helm/milvus/milvus) and [Helm documentation](https://helm.sh/docs/). @@ -168,23 +153,26 @@ my-release-milvus-datanode-68cb87dcbd-4khpm 1/1 Running 0 3m23s my-release-milvus-mixcoord-7fb9488465-dmbbj 1/1 Running 0 3m23s my-release-milvus-proxy-6bd7f5587-ds2xv 1/1 Running 0 3m24s my-release-milvus-querynode-5cd8fff495-k6gtg 1/1 Running 0 3m24s -my-release-milvus-streaming-node-xxxxxxxxx 1/1 Running 0 3m24s +my-release-milvus-streamingnode-7b8cfc769c-mqmsm 1/1 Running 0 3m24s +my-release-milvus-woodpecker-0 1/1 Running 0 3m24s +my-release-milvus-woodpecker-1 1/1 Running 0 3m24s +my-release-milvus-woodpecker-2 1/1 Running 0 3m24s +my-release-milvus-woodpecker-3 1/1 Running 0 3m24s my-release-minio-0 1/1 Running 0 3m23s my-release-minio-1 1/1 Running 0 3m23s my-release-minio-2 1/1 Running 0 3m23s my-release-minio-3 1/1 Running 0 3m23s -my-release-pulsar-autorecovery-86f5dbdf77-lchpc 1/1 Running 0 3m24s -my-release-pulsar-bookkeeper-0 1/1 Running 0 3m23s -my-release-pulsar-bookkeeper-1 1/1 Running 0 98s -my-release-pulsar-broker-556ff89d4c-2m29m 1/1 Running 0 3m23s -my-release-pulsar-proxy-6fbd75db75-nhg4v 1/1 Running 0 3m23s -my-release-pulsar-zookeeper-0 1/1 Running 0 3m23s -my-release-pulsar-zookeeper-metadata-98zbr 0/1 Completed 0 3m24s ``` **Key components to verify:** -- **Milvus components**: `mixcoord`, `datanode`, `querynode`, `proxy`, `streaming-node` -- **Dependencies**: `etcd` (metadata), `minio` (object storage), `pulsar` (message queue) +- **Milvus components**: `mixcoord`, `datanode`, `querynode`, `proxy`, `streamingnode` +- **Dependencies**: `etcd` (metadata), `minio` (object storage), `woodpecker` (message queue) + +
+ +With `streaming.woodpecker.embedded=false`, Woodpecker runs as a **dedicated StatefulSet** (`my-release-milvus-woodpecker`, 4 replicas by default — a quorum of 3 nodes plus one spare for fault tolerance; do not set `woodpecker.replicaCount` below 3) fronted by a headless service, using MinIO as its storage backend — so the cluster has a separate `woodpecker` pod set, distinct from the streaming node. + +
You can also access the **Milvus WebUI** at `http://127.0.0.1:9091/webui/` once port forwarding is set up (see next step). For details, refer to [Milvus WebUI](milvus-webui.md). @@ -284,7 +272,7 @@ The above command renders chart templates for a Milvus cluster and saves the out
-- To install a Milvus instance in the standalone mode where all Milvus components are contained within a single pod, you should run `helm template my-release --set cluster.enabled=false --set etcd.replicaCount=1 --set minio.mode=standalone --set pulsarv3.enabled=false zilliztech/milvus > milvus_manifest.yaml` instead to render chart templates for a Milvus instance in a standalone mode. +- To install a Milvus instance in the standalone mode where all Milvus components are contained within a single pod, you should run `helm template my-release --set cluster.enabled=false --set etcd.replicaCount=1 --set minio.mode=standalone --set pulsarv3.enabled=false --set standalone.messageQueue=woodpecker --set woodpecker.enabled=true --set streaming.enabled=true zilliztech/milvus > milvus_manifest.yaml` instead to render chart templates for a Milvus instance in a standalone mode. - To change Milvus configurations, download the [`value.yaml`](https://raw.githubusercontent.com/milvus-io/milvus-helm/master/charts/milvus/values.yaml) template, place your desired settings in it, and use `helm template -f values.yaml my-release zilliztech/milvus > milvus_manifest.yaml` to render the manifest accordingly.
@@ -323,7 +311,7 @@ $ for image in $(find . -type f -name "*.tar.gz") ; do gunzip -c $image | docker $ kubectl apply -f milvus_manifest.yaml ``` -Till now, you can follow steps [2](#2-Check-Milvus-cluster-status) and [3](#3-Forward-a-local-port-to-Milvus) of the online install to check the cluster status and forward a local port to Milvus. +Till now, you can follow steps [2](#2-Check-Milvus-cluster-status) and [3](#3-Connect-to-Milvus) of the online install to check the cluster status and forward a local port to Milvus. ## Upgrade running Milvus cluster @@ -342,6 +330,14 @@ Run the following command to uninstall Milvus. $ helm uninstall my-release ``` +## Optional dependencies + +This deployment runs **Woodpecker** as the message queue, **etcd** for metadata, and **MinIO** for object storage. To use a different message queue or connect external object storage / metadata, see: + +- Message queue: [Woodpecker](woodpecker.md) (default) · [Pulsar](mq_pulsar.md) · [Kafka](mq_kafka.md) · [RocksMQ](mq_rocksmq.md) +- Object storage: [MinIO](deploy_s3.md) (default) · [AWS S3](deploy_s3.md) · [Azure Blob](abs.md) · [GCP Cloud Storage](gcs.md) · [Aliyun OSS](deploy_s3.md) · [Tencent COS](deploy_s3.md) · [Huawei OBS](deploy_s3.md) · [S3-compatible](deploy_s3.md) +- Metadata: [etcd](deploy_etcd.md) + ## What's next Having installed Milvus in Docker, you can: diff --git a/site/en/getstarted/run-milvus-k8s/install_cluster-milvusoperator.md b/site/en/getstarted/run-milvus-k8s/install_cluster-milvusoperator.md index 152bdac32..06219a2f2 100644 --- a/site/en/getstarted/run-milvus-k8s/install_cluster-milvusoperator.md +++ b/site/en/getstarted/run-milvus-k8s/install_cluster-milvusoperator.md @@ -12,7 +12,7 @@ This page illustrates how to start a Milvus instance in Kubernetes using [Milvus ## Overview -Milvus Operator is a solution that helps you deploy and manage a full Milvus service stack to target Kubernetes (K8s) clusters. The stack includes all Milvus components and relevant dependencies like etcd, Pulsar, and MinIO. +Milvus Operator is a solution that helps you deploy and manage a full Milvus service stack to target Kubernetes (K8s) clusters. The stack includes all Milvus components and relevant dependencies like etcd and MinIO, with Woodpecker as the built-in message queue. ## Prerequisites @@ -136,7 +136,7 @@ $ kubectl apply -f https://raw.githubusercontent.com/zilliztech/milvus-operator/ The command above deploys a Milvus cluster with **Woodpecker** as the message queue (recommended for v{{var.milvus_release_version}}) and all new architectural components including the Streaming Node. **Architecture highlights in this deployment:** -- **Message Queue**: [Uses Woodpecker](use-woodpecker.md) (reduces infrastructure maintenance) +- **Message Queue**: [Uses Woodpecker](woodpecker.md) (reduces infrastructure maintenance) - **Streaming Node**: Enabled for enhanced data processing - **Mix Coordinator**: Consolidated coordinator components for improved efficiency @@ -170,11 +170,6 @@ status: reason: StorageReady status: "True" type: StorageReady - - lastTransitionTime: "xxxx-xx-xxTxx:xx:xxZ" - message: Pulsar is ready - reason: PulsarReady - status: "True" - type: PulsarReady - lastTransitionTime: "xxxx-xx-xxTxx:xx:xxZ" message: Etcd endpoints is healthy reason: EtcdReady @@ -189,7 +184,7 @@ status: status: Healthy ``` -Milvus Operator creates Milvus dependencies, such as etcd, Pulsar, and MinIO, and then Milvus components, such as proxy, coordinators, and nodes. +Milvus Operator creates Milvus dependencies, such as etcd and MinIO, and then Milvus components, such as proxy, coordinators, and nodes. Once your Milvus cluster is ready, the status of all pods in the Milvus cluster should be similar to the following. @@ -293,7 +288,7 @@ $ kubectl delete milvus my-release
-- When you delete the Milvus cluster using the default configuration, dependencies like etcd, Pulsar, and MinIO are not deleted. Therefore, next time when you install the same Milvus cluster instance, these dependencies will be used again. +- When you delete the Milvus cluster using the default configuration, dependencies like etcd and MinIO are not deleted. Therefore, next time when you install the same Milvus cluster instance, these dependencies will be used again. - To delete the dependencies and persistent volume claims (PVCs) along with the Milvus cluster, see [configuration file](https://github.com/zilliztech/milvus-operator/blob/main/config/samples/milvus_deletion.yaml).
@@ -317,6 +312,14 @@ $ helm -n milvus-operator uninstall milvus-operator $ kubectl delete -f https://raw.githubusercontent.com/zilliztech/milvus-operator/v{{var.milvus_operator_version}}/deploy/manifests/deployment.yaml ``` +## Optional dependencies + +This deployment uses **Woodpecker** as the message queue, **etcd** for metadata, and **MinIO** for object storage, all provisioned by Milvus Operator. To use a different backend with the Operator, see: + +- Message queue: [Woodpecker](woodpecker.md) (default) · [Pulsar](message_storage_operator.md#Configure-Pulsar) · [Kafka](message_storage_operator.md#Configure-Kafka) · [RocksMQ](message_storage_operator.md#Configure-RocksMQ) +- Object storage: [Configure object storage with Milvus Operator](object_storage_operator.md) +- Metadata: [Configure etcd with Milvus Operator](meta_storage_operator.md) + ## What's next Having installed Milvus in Docker, you can: diff --git a/site/en/getstarted/run-milvus-k8s/prerequisite-helm.md b/site/en/getstarted/run-milvus-k8s/prerequisite-helm.md index 10a2e6165..b47a79c23 100644 --- a/site/en/getstarted/run-milvus-k8s/prerequisite-helm.md +++ b/site/en/getstarted/run-milvus-k8s/prerequisite-helm.md @@ -35,7 +35,8 @@ minikube is required when running Kubernetes cluster locally. minikube requires | -------- | ----------------------------- | ---- | | etcd | 3.5.0 | See [additional disk requirements](#Additional-disk-requirements). | | MinIO | RELEASE.2024-12-18T13-15-44Z | | -| Pulsar | 2.8.2 | | +| Woodpecker | Bundled with Milvus | Default message queue; runs as a dedicated service for distributed deployments. | +| Pulsar | 2.8.2 | Optional — only if you switch the message queue to Pulsar; not installed by default. | ### Additional disk requirements diff --git a/site/en/menuStructure/en.json b/site/en/menuStructure/en.json index 3c72e787c..33df2f637 100644 --- a/site/en/menuStructure/en.json +++ b/site/en/menuStructure/en.json @@ -1471,11 +1471,96 @@ } ] }, + { + "label": "Data Infrastructure & Integration", + "id": "data_infra_integration", + "isMenu": true, + "order": 9, + "children": [ + { + "label": "Overview", + "id": "data-infra-integration-overview.md", + "order": 0, + "children": [] + }, + { + "label": "Metadata", + "id": "metadata", + "isMenu": true, + "order": 1, + "children": [ + { + "label": "etcd", + "id": "etcd.md", + "order": 0, + "children": [] + } + ] + }, + { + "label": "Object Storage", + "id": "object-storage.md", + "order": 2, + "children": [] + }, + { + "label": "Message Queue (mqType)", + "id": "mqtype", + "isMenu": true, + "order": 4, + "children": [ + { + "label": "Overview", + "id": "mqtype-overview.md", + "order": 0, + "children": [] + }, + { + "label": "Woodpecker", + "id": "woodpecker.md", + "order": 1, + "children": [] + }, + { + "label": "Pulsar", + "id": "mq_pulsar.md", + "order": 2, + "children": [ + { + "label": "Upgrade to Pulsar v3", + "id": "upgrade-pulsar-v3.md", + "order": 0, + "children": [] + }, + { + "label": "Continue Using Pulsar v2", + "id": "use-pulsar-v2.md", + "order": 1, + "children": [] + } + ] + }, + { + "label": "Kafka", + "id": "mq_kafka.md", + "order": 3, + "children": [] + }, + { + "label": "RocksMQ", + "id": "mq_rocksmq.md", + "order": 4, + "children": [] + } + ] + } + ] + }, { "label": "Administration Guide", "id": "admin_guide", "isMenu": true, - "order": 9, + "order": 10, "children": [ { "label": "Deployment", @@ -1645,86 +1730,6 @@ } ] }, - { - "label": "Manage Dependencies", - "id": "manage_dependencies", - "order": 2, - "isMenu": true, - "children": [ - { - "label": "With Docker or Helm", - "id": "docker_helm_dependencies", - "order": 0, - "isMenu": true, - "children": [ - { - "label": "Object Storage", - "id": "deploy_s3.md", - "order": 0, - "children": [] - }, - { - "label": "Meta Storage", - "id": "deploy_etcd.md", - "order": 1, - "children": [] - }, - { - "label": "Message Storage", - "id": "deploy_pulsar.md", - "order": 2, - "children": [] - } - ] - }, - { - "label": "With Milvus Operator", - "id": "milvus_operator_dependencies", - "order": 1, - "isMenu": true, - "children": [ - { - "label": "Object Storage", - "id": "object_storage_operator.md", - "order": 0, - "children": [] - }, - { - "label": "Meta Storage", - "id": "meta_storage_operator.md", - "order": 1, - "children": [] - }, - { - "label": "Message Storage", - "id": "message_storage_operator.md", - "order": 2, - "children": [] - } - ] - }, - { - "label": "Use Pulsar with Milvus", - "id": "pulsar_dependencies", - "order": 2, - "isMenu": true, - "children": [ - { - "label": "Upgrade to Pulsar v3", - "id": "upgrade-pulsar-v3.md", - "order": 0, - "children": [] - }, - { - "label": "Continue Using Pulsar v2", - "id": "use-pulsar-v2.md", - "order": 1, - "children": [] - } - ] - } - ] - }, { "label": "Scaling", "id": "scaling", @@ -1953,28 +1958,34 @@ "order": "12", "children": [] }, - { - "label": "Woodpecker", - "id": "use-woodpecker.md", - "order": 13, - "children": [] - }, { "label": "Switch MQ Type", "id": "switch_milvus_mq_type", "order": 14, "children": [ { - "label": "Milvus Cluster switch MQ Type", - "id": "switch_milvus_cluster_mq_type-operator.md", + "label": "Overview", + "id": "switch-mq-type.md", "order": 0, "children": [] }, { - "label": "Milvus Standalone switch MQ type", - "id": "switch_milvus_standalone_mq_type-operator.md", + "label": "RocksMQ ↔ Woodpecker", + "id": "switch-rocksmq-woodpecker.md", "order": 1, "children": [] + }, + { + "label": "Pulsar ↔ Woodpecker", + "id": "switch-pulsar-woodpecker.md", + "order": 2, + "children": [] + }, + { + "label": "Kafka ↔ Woodpecker", + "id": "switch-kafka-woodpecker.md", + "order": 3, + "children": [] } ] } @@ -3049,7 +3060,7 @@ "label": "FAQs", "id": "faq", "isMenu": true, - "order": 9, + "order": 11, "children": [ { "label": "Performance FAQs", diff --git a/site/en/reference/glossary.md b/site/en/reference/glossary.md index f8a99e9ce..5faf5a1c3 100644 --- a/site/en/reference/glossary.md +++ b/site/en/reference/glossary.md @@ -43,7 +43,7 @@ In Milvus, a collection is equivalent to a table in a relational database manage ## Dependency -A dependency is a program that another program relies on to work. Milvus' dependencies include etcd (stores meta data), MinIO or S3 (object storage), and Pulsar (manages snapshot logs). For more information, refer to [Manage Dependencies](https://milvus.io/docs/manage_dependencies.md#Manage-Dependencies). +A dependency is a program that another program relies on to work. Milvus' dependencies include etcd (stores meta data), MinIO or S3 (object storage), and a message queue such as Woodpecker (manages snapshot logs). For more information, refer to [Data Infrastructure & Integration](https://milvus.io/docs/data-infra-integration-overview.md). ## Dynamic schema