Skip to content
This repository was archived by the owner on Aug 16, 2023. It is now read-only.

Commit dc24e60

Browse files
LoveEachDayLoveEachDay
authored
Refine milvus config parameters (#403)
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com> Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
1 parent 7c78c97 commit dc24e60

4 files changed

Lines changed: 27 additions & 509 deletions

File tree

‎charts/milvus/Chart.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
apiVersion: v1
22
name: milvus
3-
appVersion: "2.2.0"
3+
appVersion: "2.2.1"
44
kubeVersion: "^1.10.0-0"
55
description: Milvus is an open-source vector database built to power AI applications and vector similarity search.
6-
version: 3.3.6
6+
version: 4.0.0
77
keywords:
88
- milvus
99
- elastic

‎charts/milvus/README.md‎

Lines changed: 14 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ Assume the release name is `my-release`:
3535
$ helm upgrade --install my-release --set cluster.enabled=false --set etcd.replicaCount=1 --set pulsar.enabled=false --set minio.mode=standalone milvus/milvus
3636
```
3737
By default, milvus standalone uses `rocksmq` as message queue. You can also use `pulsar` or `kafka` as message queue:
38+
3839
```bash
3940
# Helm v3.x
4041
# Milvus Standalone with pulsar as message queue
@@ -67,11 +68,24 @@ $ helm upgrade --install my-release milvus/milvus --set pulsar.enabled=false --s
6768
> **IMPORTANT** If you have installed a milvus cluster with version below v2.1.x, you need follow the instructions at here: https://github.com/milvus-io/milvus/blob/master/deployments/migrate-meta/README.md. After meta migration, you use `helm upgrade` to update your cluster again.
6869
6970
E.g. to scale out query node from 1(default) to 2:
71+
7072
```bash
7173
# Helm v3.x
7274
$ helm upgrade --install --set queryNode.replicas=2 my-release milvus/milvus
7375
```
7476

77+
### Breaking Changes
78+
> **IMPORTANT** Milvus helm chart 4.0.0 has breaking changes for milvus configuration. Previously, you can set segment size like this `--set dataCoordinator.segment.maxSize=1024`. Now we have remove all the shortcut config option. Instead, you can set using `extraConfigFiles` like this:
79+
```bash
80+
extraConfigFiles:
81+
user.yaml: |+
82+
dataCoord:
83+
segment:
84+
maxSize: 1024
85+
```
86+
87+
So if you had deployed a cluster with helm chart version below 4.0.0 and also specified extra config, you need set the configs under `extraConfigFiles` when running `helm upgrade`.
88+
7589
## Uninstall the Chart
7690

7791
```bash
@@ -125,8 +139,6 @@ The following table lists the configurable parameters of the Milvus Service and
125139
| `metrics.enabled` | Export Prometheus monitoring metrics | `true` |
126140
| `metrics.serviceMonitor.enabled` | Create ServiceMonitor for Prometheus operator | `false` |
127141
| `metrics.serviceMonitor.additionalLabels` | Additional labels that can be used so ServiceMonitor will be discovered by Prometheus | `unset` |
128-
| `metadata.rootPath` | Root of key prefix to etcd | `by-dev` |
129-
| `authorization.enabled` | Enable milvus authorization | `false` |
130142
| `log.level` | Logging level to be used. Valid levels are `debug`, `info`, `warn`, `error`, `fatal` | `info` |
131143
| `log.file.maxSize` | The size limit of the log file (MB) | `300` |
132144
| `log.file.maxAge` | The maximum number of days that the log is retained. (day) | `10` |
@@ -140,38 +152,6 @@ The following table lists the configurable parameters of the Milvus Service and
140152
| `log.persistence.persistentVolumeClaim.accessModes` | The Milvus logs data Persistence access modes | `ReadWriteOnce` |
141153
| `log.persistence.persistentVolumeClaim.size` | The size of Milvus logs data Persistent Volume Storage Class | `5Gi` |
142154
| `log.persistence.persistentVolumeClaim.subPath` | SubPath for Milvus logs data mount | `unset` |
143-
| `msgChannel.chanNamePrefix.cluster` | Pulsar topic name prefix | `by-dev` |
144-
| `quotaAndLimits.enabled` | Enable milvus quota and limits | `false` |
145-
| `quotaAndLimits.quotaCenterCollectInterval` | Collect metrics interval | `3` |
146-
| `quotaAndLimits.ddl.eabled` | Enable milvus ddl limit | `false` |
147-
| `quotaAndLimits.ddl.collectionRate` | Milvus ddl collection rate qps | `unset` |
148-
| `quotaAndLimits.ddl.partitionRate` | Milvus ddl partition rate qps | `unset` |
149-
| `quotaAndLimits.indexRate.eabled` | Enable milvus index rate limit | `false` |
150-
| `quotaAndLimits.indexRate.max` | Milvus max index rate qps | `unset` |
151-
| `quotaAndLimits.flushRate.eabled` | Enable milvus flush rate limit | `false` |
152-
| `quotaAndLimits.flushRate.max` | Milvus max flush rate qps | `unset` |
153-
| `quotaAndLimits.compactionRate.eabled` | Enable milvus compaction rate limit | `false` |
154-
| `quotaAndLimits.compactionRate.max` | Milvus max compaction rate qps | `unset` |
155-
| `quotaAndLimits.dml.eabled` | Enable milvus dml limit | `false` |
156-
| `quotaAndLimits.dml.insertRate.max` | Milvus dml max insert rate MB/s | `unset` |
157-
| `quotaAndLimits.dml.deleteRate.max` | Milvus dml max delete rate MB/s | `unset` |
158-
| `quotaAndLimits.dml.bulkLoadRate.max` | Milvus dml max bulk load rate MB/s | `unset` |
159-
| `quotaAndLimits.dql.eabled` | Enable milvus dql limit | `false` |
160-
| `quotaAndLimits.dql.searchRate.max` | Milvus dml max search vps | `unset` |
161-
| `quotaAndLimits.dql.queryRate.max` | Milvus dml max query qps | `unset` |
162-
| `quotaAndLimits.limitWriting.forceDeny` | Deny write requests if quota exceeded | `false` |
163-
| `quotaAndLimits.limitWriting.ttProtection.enabled` | Enable milvus time tick protection | `true` |
164-
| `quotaAndLimits.limitWriting.ttProtection.maxTimeTickDelay` | Max time tick delay in seconds | `30` |
165-
| `quotaAndLimits.limitWriting.memProtection.enabled` | Enable milvus memory protection | `true` |
166-
| `quotaAndLimits.limitWriting.memProtection.dataNodeMemoryLowWaterLevel` | Low water level for data node | `0.85` |
167-
| `quotaAndLimits.limitWriting.memProtection.dataNodeMemoryHighWaterLevel` | High water level for data node | `0.95` |
168-
| `quotaAndLimits.limitWriting.memProtection.queryNodeMemoryLowWaterLevel` | Low water level for query node | `0.85` |
169-
| `quotaAndLimits.limitWriting.memProtection.queryNodeMemoryHighWaterLevel` | High water level for query node | `0.95` |
170-
| `quotaAndLimits.limitReading.forceDeny` | Deny read requests if quota exceeded | `false` |
171-
| `quotaAndLimits.limitReading.queueProtection.enabled` | Enable queue protection | `false` |
172-
| `quotaAndLimits.limitReading.queueProtection.nqInQueueThreshold` | NQ in queue threshold | `unset` |
173-
| `quotaAndLimits.limitReading.queueProtection.queueLatencyThreshold` | Queue latency threshold | `unset` |
174-
| `quotaAndLimits.limitReading.queueProtection.coolOffSpeed` | Cooloff speed | `0.9` |
175155
| `externalS3.enabled` | Enable or disable external S3 | `false` |
176156
| `externalS3.host` | The host of the external S3 | `unset` |
177157
| `externalS3.port` | The port of the external S3 | `unset` |
@@ -221,8 +201,6 @@ The following table lists the configurable parameters of the Milvus Standalone c
221201
| `standalone.profiling.enabled` | Whether to enable live profiling | `false` |
222202
| `standalone.extraEnv` | Additional Milvus Standalone container environment variables | `[]` |
223203
| `standalone.messageQueue` | Message queue for Milvus Standalone: rocksmq, pulsar, kafka | `rocksmq` |
224-
| `standalone.rocksmq.retentionTimeInMinutes` | Set the retention time of rocksmq | `10080` |
225-
| `standalone.rocksmq.retentionSizeInMB` | Set the retention size of rocksmq | `0` |
226204
| `standalone.persistence.enabled` | Use persistent volume to store Milvus standalone data | `true` |
227205
| `standalone.persistence.mountPath` | Milvus standalone data persistence volume mount path | `/var/lib/milvus` |
228206
| `standalone.persistence.annotations` | PersistentVolumeClaim annotations | `{}` |
@@ -314,9 +292,6 @@ The following table lists the configurable parameters of the Milvus Query Node c
314292
| `queryNode.disk.enabled` | Whether to enable disk for query | `true` |
315293
| `queryNode.profiling.enabled` | Whether to enable live profiling | `false` |
316294
| `queryNode.extraEnv` | Additional Milvus Query Node container environment variables | `[]` |
317-
| `queryNode.grouping.enabled` | Enable grouping small nq search | `true` |
318-
| `queryNode.grouping.maxNQ` | Grouping small nq search max threshold | `1000` |
319-
| `queryNode.scheduler.maxReadConcurrentRatio` | Concurrency ratio of read tasks | `2.0` |
320295

321296
### Milvus Index Coordinator Deployment Configuration
322297

@@ -341,7 +316,6 @@ The following table lists the configurable parameters of the Milvus Index Coordi
341316
| `indexCoordinator.service.loadBalancerIP` | IP address to assign to load balancer (if supported) | `unset` |
342317
| `indexCoordinator.service.loadBalancerSourceRanges` | List of IP CIDRs allowed access to lb (if supported) | `[]` |
343318
| `indexCoordinator.service.externalIPs` | Service external IP addresses | `[]` |
344-
| `indexCoordinator.gc.interval` | GC interval in seconds | `600` |
345319

346320
### Milvus Index Node Deployment Configuration
347321

@@ -359,7 +333,6 @@ The following table lists the configurable parameters of the Milvus Index Node c
359333
| `indexNode.disk.enabled` | Whether to enable disk for index node | `true` |
360334
| `indexNode.profiling.enabled` | Whether to enable live profiling | `false` |
361335
| `indexNode.extraEnv` | Additional Milvus Index Node container environment variables | `[]` |
362-
| `indexNode.scheduler.buildParallel` | Index task build paralellism | `1` |
363336

364337
### Milvus Data Coordinator Deployment Configuration
365338

@@ -374,13 +347,6 @@ The following table lists the configurable parameters of the Milvus Data Coordin
374347
| `dataCoordinator.tolerations` | Toleration labels for Milvus Data Coordinator pods assignment | `[]` |
375348
| `dataCoordinator.heaptrack.enabled` | Whether to enable heaptrack | `false` |
376349
| `dataCoordinator.profiling.enabled` | Whether to enable live profiling | `false` |
377-
| `dataCoordinator.segment.maxSize` | Maximum size of a segment in MB | `512` |
378-
| `dataCoordinator.segment.diskSegmentMaxSize` | Maximum size of a segment in MB for disk index collection | `2048` |
379-
| `dataCoordinator.segment.sealProportion` | Minimum proportion for a segment which can be sealed | `0.25` |
380-
| `dataCoordinator.segment.maxLife` | Maximum lifetime of a segment in seconds | `3600` |
381-
| `dataCoordinator.segment.maxIdleTime` | Maximum idle time for growing segment in seconds | `300` |
382-
| `dataCoordinator.segment.minSizeFromIdleToSealed` | The minimum size in MB of segment which can be idle from sealed | `16` |
383-
| `dataCoordinator.segment.smallProportion` | The proportion for a sealed segment, which would not be compacted | `0.9` |
384350
| `dataCoordinator.extraEnv` | Additional Milvus Data Coordinator container environment variables | `[]` |
385351
| `dataCoordinator.service.type` | Service type | `ClusterIP` |
386352
| `dataCoordinator.service.port` | Port where service is exposed | `19530` |

0 commit comments

Comments
 (0)