@@ -57,6 +57,53 @@ Refer to the specific cloud provider guidelines:
57
57
- [ Azure Deployment
58
58
guidelines] ( /installation/install-on-azure/appendix-deployment-guidelines/ )
59
59
60
+ ## Custom Cluster Sizes
61
+
62
+ When installing the materialize helm chart you may specify a list of cluster sizes.
63
+ ``` yaml
64
+ operator :
65
+ clusters :
66
+ sizes :
67
+ ...
68
+ ```
69
+ These cluster sizes will be used for internal clusters, such as the ` system_cluster ` as well as
70
+ user clusters. For that reason we recommend that you at minimum keep the 25-200cc cluster sizes.
71
+ If you wish to have have cluster sizes
72
+ | Field | type | Description |
73
+ | workers | int | The number of timely workers in your cluster replica. |
74
+ | scale | int | The number of processes or pods to use in a cluster replica. |
75
+ | cpu_exclusive | bool | Whether the workers should attempt to pin to a particular cpu core. |
76
+ | cpu_limit | float | The k8s limit for CPU for a replica pod in cores. |
77
+ | memory_limit | float | The k8s limit for memory for a replica pod in bytes. |
78
+ | disk_limit | float | The size of the nvme persistent volume to provision for a replica pod in bytes. |
79
+ | credits_per_hour | string | This is a cloud attribute that should be set to "0.00" in self-managed. |
80
+
81
+ ### Recommendations for cluster sizes:
82
+ ** workers**
83
+ * We recommend using 1 worker per CPU core while maintaining a minimum of 1 worker.
84
+
85
+ ** scale**
86
+ * Scale is used to scale out replicas horrizontally. Each pod will be
87
+ provisioned using the settings defined in the size definition. This should
88
+ only be greater than one when a replica needs to take on limits that are
89
+ greater than the maximum limits permitted on a single node.
90
+
91
+ ** cpu_exclusive**
92
+ * set this to true if and only if a whole number is used for cpu_limit and the cpu management policy is set to static on the k8s cluster.
93
+
94
+ ** cpu_limit**
95
+ * K8s will only allow CPU Affinity for pods taking a whole number of cores (not hyperthreads). When possible use a whole number.
96
+
97
+ Ratios:
98
+ ** CPU : Memory**
99
+ * For most workloads, we find find a 1:8 ratio of cores to Gib to work well, but this can be quite workload dependent.
100
+
101
+ ** Memory : Disk**
102
+ * Materialize attempts to keep actively used data in memory. In order to allow
103
+ for larger workloads data can spill to disk at some cost for performance.
104
+ Our current recommendation is a 1:2 ratio of memory to disk.
105
+
106
+
60
107
## See also
61
108
62
109
- [ Configuration] ( /installation/configuration/ )
0 commit comments