@@ -57,6 +57,50 @@ Refer to the specific cloud provider guidelines:
57
57
- [ Azure Deployment
58
58
guidelines] ( /installation/install-on-azure/appendix-deployment-guidelines/ )
59
59
60
+ ## Custom Cluster Sizes
61
+
62
+ When installing the materialize helm chart you may specify a list of cluster sizes.
63
+ ``` yaml
64
+ operator :
65
+ clusters :
66
+ sizes :
67
+ ...
68
+ ```
69
+ These cluster sizes will be used for internal clusters, such as the ` system_cluster ` as well as
70
+ user clusters. For that reason we recommend that you at minimum keep the 25-200cc cluster sizes.
71
+ If you wish to have have cluster sizes
72
+ | Field | type | Description |
73
+ | workers | int | The number of timely workers in your cluster replica. |
74
+ | scale | int | The number of processes or pods to use in a cluster replica. |
75
+ | cpu_exclusive | bool | Whether the workers should attempt to pin to a particular cpu core. |
76
+ | cpu_limit | float | The k8s limit for CPU for a replica pod in cores. |
77
+ | memory_limit | float | The k8s limit for memory for a replica pod in bytes. |
78
+ | disk_limit | float | The size of the nvme persistent volume to provision for a replica pod in bytes. |
79
+ | credits_per_hour | string | This is a cloud attribute that should be set to "0.00" in self-managed. |
80
+
81
+ ### Recommendations for cluster sizes:
82
+ ** workers**
83
+ * This should match the number of CPU cores ~ max(1, floor(cpu_limit))
84
+ ** scale**
85
+ * Scale is used to scale out replicas horrizontally. Each pod will be
86
+ provisioned using the settings defined in the size definition. This should
87
+ only be greater than one when a replica needs to take on limits that are
88
+ greater than the maximum limits permitted on a single node.
89
+ ** cpu_exclusive**
90
+ * This should be set to true if and only if a whole number is used for cpu_limit and cpu affinity is turned on in the k8s cluster.
91
+ ** cpu_limit**
92
+ * K8s will only allow CPU Affinity for pods taking a whole number of cores (not hyperthreads). When possible use a whole number.
93
+
94
+ Ratios:
95
+ ** CPU : Memory**
96
+ * For most workloads, we find find a 1:8 ratio of cores to Gib to work well, but this can be quite workload dependent.
97
+
98
+ ** Memory : Disk**
99
+ * Materialize attempts to keep actively used data in memory. In order to allow
100
+ for larger workloads data can spill to disk at some cost for performance.
101
+ Our current recommendation is a 1:2 ratio of memory to disk.
102
+
103
+
60
104
## See also
61
105
62
106
- [ Configuration] ( /installation/configuration/ )
0 commit comments