Skip to content

Commit e073c4b

Browse files
ahmedwaleedmalikxmudrii
ahmedwaleedmalik
and
xmudrii
authored
Support for passing additional args to API server, controller manager, and scheduler (#3162)
* Support for passing additional args to API server, controller manager, and scheduler Signed-off-by: Waleed Malik <[email protected]> * Minor fixes Signed-off-by: Marko Mudrinić <[email protected]> * Fix logic for merging feature gates Signed-off-by: Marko Mudrinić <[email protected]> --------- Signed-off-by: Waleed Malik <[email protected]> Signed-off-by: Marko Mudrinić <[email protected]> Co-authored-by: Marko Mudrinić <[email protected]>
1 parent 21ce2be commit e073c4b

File tree

11 files changed

+611
-6
lines changed

11 files changed

+611
-6
lines changed

‎docs/api_reference/v1beta2.en.md‎

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
+++
22
title = "v1beta2 API Reference"
3-
date = 2024-03-12T21:50:36+02:00
3+
date = 2024-04-29T17:38:32+05:00
44
weight = 11
55
+++
66
## v1beta2
@@ -22,6 +22,8 @@ weight = 11
2222
* [ContainerdRegistry](#containerdregistry)
2323
* [ContainerdRegistryAuthConfig](#containerdregistryauthconfig)
2424
* [ContainerdTLSConfig](#containerdtlsconfig)
25+
* [ControlPlaneComponentConfig](#controlplanecomponentconfig)
26+
* [ControlPlaneComponents](#controlplanecomponents)
2527
* [ControlPlaneConfig](#controlplaneconfig)
2628
* [CoreDNS](#coredns)
2729
* [DNSConfig](#dnsconfig)
@@ -282,6 +284,29 @@ Configures containerd TLS for a registry
282284

283285
[Back to Group](#v1beta2)
284286

287+
### ControlPlaneComponentConfig
288+
289+
290+
291+
| Field | Description | Scheme | Required |
292+
| ----- | ----------- | ------ | -------- |
293+
| flags | Flags is a set of additional flags that will be passed to the control plane component. KubeOne internally configures some flags that are eseeential for the cluster to work. Those flags set by KubeOne will be merged with the ones specified in the configuration. In case of conflict the value provided by the user will be used. Usage of `feature-gates` is not allowed here, use `FeatureGates` field instead. IMPORTANT: Use of these flags is at the user's own risk, as KubeOne does not provide support for issues caused by invalid values and configurations. | map[string]string | false |
294+
| featureGates | FeatureGates is a map of additional feature gates that will be passed on to the control plane component. KubeOne internally configures some feature gates that are eseeential for the cluster to work. Those feature gates set by KubeOne will be merged with the ones specified in the configuration. In case of conflict the value provided by the user will be used. IMPORTANT: Use of these featureGates is at the user's own risk, as KubeOne does not provide support for issues caused by invalid values and configurations. | map[string]bool | false |
295+
296+
[Back to Group](#v1beta2)
297+
298+
### ControlPlaneComponents
299+
300+
301+
302+
| Field | Description | Scheme | Required |
303+
| ----- | ----------- | ------ | -------- |
304+
| controllerManager | ControllerManagerConfig configures the Kubernetes Controller Manager | *[ControlPlaneComponentConfig](#controlplanecomponentconfig) | false |
305+
| scheduler | Scheduler configures the Kubernetes Scheduler | *[ControlPlaneComponentConfig](#controlplanecomponentconfig) | false |
306+
| apiServer | APIServer configures the Kubernetes API Server | *[ControlPlaneComponentConfig](#controlplanecomponentconfig) | false |
307+
308+
[Back to Group](#v1beta2)
309+
285310
### ControlPlaneConfig
286311

287312
ControlPlaneConfig defines control plane nodes
@@ -528,6 +553,7 @@ KubeOneCluster is KubeOne Cluster API Schema
528553
| registryConfiguration | RegistryConfiguration configures how Docker images are pulled from an image registry | *[RegistryConfiguration](#registryconfiguration) | false |
529554
| loggingConfig | LoggingConfig configures the Kubelet's log rotation | [LoggingConfig](#loggingconfig) | false |
530555
| tlsCipherSuites | TLSCipherSuites allows to configure TLS cipher suites for different components. See https://pkg.go.dev/crypto/tls#pkg-constants for possible values. | [TLSCipherSuites](#tlsciphersuites) | true |
556+
| controlPlaneComponents | ControlPlaneComponents configures the Kubernetes control plane components | *[ControlPlaneComponents](#controlplanecomponents) | false |
531557

532558
[Back to Group](#v1beta2)
533559

‎pkg/apis/kubeone/config/config.go‎

Lines changed: 31 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,9 @@ import (
4242

4343
const (
4444
// KubeOneClusterKind is kind of the KubeOneCluster object
45-
KubeOneClusterKind = "KubeOneCluster"
45+
KubeOneClusterKind = "KubeOneCluster"
46+
controlPlaneComponentsWarning = "Usage of the .controlPlaneComponents feature is at your own risk since options configured via this feature cannot properly be validated by KubeOne"
47+
flagsAndFeatureGateOverridesWarning = "\t- %s only covers %s. Some features might also need additional configuration for other components."
4648
)
4749

4850
var (
@@ -185,7 +187,7 @@ func DefaultedV1Beta1KubeOneCluster(versionedCluster *kubeonev1beta1.KubeOneClus
185187
}
186188

187189
// Check for deprecated fields/features for a cluster
188-
checkClusterFeatures(*internalCluster, logger)
190+
checkClusterConfiguration(*internalCluster, logger)
189191

190192
return internalCluster, nil
191193
}
@@ -222,7 +224,7 @@ func DefaultedV1Beta2KubeOneCluster(versionedCluster *kubeonev1beta2.KubeOneClus
222224
}
223225

224226
// Check for deprecated fields/features for a cluster
225-
checkClusterFeatures(*internalCluster, logger)
227+
checkClusterConfiguration(*internalCluster, logger)
226228

227229
return internalCluster, nil
228230
}
@@ -338,8 +340,8 @@ func isDir(dirname string) bool {
338340
return statErr == nil && stat.Mode().IsDir()
339341
}
340342

341-
// checkClusterFeatures checks clusters for usage of alpha and deprecated fields, flags etc. and print a warning if any are found
342-
func checkClusterFeatures(cluster kubeoneapi.KubeOneCluster, logger logrus.FieldLogger) {
343+
// checkClusterConfiguration checks clusters for usage of alpha, deprecated fields, flags, unrecommended features etc. and print a warning if any are found.
344+
func checkClusterConfiguration(cluster kubeoneapi.KubeOneCluster, logger logrus.FieldLogger) {
343345
if cluster.Features.PodSecurityPolicy != nil && cluster.Features.PodSecurityPolicy.Enable {
344346
logger.Warnf("PodSecurityPolicy is deprecated and will be removed with Kubernetes 1.25 release")
345347
}
@@ -351,4 +353,28 @@ func checkClusterFeatures(cluster kubeoneapi.KubeOneCluster, logger logrus.Field
351353
if cluster.CloudProvider.Vsphere != nil && !cluster.CloudProvider.External && len(cluster.CloudProvider.CSIConfig) > 0 {
352354
logger.Warnf(".cloudProvider.csiConfig is provided, but is ignored when used with the in-tree cloud provider")
353355
}
356+
357+
checkFlagsAndFeatureGateOverrides(cluster, logger)
358+
}
359+
360+
func checkFlagsAndFeatureGateOverrides(cluster kubeoneapi.KubeOneCluster, logger logrus.FieldLogger) {
361+
if cluster.ControlPlaneComponents != nil {
362+
logger.Warn(controlPlaneComponentsWarning)
363+
364+
if cluster.ControlPlaneComponents.ControllerManager != nil {
365+
if cluster.ControlPlaneComponents.ControllerManager.Flags != nil || cluster.ControlPlaneComponents.ControllerManager.FeatureGates != nil {
366+
logger.Warnf(flagsAndFeatureGateOverridesWarning, ".controlPlaneComponents.controllerManager", "kube-controller-manager")
367+
}
368+
}
369+
if cluster.ControlPlaneComponents.Scheduler != nil {
370+
if cluster.ControlPlaneComponents.Scheduler.Flags != nil || cluster.ControlPlaneComponents.Scheduler.FeatureGates != nil {
371+
logger.Warnf(flagsAndFeatureGateOverridesWarning, ".controlPlaneComponents.scheduler", "kube-scheduler")
372+
}
373+
}
374+
if cluster.ControlPlaneComponents.APIServer != nil {
375+
if cluster.ControlPlaneComponents.APIServer.Flags != nil || cluster.ControlPlaneComponents.APIServer.FeatureGates != nil {
376+
logger.Warnf(flagsAndFeatureGateOverridesWarning, ".controlPlaneComponents.apiServer", "kube-apiserver")
377+
}
378+
}
379+
}
354380
}

‎pkg/apis/kubeone/types.go‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,35 @@ type KubeOneCluster struct {
9393
// TLSCipherSuites allows to configure TLS cipher suites for different components. See
9494
// https://pkg.go.dev/crypto/tls#pkg-constants for possible values.
9595
TLSCipherSuites TLSCipherSuites `json:"tlsCipherSuites"`
96+
97+
// ControlPlaneComponents configures the Kubernetes control plane components
98+
ControlPlaneComponents *ControlPlaneComponents `json:"controlPlaneComponents,omitempty"`
99+
}
100+
101+
type ControlPlaneComponents struct {
102+
// ControllerManagerConfig configures the Kubernetes Controller Manager
103+
ControllerManager *ControlPlaneComponentConfig `json:"controllerManager,omitempty"`
104+
105+
// Scheduler configures the Kubernetes Scheduler
106+
Scheduler *ControlPlaneComponentConfig `json:"scheduler,omitempty"`
107+
108+
// APIServer configures the Kubernetes API Server
109+
APIServer *ControlPlaneComponentConfig `json:"apiServer,omitempty"`
110+
}
111+
112+
type ControlPlaneComponentConfig struct {
113+
// Flags is a set of additional flags that will be passed to the control plane component.
114+
// KubeOne internally configures some flags that are eseeential for the cluster to work. Those flags set by KubeOne
115+
// will be merged with the ones specified in the configuration. In case of conflict the value provided by the user will be used.
116+
// Usage of `feature-gates` is not allowed here, use `FeatureGates` field instead.
117+
// IMPORTANT: Use of these flags is at the user's own risk, as KubeOne does not provide support for issues caused by invalid values and configurations.
118+
Flags map[string]string `json:"flags,omitempty"`
119+
120+
// FeatureGates is a map of additional feature gates that will be passed on to the control plane component.
121+
// KubeOne internally configures some feature gates that are eseeential for the cluster to work. Those feature gates set by KubeOne
122+
// will be merged with the ones specified in the configuration. In case of conflict the value provided by the user will be used.
123+
// IMPORTANT: Use of these featureGates is at the user's own risk, as KubeOne does not provide support for issues caused by invalid values and configurations.
124+
FeatureGates map[string]bool `json:"featureGates,omitempty"`
96125
}
97126

98127
type TLSCipherSuites struct {

‎pkg/apis/kubeone/v1beta1/zz_generated.conversion.go‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎pkg/apis/kubeone/v1beta2/types.go‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,35 @@ type KubeOneCluster struct {
9090
// TLSCipherSuites allows to configure TLS cipher suites for different components. See
9191
// https://pkg.go.dev/crypto/tls#pkg-constants for possible values.
9292
TLSCipherSuites TLSCipherSuites `json:"tlsCipherSuites"`
93+
94+
// ControlPlaneComponents configures the Kubernetes control plane components
95+
ControlPlaneComponents *ControlPlaneComponents `json:"controlPlaneComponents,omitempty"`
96+
}
97+
98+
type ControlPlaneComponents struct {
99+
// ControllerManagerConfig configures the Kubernetes Controller Manager
100+
ControllerManager *ControlPlaneComponentConfig `json:"controllerManager,omitempty"`
101+
102+
// Scheduler configures the Kubernetes Scheduler
103+
Scheduler *ControlPlaneComponentConfig `json:"scheduler,omitempty"`
104+
105+
// APIServer configures the Kubernetes API Server
106+
APIServer *ControlPlaneComponentConfig `json:"apiServer,omitempty"`
107+
}
108+
109+
type ControlPlaneComponentConfig struct {
110+
// Flags is a set of additional flags that will be passed to the control plane component.
111+
// KubeOne internally configures some flags that are eseeential for the cluster to work. Those flags set by KubeOne
112+
// will be merged with the ones specified in the configuration. In case of conflict the value provided by the user will be used.
113+
// Usage of `feature-gates` is not allowed here, use `FeatureGates` field instead.
114+
// IMPORTANT: Use of these flags is at the user's own risk, as KubeOne does not provide support for issues caused by invalid values and configurations.
115+
Flags map[string]string `json:"flags,omitempty"`
116+
117+
// FeatureGates is a map of additional feature gates that will be passed on to the control plane component.
118+
// KubeOne internally configures some feature gates that are eseeential for the cluster to work. Those feature gates set by KubeOne
119+
// will be merged with the ones specified in the configuration. In case of conflict the value provided by the user will be used.
120+
// IMPORTANT: Use of these featureGates is at the user's own risk, as KubeOne does not provide support for issues caused by invalid values and configurations.
121+
FeatureGates map[string]bool `json:"featureGates,omitempty"`
93122
}
94123

95124
type TLSCipherSuites struct {

‎pkg/apis/kubeone/v1beta2/zz_generated.conversion.go‎

Lines changed: 68 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)