Skip to content

OCPNODE-3029: user namespaces: add required minimum kubelet version #1766

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

OCPNODE-3029: user namespaces: add required minimum kubelet version #1766

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions enhancements/kubelet/user-namespaces-support.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,13 @@ type NodeSpec struct {
// +optional
MinimumKubeletVersion string `json:"minimumKubeletVersion,omitempty"`
}
type NodeStatus struct {
...
// MinimumKubeletVersion is the lowest version of a kubelet that can join the cluster
// +kubebuilder:validation:Pattern=`^([0-9]*\.[0-9]*\.[0-9]*$`
// +optional
MinimumKubeletVersion string `json:"minimumKubeletVersion,omitempty"`
}
```

##### MinimumKubeletVerison authorization plugin
Expand All @@ -373,6 +380,71 @@ that are running with a version lower than the configured version. If so, the cr

MCO will read the MinimumKubeletVersion and mark machines as degraded if the node is not at least MinimumKubeletVersion.

##### RequiredMinimumComponentVersion

The openshift featuregate API is the way to configure featuregates in a cluster. We will plumb awareness of the minimum kubelet version to the featuregate API by adding
a RequiredMinimumComponentVersion to the FeatureGateAttributes structure:

```
type FeatureGateAttributes struct {
// name is the name of the FeatureGate.
// +required
Name FeatureGateName `json:"name"`

// requiredMinimumComponentVersion is a list of component/version pairs that declares the is the lowest version the given
// component may be in this cluster.
// Currently, the only supported component is Kubelet, and setting a required minimum kubelet component will set the
// minimumKubeletVersion field in the nodes.config.openshift.io CRD.
// +kubebuilder:validation:MaxItems:=1
// +listType=map
// +listMapKey=component
// +openshift:enable:FeatureGate=MinimumKubeletVersion
// +optional
RequiredMinimumComponentVersions []RequiredMinimumComponentVersion `json:"requiredMinimumComponentVersions,omitempty"`

// possible (probable?) future additions include
// 1. support level (Stable, ServiceDeliveryOnly, TechPreview, DevPreview)
// 2. description
}

// RequiredMinimumComponentVersion is a pair of Component and Version that specifies the required minimum Version of the given Component
// to enable this feature.
type RequiredMinimumComponentVersion struct {
// component is the entity whose version must be above a certain version.
// +required
Component RequiredMinimumComponent `json:"component"`
// version is the minimum version the given component may be in this cluster.
// +kubebuilder:validation:XValidation:rule="self.matches('^[0-9]*.[0-9]*.[0-9]*$')",message="minmumKubeletVersion must be in a semver compatible format of x.y.z, or empty"
// +kubebuilder:validation:MaxLength:=8
// +required
Version string `json:"version"`
}

// +kubebuilder:validation:Enum:=Kubelet
type RequiredMinimumComponent string

var RequiredMinimumComponentKubelet RequiredMinimumComponent = "Kubelet"
```

Then, the featuregate controller in the cluster-config-operator will be extended to filter features from the Enabled set if the minimum kubelet version isn't set old enough.
If multiple feature gates have minimum versions set, only those that are higher than the defined minimum version will be filtered. If feature A has required minimum version
1.0.0 and feature B has required minimum version 1.1.0, and the cluster has minimum version set to 1.0.0, the resulting featureset would have feature A enabled, but B disabled.

This will give openshift feature developers a mechansim to programatically declare a feature requires a certain version of the kubelet. This will be used for user namespaces,
and the corresponding version for user namespaces and related feature gates will be "1.30.0"

###### Downgrades to minimumKubeletVersion

Given we interpret the minimum kubelet version and render a featureset given it, we need to handle rollbacks of the minimum kubelet version.
Assume the nodes.config.Spec.MinimumKubeletVersion == 1.29 (new intent) and the nodes.config.Status.MinimumKubeletVersion == 1.30 (old intent).
The workflow will look something like:
* cluster-config-operator will choose min of Spec and Status MinimumKubeletVersion field, and render the featuregates given that (1.29). It will imbed the RenderedMinimumComponentVersion in the featuregate object.
* MCO kubelet config controller will see the nodes.config.Status.MinimumKubeletVersion does not match the RenderedMinimumComponentVersion in the featuregate object, and will wait for one node to rollout
in each MCP. Once that happens, it will update the nodes.Config.status.MinimumKubeletVersion
* Node authorizer will act on the status field, not the spec field, thus allowing a safe transition to the lower minimum kubelet version

Hypershift will follow a similar track, but instead of MCO it will be the nodepool controller, and it will all be handled as updates to the HCCO.

##### Alternatives to MinimumKubeletVersion

It is also possible this feature should be paired with a corresponding kubelet field `minimumKubeletVersion`, where it exits if it is too old. This will prevent the kubelet from
Expand Down