enhancements/authentication: Pod Security Admission Enforcement #1747

ibihim · 2025-01-30T17:29:01Z

What

A proposal how to safely start enforcing.

Why

It isn't as trivial as we hoped for in 4.11 :)

openshift-ci · 2025-01-30T17:29:16Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign dustymabe for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

enhancements/authentication/pod-security-admission-enforcement.md

everettraven · 2025-02-03T20:39:32Z

enhancements/authentication/pod-security-admission-enforcement.md

+	// TargetModeConditional indicates that the user is willing to let the cluster
+	// automatically enforce a stricter enforcement once there are no violating Namespaces.
+	// If violations exist, the cluster stays in its previous state until those are resolved.
+	// This allows a gradual move towards label and global config enforcement without
+	// immediately breaking workloads that are not yet compliant.
+	TargetModeConditional PSATargetMode = "Conditional"


Is there a particular reason why we keep the cluster in the previous state? Is there another middle ground we can use?

The automatic enforcement thing is something that gives me pause and could be prone to some confusion. What if we had something like:

Privileged - no pod security restrictions

RestrictedExceptViolating - strictest possible enforcement, automatically exempting namespaces that would violate

Restricted - strictest possible configuration. Does not rollout if there are violating namespaces.

Also, do we already have an API mapping to the PodSecurity Admission Controller configuration? If not, would having an API surface that maps to that help us come up with something that isn't an all-or-nothing type transition?

The mapping to the PodSecurity Admission configuration is linked in the enhancement:

https://github.com/openshift/cluster-kube-apiserver-operator/blob/218530fdea4e89b93bc6e136d8b5d8c3beacdd51/pkg/cmd/render/render.go#L350-L358.

It is an interesting idea, but how would such a thing work?

I mean it would be pointless, if violating namespaces would be added dynamically to such a list, wouldn't it?
So would it be a list that is set initially and could only be reduced? What if there are like 100 namespaces out of 102 in there? Does it signal to the customer, that they are "enforcing restricted" (false sense of security)?

Giving the customer the capability to exclude NS upfront in a central place (through an API that maps the PodSecurity Configuration), would be an interesting concept, but it would mean that the kube-apiserver needs to roll-out a new version. It would be easier to label the particular namespace by hand with privileged.

The mapping to the PodSecurity Admission configuration is linked in the enhancement:

https://github.com/openshift/cluster-kube-apiserver-operator/blob/218530fdea4e89b93bc6e136d8b5d8c3beacdd51/pkg/cmd/render/render.go#L350-L358.

Ah, it looks like we don't allow for any configuration from the customer side to control the granularity of this beyond labeling the namespaces. I do wonder if it would make sense to provide the customer more nuanced control through an API that maps to this with some of our own defaulting logic? Why don't we do this today?

It is an interesting idea, but how would such a thing work?

Haven't gone into detailed thinking of how, but my general line of thinking is instead of all-or-nothing, something like RestrictedExceptViolating could set the default enforcement to restricted with some kind of configuration to not enforce that mode on Namespaces that would violate the restricted enforcement mode. Have to spend some more time thinking on what approach would be best, but I think that could be:

configures the exemptions.namespaces in the PodSecurity admission plugin configuration with violating namespaces

controller adds label to the violating namespaces to specify that the namespace should be evaluated with the privileged mode.

I mean it would be pointless, if violating namespaces would be added dynamically to such a list, wouldn't it?
So would it be a list that is set initially and could only be reduced? What if there are like 100 namespaces out of 102 in there?

From the motivation section of this enhancement, it sounded like we are pretty confident that the number of violating namespaces are generally pretty low. The scenario you point out is definitely something that could happen, but how likely is it?

Does it signal to the customer, that they are "enforcing restricted" (false sense of security)?

Maybe, but I would expect that we should still have some mechanism in place to bring violating namespaces to the attention of cluster administrators. Is it better to be "secure by default, except for these cases we have made you aware of" or "not secure at all"?

I think we discussed this in one of the recent Arch Calls, right?

It is nearly impossible for automated auditing tools to recognize. Which would make us incompatible with some auditing tools. And this is an important thing for us to avoid.

The OCP Arch Call - 4th of February 2025

I think was specifically related to the exemptions piece. I believe a labeling/annotation approach would be auditable because it would be making an update call to the kube-apiserver to update the namespace labels/annotations.

We could modify the enhancement, such that the psa label syncer enforces labels, where it works and doesn't where it does not.

Currently, I proposed that if not all of those namespaces would work, we don't proceed to label them at all and wait for those violations to be resolved.

everettraven

Did another pass focusing on the new API. Left a couple more ideas for how we may be able to get away from an "automated" transition thing.

The automated transitioning piece is still giving me pause and if we can come up with a solution that achieves the same goal with a fully declarative approach it might result in a better user and maintainer experience

everettraven · 2025-02-04T18:45:47Z

enhancements/authentication/pod-security-admission-enforcement.md

+In addition to adjusting how the `OpenShiftPodSecurityAdmission` `FeatureGate` behaves, administrators need visibility and control throughout this transition.
+A new API is necessary to provide this flexibility.
+
+#### New API


@sjenning Asked in slack which group this API would be added to. Would be good to include that information here.

According to the very first pitch in the Arch Call and @JoelSpeed's comment, I would put it into config/v1alpha1. I think this is the right package for configs that should be modifiable by users.

enhancements/authentication/pod-security-admission-enforcement.md

everettraven · 2025-02-04T18:53:45Z

enhancements/authentication/pod-security-admission-enforcement.md

+	// to "Restricted" on the kube-apiserver.
+	// This represents full enforcement, where both Namespace labels and the global config
+	// enforce Pod Security Admission restrictions.
+	EnforcmentModeFull PSAEnforcementMode = "FullEnforcement"


Is this just Restricted?

It is the "LabelEnforcement" mode, which tells the PSA label syncer to set the enforce label AND on top of that it modifies the PodSecurity configuration of the kube apiserver to restricted by default.

Should I revisit the comment above to make it more explicit?

enhancements/authentication/pod-security-admission-enforcement.md

everettraven · 2025-02-04T19:01:40Z

enhancements/authentication/pod-security-admission-enforcement.md

+	// in a fully privileged (pre-rollout) state, ignoring any label enforcement
+	// or global config changes.


Would this mode actually ignore label enforcement?

If I am following https://kubernetes.io/docs/tasks/configure-pod-container/enforce-standards-admission-controller/#configure-the-admission-controller correctly, it seems like the defaults only apply when there are no labels. Is Privileged effectively the same as disabling the PodSecurity admission plugin?

If TargetModePrivileged is set, based on what I tried to articulate, is that we effectively disable any restrictions, which means it should behave as it does in all cluster before.

In detail, we could argue about how we would like to do it. I would just revert everything, like removing the enforce label set by the PSA label syncer and set the global config back to privileged.
Logically (based on the name) more correct would be to set the enforce label to privileged. But as mentioned above, I would like to keep the permutations of possible state as minimal as possible.

I'm trying to understand the expected behavior from a user perspective. If I, as a user, were to specify Privileged as the target mode, what happens? If in this state I add a PSa label to a namespace to enforce a different mode like restricted or baseline for that namespace, will it be enforced?

I think there is a distinct difference between "I don't want Pod Security at all" and "I want the most privileged Pod Security default, but still respect anything that would override the default" and I'm trying to make sure I understand which one of those this value is supposed to represent.

This value currently represents, currently as in this enhancement, the intend to "I don't want Pod Security at all".
From a user perspective it might be more logical to have it actually set to "I want the most privileged Pod Security default"...

... until something goes wrong and they want "the previous state, before it started to fall apart".

I am fine both ways.

For the "I don't want Pod Security at all", maybe something like None is more reflective of that and in that case should disable the Pod Security admission plugin all together?

everettraven · 2025-02-04T19:12:51Z

enhancements/authentication/pod-security-admission-enforcement.md

+	TargetModeConditional PSATargetMode = "Conditional"
+
+	// TargetModeRestricted indicates that the user wants the strictest possible
+	// enforcement, causing the cluster to ignore any existing violations and


Instead of ignoring existing violations, should the cluster operator go degraded with any existing violations being the reason?

Maybe a knob for users to control this would be useful here to support use cases of:

Forcefully enable restricted PSA configuration

Enable restricted PSA configuration only if no violations (I realize this is what the Conditional enum type is for, but maybe that shouldn't be a separate option?)

Note: I elaborated a bit more on my thinking here in #1747 (comment)

enhancements/authentication/pod-security-admission-enforcement.md

everettraven · 2025-02-04T19:28:56Z

enhancements/authentication/pod-security-admission-enforcement.md

+// PSATargetMode reflects the user’s chosen (“target”) enforcement level.
+type PSATargetMode string
+
+const (


Should there be an option that is reflective of the LabelEnforcement enforcement mode that would be specified in the status?

We could, but I mentioned it in the Non-Goals.

The reason is, that I want to keep the permutations of possible state as minimal as possible.

I don't know that this question relates to the non goals you outlined. What target mode(s) might result in the LabelEnforcement enforcement mode being populated in the status?

It is part of this non-goal.

I am not eager to maintain too many states. Either you are privileged, restricted or on the way progressing forward through "LabelOnly" to "FullEnforcement".

everettraven · 2025-02-04T19:30:30Z

enhancements/authentication/pod-security-admission-enforcement.md

+type PSAEnforcementConfigSpec struct {
+	// targetMode is the user-selected Pod Security Admission enforcement level.
+	// Valid values are:
+	//   - "Privileged": ensures the cluster runs with no restrictions
+	//   - "Conditional": defers the decision to cluster-based evaluation
+	//   - "Restricted": enforces the strictest Pod Security admission
+	//
+	// If this field is not set, it defaults to "Conditional".
+	//
+	// +kubebuilder:default=Conditional
+	TargetMode PSATargetMode `json:"targetMode"`
+}


Continuing from my comment on the Restricted target mode, maybe instead of only being able to specify a single enum value this should be a discriminated union that enables more flexibility in configuration?

To further show my line of thinking:

To represent the Privileged option a user would still do:

targetMode: Privileged

To represent a forceful enablement of Restricted:

targetMode: Restricted restricted: violationPolicy: Ignore # room for future configuration options

To represent an enablement of Restricted only if no violations (would go to degraded on failure to enable):

targetMode: Restricted restricted: violationPolicy: Block

To facilitate the transition case that the proposed Conditional targetMode seems to be targeting, maybe the default can be something like:

targetMode: Restricted restricted: violationPolicy: LabelMinimallyRestrictive

Where any violating namespaces encountered will be labeled with the appropriately minimally restrictive PSS mode (which might be privileged), prior to attempting to set the default enforced by the PodSecurity admission controller to Restricted.

It is slightly more expressive. Maybe something like:

targetMode: level: restricted violationPolicy: Block

or

targetMode: restricted violationPolicy: Block

The default could be Block then, which would also work for targetMode: privileged 😄

With labelMinimallyRestrictive... the label syncer isn't able to determine this yet and it changes its view point. Currently it determines the future outlook based on the SA-based RBAC. Now it would need to have another level of concern: testing individual workloads against up to 2 levels (do you satisfy restricted, do you satisfy privileged?).

enhancements/authentication/pod-security-admission-enforcement.md

JoelSpeed

A lot of my questions probably result from a lack of knowledge about PSA and how it actually works/what configurations exist today. Where can I learn more about the various options for it's configuration so that I can better review this EP?

enhancements/authentication/pod-security-admission-enforcement.md

JoelSpeed · 2025-02-18T16:35:52Z

enhancements/authentication/pod-security-admission-enforcement.md

+This enhancement introduces a **new cluster-scoped API** and changes to the relevant controllers to rollout [Pod Security Admission (PSA)](https://kubernetes.io/docs/concepts/security/pod-security-admission/) enforcement [in OpenShift](https://www.redhat.com/en/blog/pod-security-admission-in-openshift-4.11).
+Enforcement means that the `PodSecurityAdmissionLabelSynchronizationController` sets the `pod-security.kubernetes.io/enforce` label on Namespaces, and the PodSecurityAdmission plugin enforces the `Restricted` [Pod Security Standard (PSS)](https://kubernetes.io/docs/concepts/security/pod-security-standards/) globally on Namespaces without any label.
+
+The new API allows users to either enforce the `Restricted` PSS or maintain `Privileged` PSS for several releases. Eventually, all clusters will be required to use `Restricted` PSS.


Eventually, all clusters will be required to use Restricted PSS.

How and why? There will be no get out of jail card at all eventually? Why is this not a cluster admins choice?

Secure by default? 😄

We would like to nudge our users to move to the most secure setup at some point. @deads2k suggested that there could be a period, where we tolerate being Privileged and start blocking upgrades after x-releases.

There is no reason to run with a Privileged global setup. A user can set namespaces individually to match the required PSS, if not properly labeled by the PodSecurityAdmissionLabelSynchronizationController (which should only happen in the case that the SCCs are user-based).

I will add a section into Motivation.

https://github.com/openshift/enhancements/pull/1747/files#diff-e48dd33b3f59b545b505293e5c81d96a29a2e9cc994f4527b46bde63a2def9d8R48-R50

While I understand the motivation of "secure by default", I don't think we should totally prevent a cluster admin from choosing the security profile they desire for their cluster.

I think it is totally reasonable for us to have an opinionated and strongly encouraged default.

JoelSpeed · 2025-02-18T16:37:49Z

enhancements/authentication/pod-security-admission-enforcement.md

+Although these numbers are now quite low, it is essential to avoid any scenario where users end up with failing workloads.
+
+To ensure a safe transition, this proposal suggests that if a potential failure of workloads is being detected in release `n`, that the operator moves into `Upgradeable=false`.
+The user would need to either resolve the potential failures or set the enforcing mode to `Privileged` for now in order to be able to upgrade.


Or perhaps they label the violating namespaces as privileged and allow the rest of the cluster to be restricted?

Yes, I will add this option.

enhancements/authentication/pod-security-admission-enforcement.md

JoelSpeed · 2025-02-18T17:35:14Z

enhancements/authentication/pod-security-admission-enforcement.md

+If a user encounters `status.violatingNamespaces` it is expected to:
+
+- resolve the violating Namespaces in order to be able to `Upgrade` or
+- set the `spec.enforcementMode=Privileged` and solve the violating Namespaces later.


When a user defers this, how do we know that they will fix it later? What guidance can we provide them?

Yes, we need provide guidance to the customer, the question would be what is the most effective way? Blog entries? This enhancement?

https://github.com/openshift/enhancements/pull/1747/files#diff-e48dd33b3f59b545b505293e5c81d96a29a2e9cc994f4527b46bde63a2def9d8R354-R398

I would seek the guidance of the appropriate technical writer. I could imagine a combination of release notes and product docs/KCS

JoelSpeed · 2025-02-18T17:39:15Z

enhancements/authentication/pod-security-admission-enforcement.md

+#### PodSecurityAdmissionLabelSynchronizationController
+
+The [PodSecurityAdmissionLabelSynchronizationController (PSA label syncer)](https://github.com/openshift/cluster-policy-controller/blob/master/pkg/psalabelsyncer/podsecurity_label_sync_controller.go) must watch the `status.enforcementMode` and the `OpenShiftPodSecurityAdmission` `FeatureGate`.
+If `spec.enforcementMode` is `Restricted` and the `FeatureGate` `OpenShiftPodSecurityAdmission` is enabled, the syncer will set the `pod-security.kubernetes.io/enforce` label.


This label goes on each namespace? Or?

On each namespace, except it is:

a run-level 0 namespace (default, kube-system,...), an

openshift pre-fixed namespace or

a namespace which disabled the PSA label syncer.

I will list those Namespaces.

Clarifying this behaviour briefly in the EP would be useful for readers IMO

JoelSpeed · 2025-02-18T17:40:08Z

enhancements/authentication/pod-security-admission-enforcement.md

+
+### Fresh Installs
+
+Needs to be evaluated. The System Administrator needs to pre-configure the new API’s `spec.enforcementMode`, choosing whether the cluster will be `Privileged` or `Restricted` during a fresh install.


Why not have the installer generate the correct (Restricted) configuration to be installed? Then most people won't care, but if someone really wanted to, they could pause at the manifests stage and edit the resource to Privileged instead

I think, if the default will be Restricted, it would default to enforcement, right? The question that is left open for me is: if a customer wants to default to Privileged on fresh installs, is there an option to do it? Is there some documentation wrt? Or is it common to let the SysAdmin change that config post-installation when setting up a cluster?

They have this ability already generally. Lets say we did nothing here, then the cluster installs, the resource gets generated and the enforcement field is empty. They can now go set that to privileged and continue with their cluster until we force them backwards.

Or, we can enable them to change this at install time. Which comes in two forms.

We add an explicit option to the install config, and plumb the value through.

Or, we generate the manifests and document that a modified manifest at the manifests stage could be used to change the behaviour for fresh clusters.

The first of the install time options requires the second to be implemented anyway.

Setting it from the installer in this manner will allow us then to have a distinction between clusters that were installed with PSA restricted, vs those that upgraded and had no opinion. I suspect there is some value in that 🤔

enhancements/authentication/pod-security-admission-enforcement.md

ibihim · 2025-02-20T10:49:15Z

enhancements/authentication/pod-security-admission-enforcement.md

+
+While the root causes need to be identified in some cases, the result of identifying a violating Namespace is understood.
+
+#### New SCC Annotation: `security.openshift.io/ValidatedSCCSubjectType`


TODO: fix casing to match change

- List which namespaces aren't managed by the PSA label syncer - Explain, why we want to force restricted PSA enforcement eventually. - Add a guide on how to handle violations.

ibihim · 2025-02-24T08:10:03Z

A lot of my questions probably result from a lack of knowledge about PSA and how it actually works/what configurations exist today. Where can I learn more about the various options for it's configuration so that I can better review this EP?

@JoelSpeed, I tried to set as many links as possible. Especially at the beginning, but it doesn't seem to be obvious as you are not the first to ask 😄

Upstream resources:

High Level Description of Pod Security Admission and
the definition of the Pod Security Standards.

Downstream resources:

Initial blog post from the authors of the PSA label syncer controller: blog post.
Integrating PSA into OpenShift: enhancement.
The enhancement for the PSA label syncer: enhancement.
OpenShift Docs for the user perspective: docs.
Some other Red Hat labeled blog post wrt PSA and SCCs: blog.

PSA label syncer vs Global Config, while the earlier being a subset.

enhancements/authentication/pod-security-admission-enforcement.md

xingxingxia · 2025-02-25T09:32:32Z

enhancements/authentication/pod-security-admission-enforcement.md

+- Are prefixed with `openshift`,
+- Have the label `security.openshift.io/scc.podSecurityLabelSync=false`.
+- Have the `pod-security.kubernetes.io/enforce` label set manually.
+- Are not a run-level zero Namespace:


Suggested change

- Are not a run-level zero Namespace:

- Are a run-level zero Namespace:

Actually the label syncer doesn't ignore run-level zero (or one) namespaces explicitly; it just happens that the ones listed here are run-level 0.

Having said that, admission plugins (and also PSa) are not enabled for runlevel 0/1 namespaces, so any labels will be ineffective.

I will correct the whole list to reflect this: https://github.com/openshift/enhancements/pull/1747/files#r1971301549

everettraven · 2025-02-25T12:53:57Z

enhancements/authentication/pod-security-admission-enforcement.md

+
+To ensure a safe transition, this proposal suggests that if a potential failure of workloads is being detected in release `n`, that the operator moves into `Upgradeable=false`.
+The user would need to either resolve the potential failures, setting a higher PSS label for that Namespace or set the enforcing mode to `Privileged` for now in order to be able to upgrade.
+`Privileged` will keep the cluster in the previous state, the non enforcing state.


Might be worth elaborating on what the "non enforcing state" entails. Does this mean the default global enforcement is privileged but cluster admins can make certain namespaces more restrictive? Or does it mean that there is no pod security enforcement at all (i.e PodSecurity admission plugin disabled)?

everettraven · 2025-02-25T13:00:44Z

enhancements/authentication/pod-security-admission-enforcement.md

+The temporary `Privileged` mode (opt-out from PSA enforcement) exists solely to facilitate a smooth transition. Once a vast majority of clusters have adapted their workloads to operate under `Restricted` PSS, maintaining the option to run in `Privileged` mode would undermine these security objectives.
+
+OpenShift strives to offer the highest security standards. Enforcing PSS ensures that OpenShift is at least as secure as upstream Kubernetes and that OpenShift complies with upstreams security best practices.


Doesn't upstream Kubernetes ship with privileged global enforcement by default? Even though it is strongly recommended to set a more restrictive policy, ultimately it's up to the cluster administrator.

Are there any use cases that customers may have that could warrant an option to set the cluster to a less restrictive global state than Restricted? Maybe there is a middle ground that allows cluster administrators to do this, but they explicitly acknowledge that they are moving to an explicitly unsupported state?

Something else to think about purely from an API change perspective - removing an allowed value for a field is going to be a breaking change to users and will likely require an explicit admin action when we want to remove it as an allowed value.

enhancements/authentication/pod-security-admission-enforcement.md

everettraven · 2025-02-25T13:39:29Z

enhancements/authentication/pod-security-admission-enforcement.md

+### New API
+
+This API is used to support users to enforce PSA.
+As this is a transitory process, this API will loose its usefulness once PSA enforcement isn't optional anymore.


Similar to previous comments I've made, I think it might be worth keeping this API around to allow cluster admins flexibility in how they configure the PodSecurity admission plugin. I think it is fine for us to have opinionated and encouraged defaults.

Have we thought through different use cases that admins may have and why they may or may not want to be able to configure the global enforcement mode?

Additionally, I think the introduction of an API means we have to support it. If we don't want to support a new API long term, is there an alternative approach that does not use an API?

David's suggestion is to use LegacyConfig, which would infer that the field is introduced and deprecated immediately, and can then go through the deprecation cycle

I can't imagine a use-case, where someone wouldn't want to have the highest possible security setting on their production cluster.
If I would imagine that I have a test-cluster, this could be annoying to deal with, if the legacyMode is gone. But it isn't like I could not add a mechanism to auto-label namespaces with another PSS.

everettraven · 2025-02-25T13:46:03Z

enhancements/authentication/pod-security-admission-enforcement.md

+
+#### PodSecurityAdmissionLabelSynchronizationController
+
+The [PodSecurityAdmissionLabelSynchronizationController (PSA label syncer)](https://github.com/openshift/cluster-policy-controller/blob/master/pkg/psalabelsyncer/podsecurity_label_sync_controller.go) must watch the `status.enforcementMode` and the `OpenShiftPodSecurityAdmission` `FeatureGate`.


Suggested change

The [PodSecurityAdmissionLabelSynchronizationController (PSA label syncer)](https://github.com/openshift/cluster-policy-controller/blob/master/pkg/psalabelsyncer/podsecurity_label_sync_controller.go) must watch the `status.enforcementMode` and the `OpenShiftPodSecurityAdmission` `FeatureGate`.

The [PodSecurityAdmissionLabelSynchronizationController (PSA label syncer)](https://github.com/openshift/cluster-policy-controller/blob/master/pkg/psalabelsyncer/podsecurity_label_sync_controller.go) must watch the `spec.enforcementMode` and the `OpenShiftPodSecurityAdmission` `FeatureGate`.

?

Well, I would assume, based on the current state of the enhancement, that spec indicates the intend and status is its actual state aka green-lighting of the intend. So the psa label syncer would not care what the intend is, but what the PodSecurityReadinessController signals in the status.

In the Upgradeable=false scenario, this shouldn't be the case as the cluster wouldn't upgrade, if the spec doesn't reflect the doable state and in the auto-legacy-mode-switching scenario, the "status" would be the spec and vice versa...

So I think you are correct.

openshift-bot · 2025-04-25T00:15:34Z

Rotten enhancement proposals close after 7d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Reopen the proposal by commenting /reopen.
Mark the proposal as fresh by commenting /remove-lifecycle rotten.
Exclude this proposal from closing again by commenting /lifecycle frozen.

/close

openshift-ci · 2025-04-25T00:15:56Z

@openshift-bot: Closed this PR.

In response to this:

Rotten enhancement proposals close after 7d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Reopen the proposal by commenting /reopen.
Mark the proposal as fresh by commenting /remove-lifecycle rotten.
Exclude this proposal from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ibihim · 2025-05-05T11:55:10Z

/reopen
remove-lifecycle rotten

openshift-ci · 2025-05-05T11:55:19Z

@ibihim: Reopened this PR.

In response to this:

/reopen
remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ibihim · 2025-05-05T11:55:49Z

/remove-lifecycle rotten

…ry tests

ibihim · 2025-05-12T13:17:39Z

enhancements/authentication/pod-security-admission-enforcement.md

+
+## Open Questions
+
+## Rerun the evaluation before enforcing


It sounds reasonable to add alerts in release n and run the actual final evaluation right before potentially enforcing as between release n and release n+1 could be a gap of months.

openshift-ci · 2025-05-12T13:47:37Z

@ibihim: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/markdownlint	`7aad97c`	link	true	`/test markdownlint`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

JoelSpeed · 2025-05-15T16:43:28Z

enhancements/authentication/pod-security-admission-enforcement.md

+// PSAEnforcementConfig is a config that supports the user in the PSA enforcement transition.
+// The spec struct enables a user to stop the PSA enforcement, if necessary.
+// The status struct supports the user in identifying obstacles in PSA enforcement.
+type PSAEnforcementConfig struct {


My understanding was to create a top level LegacyConfig that could be used again in the future for other similar scenarios. I wasn't party to any conversation where that was reversed at least

JoelSpeed · 2025-05-15T16:45:30Z

enhancements/authentication/pod-security-admission-enforcement.md

+// PSAEnforcementConfigSpec is a configuration option that enables the customer to dictate the PSA enforcement outcome.
+type PSAEnforcementConfigSpec struct {
+	// enforcementMode gives the user different options:
+	// - "" will be interpreted as "Restricted".


In release N, this API doesn't actually do anything. We would want to reword this to be more like

Valid values are Restricted, Legacy and omitted. Omitted means no opinion and the platform is left to choose a reasonable default, which is subject to change over time. The current default is Legacy.

Then in the release where we enforce restricted, we would update that doc

JoelSpeed · 2025-05-15T16:46:24Z

enhancements/authentication/pod-security-admission-enforcement.md

+	// taken place, which is particularly important during upgrades to ensure the
+	// controller had an opportunity to check for violations before enforcing PSA.
+	// +optional
+	LastEvaluationTime metav1.Time `json:"lastEvaluationTime,omitempty"`


This will tick, and cause regular updates. Do we need to keep a timestamp? Or can we use a condition like LastTransitionTime kind of thing?

Perhaps we want to track the last time we saw a new violating namespace?

JoelSpeed · 2025-05-15T16:48:25Z

enhancements/authentication/pod-security-admission-enforcement.md

+If a user encounters `status.violatingNamespaces` it is expected to:
+
+- resolve the violations in the Namespaces to be able to safely move to `Restricted` or
+- set the `spec.enforcementMode=Legacy` and solve the violating Namespaces later.


We aren't doing this automatically for them?

JoelSpeed · 2025-05-15T16:51:35Z

enhancements/authentication/pod-security-admission-enforcement.md

+
+## Rerun the evaluation before enforcing
+
+It could be better to run the evaluation before enforcing `Restricted` mode as the last run can be a couple of months old.


Why would it be a couple of months old?

JoelSpeed · 2025-05-16T12:47:02Z

enhancements/authentication/pod-security-admission-enforcement.md

+- Release `n-1`:
+  - Backport the API.
+- Release `n`:
+  - Enable the `PodSecurityReadinessController` to use the API by setting it's `status` and if violating Namespaces are found also `spec.enforcementMode = Legacy`.


Do these need to be separate? What's the benefit of having the API with no consumer in the previous release?

openshift-bot · 2025-06-14T01:15:47Z

Inactive enhancement proposals go stale after 28d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Mark the proposal as fresh by commenting /remove-lifecycle stale.
Stale proposals rot after an additional 7d of inactivity and eventually close.
Exclude this proposal from closing by commenting /lifecycle frozen.

If this proposal is safe to close now please do so with /close.

/lifecycle stale

openshift-bot · 2025-06-21T08:45:30Z

Stale enhancement proposals rot after 7d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Mark the proposal as fresh by commenting /remove-lifecycle rotten.
Rotten proposals close after an additional 7d of inactivity.
Exclude this proposal from closing by commenting /lifecycle frozen.

If this proposal is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

openshift-bot · 2025-06-29T00:15:18Z

Rotten enhancement proposals close after 7d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Reopen the proposal by commenting /reopen.
Mark the proposal as fresh by commenting /remove-lifecycle rotten.
Exclude this proposal from closing again by commenting /lifecycle frozen.

/close

openshift-ci · 2025-06-29T00:15:28Z

@openshift-bot: Closed this PR.

In response to this:

Rotten enhancement proposals close after 7d of inactivity.

See https://github.com/openshift/enhancements#life-cycle for details.

Reopen the proposal by commenting /reopen.
Mark the proposal as fresh by commenting /remove-lifecycle rotten.
Exclude this proposal from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 30, 2025

openshift-ci bot requested a review from jan--f January 30, 2025 17:29

openshift-ci bot requested a review from jwmatthews January 30, 2025 17:29

ibihim force-pushed the ibihim/psa-enforcement branch from 12f022d to f95e7a9 Compare February 3, 2025 15:09

ibihim changed the title ~~[WIP] enhancements/authentication: Pod Security Admission Enforcement Config~~ enhancements/authentication: Pod Security Admission Enforcement Config Feb 3, 2025

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 3, 2025

ibihim changed the title ~~enhancements/authentication: Pod Security Admission Enforcement Config~~ enhancements/authentication: Pod Security Admission Enforcement Feb 3, 2025

everettraven reviewed Feb 3, 2025

View reviewed changes

everettraven reviewed Feb 4, 2025

View reviewed changes

kramaranya reviewed Feb 5, 2025

View reviewed changes

ibihim commented Feb 13, 2025

View reviewed changes

enhancements/authentication/pod-security-admission-enforcement.md Outdated Show resolved Hide resolved

enhancements/authentication: psa enforcment

930c8a4

ibihim force-pushed the ibihim/psa-enforcement branch from 5d1a078 to 930c8a4 Compare February 17, 2025 10:44

ibihim added 2 commits February 17, 2025 15:10

enhancements/auth: modified until Design Details, before big removal

cee6407

enhancements/authentication: trim down psa-enforcement to modified API

2aabd1b

JoelSpeed reviewed Feb 18, 2025

View reviewed changes

ibihim commented Feb 20, 2025

View reviewed changes

enhancements/authentication: add explanations

9614901

- List which namespaces aren't managed by the PSA label syncer - Explain, why we want to force restricted PSA enforcement eventually. - Add a guide on how to handle violations.

ibihim added 2 commits February 24, 2025 09:14

enhancements/authentication: fix indent / heading

b210bae

enhancements/authentication: add race condition concern

2e417e4

PSA label syncer vs Global Config, while the earlier being a subset.

JoelSpeed reviewed Feb 24, 2025

View reviewed changes

enhancements/authentication/pod-security-admission-enforcement.md Show resolved Hide resolved

ibihim mentioned this pull request Feb 25, 2025

security: add annotation for validated SCC type openshift/api#2211

Merged

xingxingxia reviewed Feb 25, 2025

View reviewed changes

everettraven reviewed Feb 25, 2025

View reviewed changes

enhancements/authentication/pod-security-admission-enforcement.md Show resolved Hide resolved

everettraven reviewed Feb 25, 2025

View reviewed changes

openshift-ci bot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Apr 17, 2025

openshift-ci bot closed this Apr 25, 2025

openshift-ci bot reopened this May 5, 2025

openshift-ci bot removed the lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. label May 5, 2025

ibihim force-pushed the ibihim/psa-enforcement branch 6 times, most recently from c935c2f to 9421702 Compare May 9, 2025 17:11

authentication: rm progressing, improve phrasing

f3e0563

ibihim force-pushed the ibihim/psa-enforcement branch from 9421702 to f3e0563 Compare May 9, 2025 17:17

authentication: add alerts, suggest final evaluation, specify necessa…

7aad97c

…ry tests

ibihim commented May 12, 2025

View reviewed changes

JoelSpeed reviewed May 16, 2025

View reviewed changes

openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jun 14, 2025

openshift-ci bot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jun 21, 2025

openshift-ci bot closed this Jun 29, 2025

		// in a fully privileged (pre-rollout) state, ignoring any label enforcement
		// or global config changes.


		### Fresh Installs

		Needs to be evaluated. The System Administrator needs to pre-configure the new API’s `spec.enforcementMode`, choosing whether the cluster will be `Privileged` or `Restricted` during a fresh install.


		While the root causes need to be identified in some cases, the result of identifying a violating Namespace is understood.

		#### New SCC Annotation: `security.openshift.io/ValidatedSCCSubjectType`

	- Are not a run-level zero Namespace:
	- Are a run-level zero Namespace:

		The temporary `Privileged` mode (opt-out from PSA enforcement) exists solely to facilitate a smooth transition. Once a vast majority of clusters have adapted their workloads to operate under `Restricted` PSS, maintaining the option to run in `Privileged` mode would undermine these security objectives.

		OpenShift strives to offer the highest security standards. Enforcing PSS ensures that OpenShift is at least as secure as upstream Kubernetes and that OpenShift complies with upstreams security best practices.


		#### PodSecurityAdmissionLabelSynchronizationController

		The [PodSecurityAdmissionLabelSynchronizationController (PSA label syncer)](https://github.com/openshift/cluster-policy-controller/blob/master/pkg/psalabelsyncer/podsecurity_label_sync_controller.go) must watch the `status.enforcementMode` and the `OpenShiftPodSecurityAdmission` `FeatureGate`.


		## Rerun the evaluation before enforcing

		It could be better to run the evaluation before enforcing `Restricted` mode as the last run can be a couple of months old.

enhancements/authentication: Pod Security Admission Enforcement #1747

enhancements/authentication: Pod Security Admission Enforcement #1747

Uh oh!

Conversation

ibihim commented Jan 30, 2025

What

Why

Uh oh!

openshift-ci bot commented Jan 30, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ibihim Feb 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ibihim Feb 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

everettraven left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ibihim Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

everettraven Feb 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ibihim Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ibihim Feb 4, 2025 •

edited

Loading

ibihim Feb 4, 2025 •

edited

Loading

ibihim Feb 13, 2025 •

edited

Loading

everettraven Feb 4, 2025 •

edited

Loading

ibihim Feb 13, 2025 •

edited

Loading

ibihim Feb 20, 2025 •

edited

Loading