Define KEP for supporting modern tls settings #8246

kannon92 · 2025-12-15T16:12:32Z

What type of PR is this?

/kind feature
/kind documentation

What this PR does / why we need it:

Which issue(s) this PR fixes:

KEP for #8190

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

netlify · 2025-12-15T16:12:39Z

✅ Deploy Preview for kubernetes-sigs-kueue ready!

Name	Link
🔨 Latest commit	`d39e134`
🔍 Latest deploy log	https://app.netlify.com/projects/kubernetes-sigs-kueue/deploys/695fe7c58f691900085a1354
😎 Deploy Preview	https://deploy-preview-8246--kubernetes-sigs-kueue.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

mimowo · 2025-12-16T07:51:19Z

ack, I will try to give an initial pass this week. Otherwise after returning in January. Maybe @tenzen-y will be able to review in the meanwhile

mimowo · 2025-12-16T15:13:49Z

keps/8190-tls-security-profiles/README.md

+	// CurveP256 represents the P-256 elliptic curve
+	CurveP256 CurveID = "CurveP256"
+	// CurveP384 represents the P-384 elliptic curve
+	CurveP384 CurveID = "CurveP384"
+	// CurveP521 represents the P-521 elliptic curve
+	CurveP521 CurveID = "CurveP521"
+	// X25519 represents the X25519 elliptic curve
+	CurveX25519 CurveID = "X25519"


Is it constraining the options to only those listed here? Are we sure there will be no other new options?

Looking at the TLS 1.3 RFC, curve488 is missing for TLS 1.3 at least, but that's not a comprehensive list (this is here IIUC).

https://github.com/golang/go/blob/release-branch.go1.25/src/crypto/tls/common.go#L145

If there are new options we can add new enum fields. They should only change when we update golang so I don't think this is too burdensome.

Will we need to re-declare the constants in our code, or we could use the exposed constants?

They should only change when we update golang so I don't think this is too burdensome.

Yes, but we may forget to do so, making sure we remember is another step in the process. So, it would be better avoiding listing of all options in our code (if feasible).

So we could with the alternative and maybe just skip validation of this and let the server fail if someone specifies a wrong value.

mimowo · 2025-12-16T15:15:40Z

keps/8190-tls-security-profiles/README.md

+	// TLS 1.3 cipher suites
+	CipherSuiteTLSAES128GCMSHA256       CipherSuite = "TLS_AES_128_GCM_SHA256"
+	CipherSuiteTLSAES256GCMSHA384       CipherSuite = "TLS_AES_256_GCM_SHA384"
+	CipherSuiteTLSCHACHA20POLY1305SHA256 CipherSuite = "TLS_CHACHA20_POLY1305_SHA256"


Also, is it closed world, or open-world set? If closed, because I see // +kubebuilder:validation:Enum=TLS_RSA_WITH_AES_128_CBC_SHA;TLS_RSA_WITH_AES_256_CBC_SHA;... then how sure we are no new options will be added. It feels better to keep it open world, but I'm not totally sure what are the policies here.

Is there something analogous in the core k8s we could use as example?

I see some examples in our component-base / apiserver.

https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiserver/pkg/server/config.go#L346

I see mostly configs accepting a raw string and then doing the validation in the code.

I was thinking it would be better to provide a list of allowed values.

We could switch to providing strings and validate.

I added this as an option in alternatives.

There's also some opinionatedness in the Go stl. For example, for TLS 1.3 there are five "official" cipher suites but Go supports three (with an ongoing discussion to add support for the CCM ones at golang/go#27484).

One thing that I noticed is that our enum lists some ciphers which are listed as "insecure" in the Go crypto library. For example TLS_RSA_WITH_AES_128_GCM_SHA256 is here in the InsecureCipherSuites list.

@kannon92 Do you remember what was the source of the list of ciphers on our end? The 1.3 ones are the same, but we have the four extra TLS_RSA_WITH... in our enum for 1.2.

Yea for TLS 1.3 there is actually no configuration allowed AFAIK.

The issue I got from the field is that there are still lots of folks that use their own ciphersuites so I think 1.2 configuration is still important.

What about the 4 insecure ciphers I mentioned though? Do we want to support them in Kueue? If so, do we want to add all the rest of the TLS 1.2 cipher suites that are insecure, but listed here https://pkg.go.dev/crypto/tls#pkg-constants.

Also - the kubelet configuration shows a warning when an insecure cipher suite is used. Something to consider here as well maybe?

Yeah, the cipher suite configuration is ignored for 1.3. Which makes me think this should probably be clear to the user via a comment. For example, someone might try to disable TLS_CHACHA20_POLY1305_SHA256 for TLS 1.3 since it's not NIST approved.

Also, if I'd like to set minVersion to 1.2 and limit the allowed cipher suites for 1.2, it's not clear from the API whether I should also include the 1.3 cipher suites, i.e. specify:

[<allowlisted_tls_1.2_cipher_suites>, <all_tls_1.3_cipher_suites>]

or just

[<allowlisted_tls_1.2_cipher_suites>]

(the result will be the same though).

EDIT: Here's how the kubelet seems to handle it.

I will drop insurecure ciphers.

I may have grabbed the wrong enums so thanks for reviewing this in detail.

I was going to update our API to just allow:

https://cs.opensource.google/go/go/+/refs/tags/go1.25.5:src/crypto/tls/cipher_suites.go;l=56

I pushed up an update for this.

keps/8190-tls-security-profiles/kep.yaml

keps/8190-tls-security-profiles/README.md

kannon92 · 2025-12-17T20:53:53Z

/hold

As I was working on the implementation, I realize that Curves are not yet present in upstream kubernetes.

and curve preferences

kannon92 · 2025-12-18T21:29:43Z

@mimowo @kshalot

I did some updates to this. I realize that curves are a lot more complicated than I thought.

There is not yet support in upstream kubernetes so it would not be possible to implement this for visibility server. Visibility inherits settings for extended apiservers and curves are not an option yet.

So for now, I refactored this to focus on TLSMinVersion and CipherSuite. Both are options already present in Kubernetes in both kubelet and apiserver.

Kubelet: https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/kubelet/config/v1beta1/types.go#L182
APIServer: https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiserver/pkg/server/secure_serving.go#L60

kannon92 · 2025-12-19T20:21:25Z

/hold cancel

kannon92 · 2026-01-05T14:31:18Z

@kshalot @mimowo @tenzen-y I'm hoping to get this in for Kueue 0.16 so let me know if there is anything you think I need to update.

tenzen-y · 2026-01-07T16:29:07Z

@kshalot @mimowo @tenzen-y I'm hoping to get this in for Kueue 0.16 so let me know if there is anything you think I need to update.

ACK

kshalot · 2026-01-08T11:57:25Z

keps/8190-tls-security-profiles/README.md

+- Environment variables are less suitable for complex list values
+- Cannot be managed through GitOps workflows as easily
+
+### String based api


One thing to mention about this approach is that it seems to be used in the KubeletConfiguration API in core.

There's also a InitializeTLS function to take some inspiration from maybe? (handling insecure ciphers etc.)

Yea what option do you think we should do?

Kubelet/APIServer have a lot of validation logic to make sure that people pass the right ciphers to the server.

I was thinking it would be better to have it at the API level and avoid the validation.

But I am open to whatever.

If I was the one implementing, I'd probably lean towards string + validation since it would make the dependency on the tls package a bit more explicit (i.e. we rely on the tls package to define the allowed values), but I don't see a huge difference in practice because the set of cipher suites is not changing in practice. We also wouldn't have to decide which suites to support and just defer to CipherSuites and InsecureCipherSuites. And this approach is also used in KubeletConfiguration.

One hypothetical scenario where maybe this would make a difference is if the Go library deems some cipher insecure. For example the CBC ones in TLS 1.2 are now "weak" (example) even though they not "insecure" in the tls library, and I think are being deprecated in some environments outside of Go. So if Go follows suit and moves those ciphers to InsecureCiphers in the tls lib, we would get different behaviour based on the implementation we choose here. Although I'm not sure how likely this is to happen, I'd wager it's not very likely so this might be moot.

On the other hand, couldn't we move from enum to string in a backwards compatible way in the future? So since right now we don't foresee any more validations other than expecting specific values, the enum seems like a good place to start.

TL;DR - I don't have a super strong opinion. Maybe someone with more experience designing Kueue APIs can chime in. cc @mimowo

IMO the main reason I want the enums is that the UX is a lot better to an admin who needs to configure the server.

They can check the API docs and see what values are allowed.

If we provide a string or []string they have to reverse engineer from the validation logic to know what are the correct enum values.

kshalot

/lgtm

One thing I'd also like to discuss further is documenting the fact you cannot change which cipher suites are enabled for 1.3, but I think this could be continued in the implementation PR.

I don't have a strong opinion on the strings/enum discussion, but I left my thoughts.

kshalot · 2026-01-08T18:18:43Z

keps/8190-tls-security-profiles/README.md

+- Environment variables are less suitable for complex list values
+- Cannot be managed through GitOps workflows as easily
+
+### String based api


If I was the one implementing, I'd probably lean towards string + validation since it would make the dependency on the tls package a bit more explicit (i.e. we rely on the tls package to define the allowed values), but I don't see a huge difference in practice because the set of cipher suites is not changing in practice. We also wouldn't have to decide which suites to support and just defer to CipherSuites and InsecureCipherSuites. And this approach is also used in KubeletConfiguration.

One hypothetical scenario where maybe this would make a difference is if the Go library deems some cipher insecure. For example the CBC ones in TLS 1.2 are now "weak" (example) even though they not "insecure" in the tls library, and I think are being deprecated in some environments outside of Go. So if Go follows suit and moves those ciphers to InsecureCiphers in the tls lib, we would get different behaviour based on the implementation we choose here. Although I'm not sure how likely this is to happen, I'd wager it's not very likely so this might be moot.

On the other hand, couldn't we move from enum to string in a backwards compatible way in the future? So since right now we don't foresee any more validations other than expecting specific values, the enum seems like a good place to start.

TL;DR - I don't have a super strong opinion. Maybe someone with more experience designing Kueue APIs can chime in. cc @mimowo

k8s-ci-robot · 2026-01-08T18:21:58Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: kannon92, kshalot
Once this PR has been reviewed and has the lgtm label, please assign tenzen-y for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

kshalot · 2026-01-08T18:23:59Z

/lgtm

k8s-ci-robot · 2026-01-08T18:24:06Z

LGTM label has been added.

Details

Git tree hash: 51ffd9fbf820a8214738e5d03173c111db8f0767

kannon92 · 2026-01-09T19:28:16Z

cc @everettraven

as he is interested in helping out on API reviews for Kueue.

k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. kind/feature Categorizes issue or PR as related to a new feature. kind/documentation Categorizes issue or PR as related to documentation. labels Dec 15, 2025

k8s-ci-robot requested review from mimowo and pajakd December 15, 2025 16:12

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 15, 2025

kannon92 mentioned this pull request Dec 15, 2025

Provide the ability to configure TLS settings like TLSMinVersion and Ciphersuites on kueue servers #8190

Open

3 tasks

mimowo reviewed Dec 16, 2025

View reviewed changes

keps/8190-tls-security-profiles/kep.yaml Outdated Show resolved Hide resolved

kshalot reviewed Dec 16, 2025

View reviewed changes

keps/8190-tls-security-profiles/README.md Show resolved Hide resolved

kannon92 force-pushed the tls-min-version-kep branch from ba4319c to 620d023 Compare December 16, 2025 20:59

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 17, 2025

Define KEP for supporting modern tls settings (minVersion, cipher suites

7d599cc

and curve preferences

kannon92 force-pushed the tls-min-version-kep branch from 620d023 to 7d599cc Compare December 17, 2025 21:39

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 19, 2025

kshalot reviewed Jan 8, 2026

View reviewed changes

update KEP with secure tls options

d39e134

kshalot approved these changes Jan 8, 2026

View reviewed changes

k8s-ci-robot assigned kshalot Jan 8, 2026

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 8, 2026

Define KEP for supporting modern tls settings #8246

Are you sure you want to change the base?

Define KEP for supporting modern tls settings #8246

Conversation

kannon92 commented Dec 15, 2025

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Uh oh!

netlify bot commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for kubernetes-sigs-kueue ready!

Uh oh!

mimowo commented Dec 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kshalot Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kshalot Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kshalot Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kannon92 commented Dec 17, 2025

Uh oh!

kannon92 commented Dec 18, 2025

Uh oh!

kannon92 commented Dec 19, 2025

Uh oh!

kannon92 commented Jan 5, 2026

Uh oh!

tenzen-y commented Jan 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kshalot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Jan 8, 2026

netlify bot commented Dec 15, 2025 •

edited

Loading

kshalot Jan 7, 2026 •

edited

Loading

kshalot Jan 8, 2026 •

edited

Loading

kshalot Jan 8, 2026 •

edited

Loading

kshalot left a comment •

edited

Loading