Doc clarification regarding Logstash and OpenShift SCC in ECK 3.4.0+#9552
Doc clarification regarding Logstash and OpenShift SCC in ECK 3.4.0+#9552kunisen wants to merge 25 commits into
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
🔍 Preview links for changed docs |
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
pkoutsovasilis
left a comment
There was a problem hiding this comment.
Thanks for taking this on @kunisen! I agree with the known issues entry and left a comment about the breaking-changes section
| # seccompProfile intentionally omitted — not permitted by anyuid SCC | ||
| ``` | ||
|
|
||
| This override can be removed once you upgrade to the ECK version that includes the fix for [#9550](https://github.com/elastic/cloud-on-k8s/issues/9550). |
There was a problem hiding this comment.
Thanks @pkoutsovasilis
Also to @eedugon
I found a better phrase to avoid potential confusion - or later once available, per https://www.elastic.co/docs/release-notes/cloud-on-k8s/known-issues
I will use the below phrase since #9551 seems to be included in 3.5.0:
Alternatively, upgrade to ECK 3.5.0 or later once available.
There was a problem hiding this comment.
@pkoutsovasilis @eedugon this is the updated wording - 7de745d. Thanks!
There was a problem hiding this comment.
@eedugon should we refer to future versions?
Interesting topic. The current proposal doesn't look bad, although in some way it doesn't feel ideal.
I'll check internally with the team to see if there's a better way of handling this type of statement.
There was a problem hiding this comment.
@pkoutsovasilis , we don't have a definitive solution or specific policy here.
If we are almost sure that our next minor release will solve the issue and that release is not far away in time then I'd say it's ok to use the approach shared by @kunisen's screenshot that is already in place in other parts of the release notes. For example:
Alternatively, upgrade to ECK 3.5.0 or later once available.
And ideally, when 3.5.0 is shipped it would be awesome to update the sentence and convert it to something like:
This issue is fixed in ECK 3.5.0 (or whatever way that flows properly in the narrative)
But if we are uncertain of when the issue will be fixed we should stick to the original proposal, with something like:
This override can be removed once you upgrade to the ECK version that includes the fix for #9550. (or any similar way, just implying that the issue will be fixed in a future release and the implications to versions facing the issue).
With this second approach I would consider important (not ideal) to update the known issue report as soon as the issue is really fixed, as that type of text, if it's outdated, looks worse than an outdated upgrade to ECK 3.5.0 or later once available.
In short: Better to not refer to future releases but we have to find a balance in terms of docs maintainability. So whatever you prefer and feel it's better.
Co-authored-by: Panos Koutsovasilis <panos.koutsovasilis@elastic.co>
handle this comment - #9552 (review)
| Override the Logstash pod security context explicitly to prevent ECK from injecting its defaults. Keep `fsGroup: 1000` as Logstash requires it for volume writes, but omit `seccompProfile`: | ||
|
|
||
| ```yaml | ||
| spec: | ||
| podTemplate: | ||
| spec: | ||
| securityContext: | ||
| fsGroup: 1000 | ||
| # seccompProfile intentionally omitted — not permitted by anyuid SCC | ||
| ``` |
There was a problem hiding this comment.
This workaround is not general. It's valid only in namespaces governed by an anyuid-style SCC (RunAsAny fsGroup, seccomp forbidden) that are not also enforcing restricted Pod Security Admission.
Recommend: scope this explicitly to anyuid-type SCCs without restricted PSA, add a warning that in restricted-PSA namespaces seccompProfile is required (do not omit) and fsGroup must fall within the namespace's allocated supplemental-groups range (not 1000). As written, we'd be publishing a workaround that is not generally applicable.
|
@pebrc About breaking change vs known issue, sorry I think I saw your message but I couldn't find it. @pebrc should we move this to known issue section instead of breaking changes? I added this block because of this seems to be the context we want to add per @pkoutsovasilis's comment below:
Also, I added in breaking change section because we have the "rolling restart" for ES component. From a customer perspective, if we say "ES rolling restart" in breaking change, while we say "Other components may also have rolling restart" in known issue, it may add unnecessary confusion. But I also get your point that logically, it doesn't have to be in breaking change, due to it's limited scope. Happy to move it to known issue section if @pebrc or @pkoutsovasilis you agree. OTOH, if you think we should remove this part, or other actions should be taken, please let me know. |
@kunisen you cannot find it because I deleted it. It slipped through by mistake when I submitted my review. In general I find it problematic to categorise a rolling restart as a breaking change. However given that we want to draw the users' attention to our known issue section with the workaround for OCP I am OK with it, which is why I retracted my comment. |
to address #9552 (comment)
| :::{dropdown} Logstash pods rejected by OpenShift SCC when using a non-default security context constraint | ||
| In ECK 3.4.x, the Logstash controller unconditionally injects `seccompProfile: RuntimeDefault` and `fsGroup: 1000` into the pod security context, ignoring the `--set-default-security-context=auto-detect` operator flag. On OpenShift, this flag should suppress the injection, as it does for all other ECK-managed workloads ({{es}}, {{product.kibana}}, APM Server). Clusters using a non-default SCC such as `anyuid` — which does not permit explicit seccomp configuration — will see Logstash pods rejected at admission after upgrading. | ||
|
|
||
| For more information, check this [Issue #9550](https://github.com/elastic/cloud-on-k8s/issues/9550). | ||
|
|
||
| **Workaround** | ||
|
|
||
| This workaround applies only to namespaces governed by an `anyuid`-style SCC (`RunAsAny` fsGroup, seccomp forbidden) that are **not** also enforcing [restricted Pod Security Admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/). | ||
|
|
||
| To check whether your namespace enforces restricted PSA, run: | ||
|
|
||
| ```bash | ||
| kubectl get namespace <your-namespace> -o jsonpath='{.metadata.labels}' | ||
|
|
||
| # or on OpenShift: | ||
| oc get namespace <your-namespace> -o jsonpath='{.metadata.labels}' | ||
| ``` | ||
|
|
||
| If you see `pod-security.kubernetes.io/enforce: restricted`, do **not** use this workaround — in restricted-PSA namespaces, `seccompProfile` is required and must not be omitted, and `fsGroup` must fall within the namespace's allocated `supplemental-groups` range rather than a fixed value like `1000`. | ||
|
|
||
| For namespaces using `anyuid`-style SCC without restricted PSA enforcement, override the Logstash pod security context as follows to prevent ECK from injecting its defaults: | ||
|
|
||
| ```yaml | ||
| spec: | ||
| podTemplate: | ||
| spec: | ||
| securityContext: | ||
| fsGroup: 1000 | ||
| # seccompProfile intentionally omitted — not permitted by anyuid SCC | ||
| ``` | ||
|
|
||
| This override can be removed once you upgrade to ECK 3.5.0 or later when available, which includes the fix for [#9550](https://github.com/elastic/cloud-on-k8s/issues/9550). | ||
| ::: |
There was a problem hiding this comment.
Is this known issue exactly the same as the one reported in 3.4.1? If so, and considering how long the block is, I'd strongly suggest to put the text into a snippet and import it in both versions, so we can have a single source of truth.
For the content of the workaround we need ensure @pebrc and the team validates it, as I've seen an outdated comment where he mentions that the proposed workaround wasn't completely valid. Just in case that's still applicable.
There was a problem hiding this comment.
Hi @eedugon 👋 the workaround has been validated on OpenShift with the v3.4.0 operator.
One important clarification for the docs though, @kunisen: the podTemplate.spec.securityContext override must be applied to existing Logstash CRs before upgrading to ECK 3.4.0. If the upgrade happens first and the Logstash pods enter the SCC rejection loop, updating the CR afterwards won't help - the controller detects pending pod creations and re-queues without patching the StatefulSet spec, so the fix never takes effect. For the latter, the only recovery is to delete and re-create the Logstash CR with the security context as described in place.
|
@pebrc @pkoutsovasilis sorry to ping you again, but may I trouble you to take a look and see if it is more reasonable now please? Thanks in advance! |
to handle #9552 (comment)
| ::: | ||
|
|
||
| This override can be removed once you upgrade to ECK 3.5.0 or later when available, which includes the fix for [#9550](https://github.com/elastic/cloud-on-k8s/issues/9550). | ||
| ::: |
| ## 3.4.0 [elastic-cloud-kubernetes-340-known-issues] | ||
|
|
||
| There are no known issues in ECK 3.4.0 | ||
| :::{dropdown} Logstash pods rejected by OpenShift SCC when using a non-default security context constraint |
There was a problem hiding this comment.
This 3.4.0 block (lines 62–98) is byte-for-byte identical to the 3.4.1 one above, so every suggestion above applies here too. Rather than applying each fix twice, consider @eedugon's suggestion of extracting the block into a shared snippet and importing it into both sections — single source of truth, and the structural fence fix only has to be made once.
Co-authored-by: Peter Brachwitz <peter.brachwitz@elastic.co>
Co-authored-by: Peter Brachwitz <peter.brachwitz@elastic.co>
Co-authored-by: Peter Brachwitz <peter.brachwitz@elastic.co>
Co-authored-by: Peter Brachwitz <peter.brachwitz@elastic.co>
Co-authored-by: Peter Brachwitz <peter.brachwitz@elastic.co>
Co-authored-by: Peter Brachwitz <peter.brachwitz@elastic.co>
…loud-on-k8s into kunisen-docpr-dci-1408
Co-authored-by: Peter Brachwitz <peter.brachwitz@elastic.co>
|
Thanks @pebrc for the great suggestion. [1]From here, I think we have two outstanding ones left:
That requires @eedugon's double check (but not Kuni's). [2]Also, @pebrc or @pkoutsovasilis, at this stage, is there anything still confusing or misleading from ECK tech perspective? Or are we good to go from ECK side, and next is waiting on doc team's sign off? Happy to make corrections and updates if anything else I should do. Thanks again! |
Co-authored-by: Edu González de la Herrán <25320357+eedugon@users.noreply.github.com>
Co-authored-by: Edu González de la Herrán <25320357+eedugon@users.noreply.github.com>
Co-authored-by: Edu González de la Herrán <25320357+eedugon@users.noreply.github.com>

Have you signed the contributor license agreement?
Have you followed the contributor guidelines?
If you submit code, is your pull request against main? We recommend pull requests against main. We will backport them as needed.
Description
It's to address https://github.com/elastic/docs-content-internal/issues/1408.