Skip to content

[bitnami/etcd] ETCD JWT Permission Issue on Openshift #36468

@krishnaGajabi

Description

@krishnaGajabi

Name and Version

bitnami/etcd 12.0.14

What steps will reproduce the bug?

  1. I am using openshift cluster (4.19.7) , when i install etcd chart i see pods crashing.
    Please find below logs for reference.
[root@ocp-helper ~]# kubectl get pods -n openebs -l app=etcd
NAME             READY   STATUS             RESTARTS      AGE
openebs-etcd-0   0/1     CrashLoopBackOff   5 (61s ago)   5m30s
openebs-etcd-1   0/1     CrashLoopBackOff   5 (55s ago)   5m30s
openebs-etcd-2   0/1     CrashLoopBackOff   5 (55s ago)   5m30s

{"level":"error","ts":"2026-02-13T05:18:13.646221Z","caller":"auth/jwt.go:135","msg":"problem loading JWT options","error":"open /opt/bitnami/etcd/certs/token/jwt-token.pem: permission denied","stacktrace":"go.etcd.io/etcd/server/v3/auth.newTokenProviderJWT\n\tgo.etcd.io/etcd/server/v3/auth/jwt.go:135\ngo.etcd.io/etcd/server/v3/auth.NewTokenProvider\n\tgo.etcd.io/etcd/server/v3/auth/store.go:1140\ngo.etcd.io/etcd/server/v3/etcdserver.NewServer\n\tgo.etcd.io/etcd/server/v3/etcdserver/server.go:364\ngo.etcd.io/etcd/server/v3/embed.StartEtcd\n\tgo.etcd.io/etcd/server/v3/embed/etcd.go:262\ngo.etcd.io/etcd/server/v3/etcdmain.startEtcd\n\tgo.etcd.io/etcd/server/v3/etcdmain/etcd.go:207\ngo.etcd.io/etcd/server/v3/etcdmain.startEtcdOrProxyV2\n\tgo.etcd.io/etcd/server/v3/etcdmain/etcd.go:114\ngo.etcd.io/etcd/server/v3/etcdmain.Main\n\tgo.etcd.io/etcd/server/v3/etcdmain/main.go:40\nmain.main\n\tgo.etcd.io/etcd/server/v3/main.go:31\nruntime.main\n\truntime/proc.go:272"} {"level":"warn","ts":"2026-02-13T05:18:13.646298Z","caller":"etcdserver/server.go:371","msg":"failed to create token provider","error":"auth: invalid auth options"} {"level":"info","ts":"2026-02-13T05:18:13.714678Z","caller":"embed/etcd.go:426","msg":"closing etcd server","name":"openebs-etcd-0","data-dir":"/bitnami/etcd/data","advertise-peer-urls":["http://openebs-etcd-0.openebs-etcd-headless.openebs.svc.cluster.local:2380"],"advertise-client-urls":["http://openebs-etcd-0.openebs-etcd-headless.openebs.svc.cluster.local:2379","http://openebs-etcd.openebs.svc.cluster.local:2379"]}

Observation:

global.compatibility.openshift.adaptSecurityContext
When enabled (auto or force), this logic removes:
fsGroup
runAsUser
runAsGroup

Template snippet:

{{- $adaptedContext = omit $adaptedContext "fsGroup" "runAsUser" "runAsGroup" -}}

Because fsGroup was removed:
The pod no longer sets fsGroup: 1001
Secret volumes remain owned by root:root
UID 1001 cannot read jwt-token.pem
ETCD fails during startup
This behavior is specific to OpenShift compatibility mode.

Data directory permissions (Correct, here the init container also has the uid 1001 hardcoded) (how is the random uid assigned by openshift going to be used here?)

/bitnami/etcd
drwxrwxrwx 1001 1001
The volume-permissions init container correctly assigns permissions for the data directory.
Secret mount permissions (Problem)
/opt/bitnami/etcd/certs/token
drwxrwxrwt root root
@puls8-etcd-0:/opt/bitnami/etcd$ ls -l /opt/bitnami/etcd/certs/token/..data/
total 4
-r--------. 1 root root 3243 Feb 21 04:27 jwt-token.pem

@puls8-etcd-0:/opt/bitnami/etcd$ ls -l /opt/bitnami/etcd/certs/token/..data/
total 4
-r--------. 1 root root 3243 Feb 21 04:27 jwt-token.pem

Secrets are mounted as:

Owner: root
Group: root
Not readable by UID 1001

UID is still 1001, it should have been the random uid openshift has assigned.

The etcd pods get deployed if global.compatibility.openshift.adaptSecurityContext is set to disable

What is the expected behavior?

Pod should be running

What do you see instead?

Pods are in crashloopback state

Additional information

ETCD CHART VERSION: 12.0.14

Metadata

Metadata

Assignees

Labels

etcdtech-issuesThe user has a technical issue about an applicationtriageTriage is needed

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions