Docker Swarm Deployment with secrets results in challenges with server and worker connecting to postgres

[luvablemarmot]

Describe your question/
I am attempting to deploy a working authentik solution using docker compose, docker secrets in swarm mode

Relevant info
Docker running in swarm mode
the postgres password and authentik key are both docker secrets. I'm deploying the stack via portainer

Screenshots
If applicable, add screenshots to help explain your problem.

Logs
output from worker/server

INF | event=PostgreSQL connection failed, retrying... (connection failed: connection to server at "10.0.1.137", port 5432 failed: FATAL: password authentication failed for user "authentik") logger=authentik.lib.config timestamp=1740367506.2970626
INF | event=PostgreSQL connection failed, retrying... (connection failed: connection to server at "10.0.1.137", port 5432 failed: FATAL: password authentication failed for user "authentik") logger=authentik.lib.config timestamp=1740367507.3094716

postgres i was seeing this (so there was connectivity)

2025-02-24 01:42:21.439 UTC [103] FATAL: password authentication failed for user "authentik"
2025-02-24 01:42:21.439 UTC [103] DETAIL: Connection matched file "/var/lib/postgresql/data/pg_hba.conf" line 128: "host all all all scram-sha-256"
2025-02-24 01:42:22.451 UTC [104] FATAL: password authentication failed for user "authentik"
2025-02-24 01:42:22.451 UTC [104] DETAIL: Connection matched file "/var/lib/postgresql/data/pg_hba.conf" line 128: "host all all all scram-sha-256"
2025-02-24 01:42:23.467 UTC [105] FATAL: password authentication failed for user "authentik"
2025-02-24 01:42:23.467 UTC [105] DETAIL: Connection matched file "/var/lib/postgresql/data/pg_hba.conf" line 128: "host all all all scram-sha-256"

the password is correct in /var/run/secrets but somehow its not working properly between the worker/server and postgres its self.

docker secrets output

jdunn@container-manager:~$ sudo docker secret ls
ID NAME DRIVER CREATED UPDATED
liaullk9vfcxh4x39la49hf7n AUTHENTIK_PG_PASS 12 seconds ago 12 seconds ago
btuzygv0avtttyiioek6zqkwd AUTHENTIK_SECRET_KEY 7 days ago 7 days ago

Version and Deployment (please complete the following information):

version: "3.8"

services:
postgresql-authentik:
image: docker.io/library/postgres:16-alpine
deploy:
replicas: 1
restart_policy:
condition: on-failure
healthcheck:
test: ["CMD-SHELL", "pg_isready -d authentik -U authentik"]
interval: 30s
retries: 5
start_period: 30s
timeout: 5s
volumes:
- database:/var/lib/postgresql/data
secrets:
- AUTHENTIK_PG_PASS
environment:
POSTGRES_PASSWORD: "/run/secrets/AUTHENTIK_PG_PASS"
POSTGRES_USER: authentik
POSTGRES_DB: authentik
networks:
- traefik-overlay

redis:
image: docker.io/library/redis:alpine
command: --save 60 1 --loglevel warning
deploy:
replicas: 1
restart_policy:
condition: on-failure
healthcheck:
test: ["CMD-SHELL", "redis-cli ping | grep PONG"]
interval: 30s
retries: 5
start_period: 20s
timeout: 3s
volumes:
- redis:/data
networks:
- traefik-overlay

server:
image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2024.12.3}
deploy:
replicas: 1
restart_policy:
condition: on-failure
labels:
- "traefik.enable=true"
- "traefik.http.routers.authentik.rule=Host(auth.dunnnetwork.com)"
- "traefik.http.routers.authentik.entrypoints=websecure"
- "traefik.http.routers.authentik.tls=true"
- "traefik.http.routers.authentik.tls.certresolver=myresolver"
- "traefik.http.routers.authentik.tls.domains[0].main=dunnnetwork.com"
- "traefik.http.routers.authentik.tls.domains[0].sans=*.dunnnetwork.com"
- "homepage.group=Utilities"
- "homepage.name=Authentik"
- "homepage.icon=authentik.svg"
- "homepage.href=https://auth.dunnnetwork.com"
- "homepage.description=Authentication Manager"
command: server
secrets:
- AUTHENTIK_SECRET_KEY
- AUTHENTIK_PG_PASS
environment:
AUTHENTIK_REDIS__HOST: "tasks.redis"
AUTHENTIK_POSTGRESQL__HOST: "tasks.postgresql-authentik"
AUTHENTIK_POSTGRESQL__USER: authentik
AUTHENTIK_POSTGRESQL__NAME: authentik
AUTHENTIK_POSTGRESQL__PASSWORD: "/run/secrets/AUTHENTIK_PG_PASS"
AUTHENTIK_SECRET_KEY: "/run/secrets/AUTHENTIK_SECRET_KEY"
volumes:
- media:/media
- certs:/certs
- templates:/templates
networks:
- traefik-overlay

worker:
image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2024.12.3}
deploy:
replicas: 1
restart_policy:
condition: on-failure
command: worker
secrets:
- AUTHENTIK_SECRET_KEY
- AUTHENTIK_PG_PASS
environment:
AUTHENTIK_REDIS__HOST: "tasks.redis"
AUTHENTIK_POSTGRESQL__HOST: "tasks.postgresql-authentik"
AUTHENTIK_POSTGRESQL__USER: authentik
AUTHENTIK_POSTGRESQL__NAME: authentik
AUTHENTIK_POSTGRESQL__PASSWORD: "/run/secrets/AUTHENTIK_PG_PASS"
AUTHENTIK_SECRET_KEY: "/run/secrets/AUTHENTIK_SECRET_KEY"
user: root
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- media:/media
- certs:/certs
- templates:/templates
networks:
- traefik-overlay

volumes:
database:
driver: local
redis:
driver: local
media:
driver: local
certs:
driver: local
templates:
driver: local

networks:
traefik-overlay:
external: true

secrets:
AUTHENTIK_SECRET_KEY:
external: true
AUTHENTIK_PG_PASS:
external: true

Additional context
Add any other context about the problem here.

[tommatime]

Hello, I had a similar issue. According to this page of the documentation, the Authentik environment variables can be prefixed with file:// to indicate that the variables should be read from a file. For example, those specific environment would look like this:

AUTHENTIK_POSTGRESQL__PASSWORD: "file:///run/secrets/AUTHENTIK_PG_PASS"
AUTHENTIK_SECRET_KEY: "file:///run/secrets/AUTHENTIK_SECRET_KEY"

For the Postgres image, the password can be set to the secret file with the POSTGRES_PASSWORD_FILE variable instead of POSTGRES_PASSWORD:

POSTGRES_PASSWORD_FILE: "/run/secrets/AUTHENTIK_PG_PASS"