Supabase
Support for Kubernetes Postgres Operators (CNPG) #31147
Replies: 8 comments 44 replies
-
|
I'm currently working through this. I have tinkered with it before, and I duplicated supabase's postgres Dockerfile, and added in the cnpg requirements. I got it working, but in the end didnt need it. So I'm going to try a simple Dockerfile that will build on the supabase/postgres docker image and install the barman packages (the minimum requirements for cnpg) and see how it goes. I have recently found this: I guess there are 3 ways of applying the migrations.
It would be amazing to have all the migrations built into a docker container that can run them against any postgres instance, as opposed to having them included in the supabase/postgres image. I'll see where I get to with just brute-forcing barman into the supabase/postgres image. If that works easily enough, then I think working on a community supported (or officially supported?) container that centralizes all the required bootstraping migrations in something. on a side note, there is a bug in supabase/realtime that prevents it from clustering correctly. |
Beta Was this translation helpful? Give feedback.
-
|
For my clarification, what defines the 'magic' that makes the supabase postgres container what it is. If I understand correctly, it is a combination of postgres extensions like wal2json and pgjwt to build a featureful generic postgres image, and then a series of initial migrations to create tables needed for other supabase components like auth and Realtime? A sidecar init container sounds like a great idea. Package up all the migrations with a migrate script into a separate supabase/postgres-init, and then anyone could attach that to their existing postgres container to get the needed migrations. Maybe even configure the init container if you only plan to use certain parts of the supabase ecosystem, like how the current Kubernetes chart lets you enable or disable each piece. I think the second part is how to handle all the postgres plugins themselves. I was starting with this article. where they recommend using the cnpg image at the base and installing more plugins on top. So, maybe we could use all the ansible automation from the supabase/postgres image as a way to package up supabase recommended plugins and treat it kind of like a 'standard library for postgres'. Then install on top of cnpg base. Then, that feels like it would generalize to other operators or custom images
|
Beta Was this translation helpful? Give feedback.
-
|
I agree that the magic of supabase has to be maintained! I will say that, in my experience, a "bring your own base image" isnt feasible. I think its easier to start with the supabase/postgres image, smash in barman/barman-cli/barman-cli-cloud, and figure out a way to make migrations work (sidecar, initcontainer, bootstrap from a standalone container... idk). |
Beta Was this translation helpful? Give feedback.
-
|
The following Dockerfile is bringing up a CNPG cluster for supabase. FROM supabase/postgres:15.8.1.017
# https://www.postgresql.org/download/linux/debian/
# Import the repository signing key:
RUN apt install curl ca-certificates
RUN install -d /usr/share/postgresql-common/pgdg
RUN curl -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc --fail https://www.postgresql.org/media/keys/ACCC4CF8.asc
# Create the repository configuration file:
RUN sh -c 'echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
# https://docs.pgbarman.org/release/3.12.1/user_guide/installation.html
# Install barman:
RUN apt update && apt install -y --no-install-recommends \
barman \
barman-cli \
barman-cli-cloud \
&& rm -rf /var/lib/apt/lists/*
# Change the uid of postgres to 26 - CNPG fails if this doesn't happen
RUN usermod -u 26 postgres
USER 26Make sure your image name meets the requirements: https://cloudnative-pg.io/documentation/1.24/container_images/
I'm now working on getting migrations to happen. I think I am going to build a "migrations" image with a bash script and the migrations from the various repos. The image can then run as a job and apply the migrations against the CNPG database. Only issue with this is dealing with future migrations. There is no way to check if existing migrations have been run, so it will blindly apply all the migrations (which is fine for a first run, but will probably throw loads of errors if re-run considering there are |
Beta Was this translation helpful? Give feedback.
-
|
Working on the migrations atm, but I'm getting tired. I'm using dbmates latest container here. # it seems like supabase uses dbmate?
# https://github.com/supabase/postgres/tree/develop/migrations
# https://github.com/amacneil/dbmate
FROM ghcr.io/amacneil/dbmate:2.24.2@sha256:6e245cce5580567747cf440d84a7507121793c81564772fdd75f8ee0f45bbb79
COPY ./db ./db
COPY ./migrate.sh ./migrate.sh
RUN chmod +x ./migrate.sh
ENTRYPOINT ["./migrate.sh"]the shell script #!/usr/bin/env sh
echo "--------------------"
echo "Beginning migrations"
dbmate --migrations-dir "./db" --wait up
echo " Complete"
echo "--------------------"I got to here: This is probably because of Thought I'd share some progress. Going to circle back to this in the next few days, or at least before the New Year! |
Beta Was this translation helpful? Give feedback.
-
|
Ok, so the migrations in the supabase/supabase repo use variable substitution to load env vars into the postgres db (JWT tokens mostly). The docker-compose file maps the migrations into a specific order that is not indicated by their filename. Which means those files need to be renamed to a dbmate compatible format, and organised in the correct order. Which leads onto the next problem... So, I think some manual code for these specific migrations is required. Update: looks like migrations are working. Havent tested against supabase services yet |
Beta Was this translation helpful? Give feedback.
-
|
Here is an all-in-one update of my progress so far: CNPG+Supabase Compatible Dockerfile FROM supabase/postgres:15.8.1.017
# Import the repository signing key:
RUN apt install curl ca-certificates
RUN install -d /usr/share/postgresql-common/pgdg
RUN curl -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc --fail https://www.postgresql.org/media/keys/ACCC4CF8.asc
# Create the repository configuration file:
RUN sh -c 'echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
# Update the package lists:
RUN apt update && apt install -y --no-install-recommends \
barman \
barman-cli \
barman-cli-cloud \
&& rm -rf /var/lib/apt/lists/*
# Change the uid of postgres to 26
RUN usermod -u 26 postgres
USER 26Make sure you tag the image according to CNPG requirements https://cloudnative-pg.io/documentation/1.24/container_images/ Example cluster manifest: apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: testing
spec:
instances: 3
imageName: <<<<IMAGE NAME>>>>
imagePullPolicy: Always
storage:
size: 5Gi
enableSuperuserAccess: true
postgresql:
pg_hba:
# ripped from supabase/posgres/ansible/files/postgresql_config/pg_hba.conf
- local all supabase_admin scram-sha-256
- local all all peer map=supabase_map
- host all all 127.0.0.1/32 trust
- host all all ::1/128 trust
- host all all 10.0.0.0/8 scram-sha-256
- host all all 172.16.0.0/12 scram-sha-256
- host all all 192.168.0.0/16 scram-sha-256
- host all all 0.0.0.0/0 scram-sha-256
- host all all ::0/0 scram-sha-256
pg_ident:
# ripped from supabase/posgres/ansible/files/postgresql_config/pg_ident.conf
- supabase_map postgres postgres
- supabase_map gotrue supabase_auth_admin
- supabase_map postgrest authenticator
- supabase_map adminapi postgres
# ripped from supabase/posgres/ansible/files/postgresql_config/postgresql.conf
shared_preload_libraries:
[
pg_stat_statements,
pg_stat_monitor,
pgaudit,
plpgsql,
plpgsql_check,
pg_cron,
pg_net,
timescaledb,
auto_explain,
pg_tle,
supautils,
# pgsodium # pgsodium crashes cnpg db pods
]
bootstrap:
initdb:
database: supabase
# I think this has to be supabase_admin (due to migration hard coded strings), however I will investigate further
owner: supabase_adminMigrations: NOTE: comment out the contents of Dockerfile: # it seems like supabase uses dbmate?
# https://github.com/supabase/postgres/tree/develop/migrations
# https://github.com/amacneil/dbmate
FROM ghcr.io/amacneil/dbmate:2.24.2@sha256:6e245cce5580567747cf440d84a7507121793c81564772fdd75f8ee0f45bbb79
COPY ./db ./db
COPY ./migrate.sh ./migrate.sh
RUN chmod +x ./migrate.sh
# Install
# gettext for the envsubst bin
# moreutils for the sponge bin (allows reading/writing from the same file for in-situ envsubst)
RUN apk add --no-cache gettext moreutils
ENTRYPOINT ["./migrate.sh"]migrate.sh: #!/usr/bin/env sh
echo "--------------------"
echo "Beginning migrations"
# in-situ env_var substitution
echo "Variable Substitution"
for migration in ./db/final/*.sql; do
envsubst '$POSTGRES_USER, $POSTGRES_PASSWORD, $POSTGRES_DB, $JWT_SECRET, $JWT_EXP' < $migration | sponge $migration
done
# build from parts, as the CNPG superuser targets "*" database.
# Superuser credentials, and the App DB
db_uri="postgresql://${SUPERUSER_USER}:${SUPERUSER_PASSWORD}@${SUPERUSER_HOST}:${SUPERUSER_PORT}/${POSTGRES_DB}"
echo "Running base migrations"
# MIGRATION_SCHEMA as first search_path so the dbmate migrations table gets created in a seperate schema (not polluting the public schema, and not interfering with Realtime's schema_migrations table)
dbmate_uri="${db_uri}?search_path=${MIGRATION_SCHEMA},public"
# https://github.com/amacneil/dbmate
dbmate --url $dbmate_uri --migrations-dir "./db/base" --wait --no-dump-schema up
echo "Running final migrations"
echo "_supabase.sql"
psql -d $db_uri -f "./db/final/_supabase.sql"
echo "webhooks.sql"
psql -d $db_uri -f "./db/final/webhooks.sql"
echo "jwt.sql"
psql -d $db_uri -f "./db/final/jwt.sql"
echo "logs.sql"
psql -d $db_uri -f "./db/final/logs.sql"
echo "realtime.sql"
psql -d $db_uri -f "./db/final/realtime.sql"
echo "poolers.sql"
psql -d $db_uri -f "./db/final/poolers.sql"
echo "roles.sql"
psql -d $db_uri -f "./db/final/roles.sql"
echo " Complete"
echo "--------------------"Job example manifest: apiVersion: batch/v1
kind: Job
metadata:
name: supabase-migration
spec:
template:
spec:
containers:
- name: supabase-migration
image: <<<<CONTAINER NAME>>>>
env:
- name: SUPERUSER_HOST
valueFrom:
secretKeyRef:
name: testing-superuser
key: host
# Use the Superuser CNPG secret for migration connection
- name: SUPERUSER_PORT
valueFrom:
secretKeyRef:
name: testing-superuser
key: port
# Use the Superuser CNPG secret for migration connection
- name: SUPERUSER_USER
valueFrom:
secretKeyRef:
name: testing-superuser
key: user
# Use the Superuser CNPG secret for migration connection
- name: SUPERUSER_PASSWORD
valueFrom:
secretKeyRef:
name: testing-superuser
key: password
# Schema where dbmate will store base migration details, so it doesnt pollute the `public` schema
- name: MIGRATION_SCHEMA
value: dbmate
# use the App secret, for variable substitution in supabase migrations
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: testing-app
key: password
# use the App secret, for variable substitution in supabase migrations
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: testing-app
key: user
# use the App secret, for variable substitution in supabase migrations
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: testing-app
key: dbname
# for variable substitution in supabase migrations
- name: JWT_SECRET
valueFrom:
secretKeyRef:
name: supabase-jwt
key: secret
# for variable substitution in supabase migrations
- name: JWT_EXP
valueFrom:
secretKeyRef:
name: supabase-jwt
key: expiry
restartPolicy: Never
backoffLimit: 4the secrets Notes on migration: Any new migrations in supabase/postgres can be loaded into the Probably worth adding some sort of flag or check to disable the And finally: 24/12/27 UPDATEOk, I had some issues with migrations, schemas, connection strings etc. Changes:
I will work on getting the other services up. 04/01/25. |
Beta Was this translation helpful? Give feedback.
-
|
@Towerful Thanks for the detailed description. I tried following along but im getting an error when deploying the cluster. I think its related to this I tried removing the file or placing an empty file but that causes an error did you experience something similar? |
Beta Was this translation helpful? Give feedback.
-
I'm back trying to get this working - have you made any progress? |
Beta Was this translation helpful? Give feedback.
-
|
Honestly, not much has changed. Postgres Dockerfile: ARG baseImage
FROM ${baseImage}
# Import the repository signing key:
RUN apt install curl ca-certificates
RUN install -d /usr/share/postgresql-common/pgdg
RUN curl -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc --fail https://www.postgresql.org/media/keys/ACCC4CF8.asc
# Create the repository configuration file:
RUN sh -c 'echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
# Update the package lists:
RUN apt update && apt install -y --no-install-recommends \
barman \
barman-cli \
barman-cli-cloud \
&& rm -rf /var/lib/apt/lists/*
# Change the uid of postgres to 26
RUN usermod -u 26 postgres
USER 26The migration job: migrate.sh: #!/usr/bin/env sh
echo "--------------------"
echo "Beginning migrations"
echo "Truncating migration ./db/base/10000000000000_demote-postgres.sql"
echo "CNPG handles superuser account, so dont demote postgres"
echo "-- migrate:up\n-- migrate:down" > ./db/base/10000000000000_demote-postgres.sql
# build from parts, as the CNPG superuser targets "*" database.
# user Superuser credentials, and the App DB
# dbmate as first search_path so the migrations table gets created in a seperate schema (not polluting the public schema)
db_uri="postgresql://${SUPERUSER_USER}:${SUPERUSER_PASSWORD}@${SUPERUSER_HOST}:${SUPERUSER_PORT}/${POSTGRES_DB}"
echo "Running base migrations from ./db/base/*.sql"
dbmate_uri="${db_uri}?search_path=${MIGRATION_SCHEMA},public"
echo $dbmate_uri
# https://github.com/amacneil/dbmate
dbmate --url $dbmate_uri --migrations-dir "./db/base" --wait --wait-timeout 120s --no-dump-schema up
# in-situ env_var substitution
echo "Variable Substitution in ./db/final/*.sql files"
for migration in ./db/final/*.sql; do
envsubst '$POSTGRES_USER, $POSTGRES_PASSWORD, $POSTGRES_DB, $JWT_SECRET, $JWT_EXP' < $migration | sponge $migration
done
echo "Running final migrations fom ./db/final"
echo "_supabase.sql"
psql -d $db_uri -f "./db/final/_supabase.sql"
echo "webhooks.sql"
psql -d $db_uri -f "./db/final/webhooks.sql"
echo "Skipping final/jwt.sql"
echo "postgREST handles settings these variables"
# echo "jwt.sql"
# psql -d $db_uri -f "./db/final/jwt.sql"
echo "logs.sql"
psql -d $db_uri -f "./db/final/logs.sql"
echo "realtime.sql"
psql -d $db_uri -f "./db/final/realtime.sql"
echo "poolers.sql"
psql -d $db_uri -f "./db/final/poolers.sql"
echo "roles.sql"
psql -d $db_uri -f "./db/final/roles.sql"
echo " Complete"
echo "--------------------"Migration job dockerfile: # it seems like supabase uses dbmate?
# https://github.com/supabase/postgres/tree/develop/migrations
# https://github.com/amacneil/dbmate
ARG baseImage
FROM ${baseImage}
COPY ./db ./db
COPY ./migrate.sh ./migrate.sh
RUN chmod +x ./migrate.sh
# Install
# gettext for the envsubst bin
# moreutils for the sponge bin (allows reading/writing from the same file for in-situ envsubst)
RUN apk add --no-cache gettext moreutils
ENTRYPOINT ["./migrate.sh"]the I can't be bothered subbing garden.io variables, so here is a raw of my cnpg migration job: apiVersion: batch/v1
kind: Job
metadata:
name: supabase-migration
labels:
stage: supabase
spec:
template:
metadata:
labels:
stage: supabase
spec:
containers:
- name: supabase-migration
image: ${actions.build.supabase-db-migrate-build.outputs.deploymentImageId}
env:
- name: SUPERUSER_HOST
valueFrom:
secretKeyRef:
name: ${var.supabase.postgres.cluster-name}-superuser
key: host
# Use the Superuser CNPG secret for migration connection
- name: SUPERUSER_PORT
valueFrom:
secretKeyRef:
name: ${var.supabase.postgres.cluster-name}-superuser
key: port
# Use the Superuser CNPG secret for migration connection
- name: SUPERUSER_USER
valueFrom:
secretKeyRef:
name: ${var.supabase.postgres.cluster-name}-superuser
key: user
# Use the Superuser CNPG secret for migration connection
- name: SUPERUSER_PASSWORD
valueFrom:
secretKeyRef:
name: ${var.supabase.postgres.cluster-name}-superuser
key: password
# Schema where dbmate will store base migration details, so it doesnt pollute the `public` schema
- name: MIGRATION_SCHEMA
value: dbmate
# use the App secret, for variable substitution in supabase migrations
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: ${var.supabase.postgres.cluster-name}-app
key: password
# use the App secret, for variable substitution in supabase migrations
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: ${var.supabase.postgres.cluster-name}-app
key: user
# use the App secret, for variable substitution in supabase migrations
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: ${var.supabase.postgres.cluster-name}-app
key: dbname
restartPolicy: Never
backoffLimit: 4I am not passing in JWT secret etc (found in the supabase/supabase and my cluster.yaml. again, cant be bothered with garden.io substitution (garden does funky things with build tag/name/etc, hence the reference to apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: ${var.supabase.postgres.cluster-name}
labels:
stage: supabase
spec:
instances: ${var.replicas}
imageName: "${actions.run.supabase-db-tag.var.imageId}"
imagePullPolicy: Always
storage:
storageClass: local-path
size: 5Gi
enableSuperuserAccess: true
postgresql:
pg_hba:
# ripped from supabase/posgres/ansible/files/postgresql_config/pg_hba.conf
- local all supabase_admin scram-sha-256
- local all all peer map=supabase_map
- host all all 127.0.0.1/32 trust
- host all all ::1/128 trust
- host all all 10.0.0.0/8 scram-sha-256
- host all all 172.16.0.0/12 scram-sha-256
- host all all 192.168.0.0/16 scram-sha-256
- host all all 0.0.0.0/0 scram-sha-256
- host all all ::0/0 scram-sha-256
pg_ident:
# ripped from supabase/posgres/ansible/files/postgresql_config/pg_ident.conf
- supabase_map postgres postgres
- supabase_map gotrue supabase_auth_admin
- supabase_map postgrest authenticator
- supabase_map adminapi postgres
# ripped from supabase/posgres/ansible/files/postgresql_config/postgresql.conf
shared_preload_libraries:
[
pg_stat_statements,
pg_stat_monitor,
pgaudit,
plpgsql,
plpgsql_check,
pg_cron,
pg_net,
timescaledb,
auto_explain,
pg_tle,
supautils,
plan_filter,
pg_backtrace,
]
bootstrap:
initdb:
database: supabase
owner: supabase_adminConsidering how work is going, beyond this its going to be summer if I ever get a chance to make somethng decently shareable from this |
Beta Was this translation helpful? Give feedback.
-
Sorry for the late reply. No idea how this fell through my radar. I don't think I encountered anything like that I can't find any references to How are you building the postgres image? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @Towerful ! I have been playing around with the Basically the main difference from yours is adding the manually managed
I have a full, bitnami-inspired, helm chart that I am working on the finishing touches for that now supports everything with the The |
Beta Was this translation helpful? Give feedback.
-
|
Postgres 17.2 with Supabase picked extensions (including timescaledb, pgsodium, vault etc) + Barman backup installed for CloudNative PG: https://github.com/voltade/cnpg-supabase/blob/main/Dockerfile. Usage: apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: cnpg-supabase
spec:
# https://github.com/voltade/cnpg-supabase
imageName: ghcr.io/voltade/cnpg-supabase:17.2-2
instances: 3
...
postgresql:
synchronous:
method: any
number: 1
dataDurability: required
parameters:
# https://github.com/michelp/pgsodium?tab=readme-ov-file#disabling-view-and-trigger-generation
pgsodium.enable_event_trigger: "off"
# https://cloudnative-pg.io/documentation/current/postgresql_conf/#shared-preload-libraries
shared_preload_libraries:
- pg_cron
# - pgsodium # FATAL: The getkey script \"/usr/share/postgresql/17/extension/pgsodium_getkey\" does not exists.
- timescaledb
# https://cloudnative-pg.io/documentation/current/bootstrap/
bootstrap:
initdb:
database: app
owner: supabase_admin
secret:
name: cnpg-supabase-app
postInitApplicationSQLRefs:
secretRefs:
- name: supabase-initdb-sql
key: initdb.sql
- name: cnpg-supabase-app
key: post-initdb.sql
...The And the apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
metadata:
name: password-64-characters
spec:
length: 64
symbols: 0
noUpper: false
allowRepeat: true
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: cnpg-supabase-app
annotations:
argocd.argoproj.io/sync-wave: "0"
spec:
refreshInterval: 1h
dataFrom:
- sourceRef:
generatorRef:
apiVersion: generators.external-secrets.io/v1alpha1
kind: Password
name: password-64-characters
target:
name: cnpg-supabase-app
template:
# https://external-secrets.io/latest/guides/templating/#examples
engineVersion: v2
type: kubernetes.io/basic-auth
data:
dbname: app
host: cnpg-supabase-rw
password: "{{ .password }}"
port: "5432"
uri: postgresql://supabase_admin:{{ .password }}@cnpg-supabase-rw:5432/app
user: supabase_admin
username: supabase_admin
post-initdb.sql: |
alter user authenticator with password '{{ .password }}';
alter user supabase_auth_admin with password '{{ .password }}';
alter user supabase_storage_admin with password '{{ .password }}'; |
Beta Was this translation helpful? Give feedback.
-
|
That looks good. I think there is currently work for pg17 then i assume it will be easier to just add the stuff needed for cnpg https://github.com/supabase/postgres/tree/sam/conventional-pg17 |
Beta Was this translation helpful? Give feedback.
-
|
:-D. Also see my comment from a minute ago! |
Beta Was this translation helpful? Give feedback.
-
|
:D i use timescaledb as well so i need to get a version of that running but im waiting for their official pg17 conventional image and then ill see if i can get it set up but. But looks promising :) |
Beta Was this translation helpful? Give feedback.
-
Status CheckWhat is the actual status on deploying Supabase with CloudNativePG manged Postgres on Kubernetes? Is it reliable with Supabase features working and how scalable is Supabase itself in this context? Did you figure it out? |
Beta Was this translation helpful? Give feedback.
-
|
I tried rolling the deployment with StackGres. Issues suggest that their setup is outdated. Followed these resources: |
Beta Was this translation helpful? Give feedback.
-
|
I spent quite a while with |
Beta Was this translation helpful? Give feedback.
-
|
Hi all. EDIT: updated to use a pure upstream images with The following uses @Towerful 's basic method but also has support (ie, it works!) for This has been tested and works with 17.4.1.014 and 17.0.1.063-orioledb - you should be able to use Install Required secrets: You SHOULD DEFINITELY change these for any sort of production environment! apiVersion: v1
kind: Secret
metadata:
name: supabase-pgsodium
type: Opaque
# generated with
# head -c 32 /dev/urandom | od -A n -t x1 | tr -d ' \n'
stringData:
pgsodium_root.key: b67999ffdee15d83a80cfc3efbbffff7849690747b882ad1c37bc028d6f52d68
---
apiVersion: v1
kind: Secret
metadata:
name: supabase-jwt
type: Opaque
stringData:
service-key: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyAgCiAgICAicm9sZSI6ICJzZXJ2aWNlX3JvbGUiLAogICAgImlzcyI6ICJzdXBhYmFzZS1kZW1vIiwKICAgICJpYXQiOiAxNjQxNzY5MjAwLAogICAgImV4cCI6IDE3OTk1MzU2MDAKfQ.DaYlNEoUrrEn2Ig7tqibS-PHK5vgusbcbo7X36XVt4Q
anon-key: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyAgCiAgICAicm9sZSI6ICJhbm9uIiwKICAgICJpc3MiOiAic3VwYWJhc2UtZGVtbyIsCiAgICAiaWF0IjogMTY0MTc2OTIwMCwKICAgICJleHAiOiAxNzk5NTM1NjAwCn0.dc_X5iR_VP_qT0zsiyj_I_OZ2T9FtRU2BBNWN8Bu4GE
automaton-key: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiYXV0b21hdG9uIiwiaXNzIjoic3VwYWJhc2UtZGVtbyIsImlhdCI6MTY0MTc2OTIwMCwiZXhwIjoxNzk5NTM1NjAwfQ.-lor4AcPkFRgQdEKj4kzAjVCFVAZQLDGIxzWMvxWnWk
secret: your-super-secret-jwt-token-with-at-least-32-characters-long
expiry: "86400"
---
apiVersion: v1
kind: Secret
metadata:
name: supabase-postgres
labels:
cnpg.io/reload: "true"
type: kubernetes.io/basic-auth
stringData:
username: postgres
password: 'your-super-secret-and-long-postgres-password'
---
apiVersion: v1
kind: Secret
metadata:
name: supabase-superuser
labels:
cnpg.io/reload: "true"
type: kubernetes.io/basic-auth
stringData:
username: supabase_admin
password: 'your-super-secret-and-long-postgres-password'Cluster: apiVersion: v1
kind: ConfigMap
metadata:
name: pgsodium-getkey
data:
pgsodium_getkey.sh: |
#!/bin/bash
set -euo pipefail
KEY_FILE=/projected/postgresql-custom/pgsodium_root.key
if [[ ! -f "${KEY_FILE}" ]]; then
echo "Key file ${KEY_FILE} does not exist." >&2
exit 1
fi
cat $KEY_FILE
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: supabase-cluster
spec:
instances: 1
imageName: supabase/postgres:17.4.1.014
imagePullPolicy: IfNotPresent
superuserSecret:
name: supabase-postgres
postgresUID: 105 # this needs to be 101 after 17.4.1.062, definitely requires a new cluster, you can't update in place
postgresGID: 106 # this needs to be 102 after 17.4.1.062, definitely requires a new cluster, you can't update in place
storage:
size: 5Gi
projectedVolumeTemplate:
sources:
- secret:
name: supabase-pgsodium
items:
# available at /projected/postgresql-custom/pgsodium_root.key
- key: pgsodium_root.key
path: postgresql-custom/pgsodium_root.key
- configMap:
name: pgsodium-getkey
items:
# available at /projected/postgresql-custom/pgsodium_getkey.sh
- key: pgsodium_getkey.sh
path: postgresql-custom/pgsodium_getkey.sh
mode: 511
managed:
roles:
- name: supabase_admin
ensure: present
comment: "Supabase Admin"
login: true
superuser: true
passwordSecret:
name: supabase-superuser
inRoles:
- createdb
- createrole
- replication
- bypassrls
enableSuperuserAccess: true
postgresql:
parameters:
cron.database_name: supabase
pg_net.database_name: supabase
vault.getkey_script: '/projected/postgresql-custom/pgsodium_getkey.sh'
pgsodium.getkey_script: '/projected/postgresql-custom/pgsodium_getkey.sh'
auto_explain.log_min_duration: 10s
supautils.extensions_parameter_overrides: '{"pg_cron":{"schema":"pg_catalog"}}'
supautils.policy_grants: '{"postgres":["auth.audit_log_entries","auth.identities","auth.refresh_tokens","auth.sessions","auth.users","realtime.messages","storage.buckets","storage.migrations","storage.objects","storage.s3_multipart_uploads","storage.s3_multipart_uploads_parts"]}'
supautils.drop_trigger_grants: '{"postgres":["auth.audit_log_entries","auth.identities","auth.refresh_tokens","auth.sessions","auth.users","realtime.messages","storage.buckets","storage.migrations","storage.objects","storage.s3_multipart_uploads","storage.s3_multipart_uploads_parts"]}'
supautils.privileged_extensions: 'address_standardizer, address_standardizer_data_us, autoinc, bloom, btree_gin, btree_gist, citext, cube, dblink, dict_int, dict_xsyn, earthdistance, fuzzystrmatch, hstore, http, hypopg, index_advisor, insert_username, intarray, isn, ltree, moddatetime, orioledb, pg_buffercache, pg_cron, pg_graphql, pg_hashids, pg_jsonschema, pg_net, pg_prewarm, pg_repack, pg_stat_monitor, pg_stat_statements, pg_tle, pg_trgm, pg_walinspect, pgaudit, pgcrypto, pgjwt, pgroonga, pgroonga_database, pgrouting, pgrowlocks, pgsodium, pgstattuple, pgtap, plcoffee, pljava, plls, plpgsql_check, postgis, postgis_raster, postgis_sfcgal, postgis_tiger_geocoder, postgis_topology, postgres_fdw, refint, rum, seg, sslinfo, supabase_vault, supautils, tablefunc, tcn, tsm_system_rows, tsm_system_time, unaccent, uuid-ossp, vector, wrappers'
supautils.privileged_extensions_custom_scripts_path: '/etc/postgresql-custom/extension-custom-scripts'
supautils.privileged_extensions_superuser: 'supabase_admin'
supautils.privileged_role: 'postgres'
supautils.privileged_role_allowed_configs: 'auto_explain.*, log_lock_waits, log_min_duration_statement, log_min_messages, log_replication_commands, log_statement, log_temp_files, pg_net.batch_size, pg_net.ttl, pg_stat_statements.*, pgaudit.log, pgaudit.log_catalog, pgaudit.log_client, pgaudit.log_level, pgaudit.log_relation, pgaudit.log_rows, pgaudit.log_statement, pgaudit.log_statement_once, pgaudit.role, pgrst.*, plan_filter.*, safeupdate.enabled, session_replication_role, track_io_timing, wal_compression'
supautils.reserved_memberships: 'pg_read_server_files, pg_write_server_files, pg_execute_server_program, supabase_admin, supabase_auth_admin, supabase_storage_admin, supabase_read_only_user, supabase_realtime_admin, supabase_replication_admin, dashboard_user, pgbouncer, authenticator'
supautils.reserved_roles: 'supabase_admin, supabase_auth_admin, supabase_storage_admin, supabase_read_only_user, supabase_realtime_admin, supabase_replication_admin, dashboard_user, pgbouncer, service_role*, authenticator*, authenticated*, anon*'
pg_hba:
# ripped from supabase/posgres/ansible/files/postgresql_config/pg_hba.conf
- local all supabase_admin scram-sha-256
- local all all peer map=supabase_map
- host all all 127.0.0.1/32 trust
- host all all ::1/128 trust
- host all all 10.0.0.0/8 scram-sha-256
- host all all 172.16.0.0/12 scram-sha-256
- host all all 192.168.0.0/16 scram-sha-256
- host all all 0.0.0.0/0 scram-sha-256
- host all all ::0/0 scram-sha-256
pg_ident:
# ripped from supabase/posgres/ansible/files/postgresql_config/pg_ident.conf
- supabase_map postgres postgres
- supabase_map gotrue supabase_auth_admin
- supabase_map postgrest authenticator
- supabase_map adminapi postgres
# ripped from supabase/posgres/ansible/files/postgresql_config/postgresql.conf
shared_preload_libraries:
[
pg_stat_statements,
pg_stat_monitor,
pgaudit,
plpgsql,
plpgsql_check,
pg_cron,
pg_net,
# orioledb,
# timescaledb,
auto_explain,
pg_tle,
supautils,
pgsodium,
supabase_vault,
plan_filter
]
bootstrap:
initdb:
database: supabase
owner: supabase_adminHere is a migration script and job that doesn't require creating a docker image. You can adapt the version as necessary in the parameter to the There are some things that aren't working properly, like restricting the permissions on the UPDATE: |
Beta Was this translation helpful? Give feedback.
-
|
No idea sorry. It should be working. https://github.com/supabase/postgres/blob/develop/migrations/db/init-scripts/00000000000002-storage-schema.sql#L107 should be being copied and should be being executed. That's where that user is created. Are you trying to "re-supabase" an existing database dump or something? |
Beta Was this translation helpful? Give feedback.
-
It worked! Turns out I just needed to apply the migrations from supabase/storage. |
Beta Was this translation helpful? Give feedback.
-
|
@georgiev-anton , are you sure they don't get applied by the app itself on first start of the app? From above
|
Beta Was this translation helpful? Give feedback.
-
@AntonOfTheWoods Yes, you're really right! Thank you so much! but for some reason all the tables only have permissions for the Postgres role until I run this script after the migrations and Analytics still wouldn't launch until I created the schemas myself lol |
Beta Was this translation helpful? Give feedback.
-
|
@AntonOfTheWoods Sorry, I already went silent, please help me I cleaned everything up, but it stopped working in the same way. I removed the network drives, removed CNPG, removed Supabase, and I'm trying to reinstall everything, but it still doesn't work. My CNPG configMy migration |
Beta Was this translation helpful? Give feedback.
-
|
Upcoming releases suggest a dynamic approach to installing Postgresql extensions over CNPG standard image https://www.gabrielebartolini.it/articles/2025/03/the-immutable-future-of-postgresql-extensions-in-kubernetes-with-cloudnativepg/ |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, you really shouldn't believe "Unmodified Postgres with some useful plugins" from Supabase. I wouldn't call it a "lie" but it's certainly not accurate either. There are HEAPS of little tweaks that they make and lots and lots of migrations - many in different places (some in in the image, some from elsewhere, some run by the individual applications as they are started on their first run...) that have to be run it the right order. There is now absolutely ZERO documentation that isn't very out of date. It is an absolute dog's breakfast... And they really, truly don't care! I have personally gone through a world of pain trying to host/run Supabase apps without trying to use their image directly - that way you get the right versions of plugins and you at least have a chance of things working sorta like that do on the SAAS platform... |
Beta Was this translation helpful? Give feedback.
-
|
I see, this is expected as a strategy to keep users away from self-deploying their own instances. I've seen it in many "open source" projects where it is a struggle to get a self-managed deployment right - undocumented requirements, no up-to-date example configurations, or vague explanations if any, and many times some features are just not released. For example with Ory IAM stack it took me more than I anticipated to get authorization services working well in a Kubernetes cluster - Kratos (authentication) + Hydra (authorization OIDC) + Keto (access control) + Oathkeeper (gateway); without LLMs I would have abandoned attempting to self-host them. BTW, I decided not to use Supabase, but using the extensions directly such as PostgREST, PostGraphile to autogenerate API specs, etc. |
Beta Was this translation helpful? Give feedback.
-
|
I admit it's not easy, but there's no black magic. There are 3 parts for a CNPG compatible Supabase image: extensions, roles and migrations. For extensions, pgsodium and barman is not so straightforward, but refer to my earlier comment for more details. And now CNPG recommends run barman cloud as a plugin, so you don't need to install barman in the image anymore. We've been self hosting Supabase in production for more than 1 year, and now CNPG Supabase for few months already. My recommendation is to understand the code, build your own image and migrations from scratch. Believe the code not the documentations. |
Beta Was this translation helpful? Give feedback.
-
|
I'm the maintainer of https://github.com/stack-cli/stack-cli It's a Kubernetes Operator for running Supabase in Kubernetes namespaces and uses CNPG. Its running the |
Beta Was this translation helpful? Give feedback.
-
|
Looking at |
Beta Was this translation helpful? Give feedback.
-
|
The new kubernetes 1.35 (stable, 1.33+ feature-gated, as long as you have a compatible runtime) and cloudnative-pg 1.27+, along with postgres 18+, you can do dynamic loading of extensions via Here is a dockerfile for creating the container to mount: Which you can then make available and configure like the following: I haven't tested it very extensively yet but for the moment it seems to be working well with pg18 and the versions of the extensions available in |
Beta Was this translation helpful? Give feedback.
-
|
Hi @AntonOfTheWoods i have been following your journey with interest and am trying to get it ip and running with the image extensions so thanks for you work there. Have you tried using the new asymetric auth? |
Beta Was this translation helpful? Give feedback.
-
|
not yet @nimbit-software . Supabase make it very hard to keep up a self-hosted system by refusing to have any discipline in their individual projects (quality varies very widely) and a truly shameful documentation situation. Anyone who values their customers and customers' data would never rely on Supabase's SAAS offerings... because they obviously don't have any serious internal processes. So I only come and look at stuff every few months when I want to update something and decide to update everything. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
I have been running my Postgres database on Kubernetes with 'Cloud-Native Postgres Operator' https://cloudnative-pg.io/ with great success on the base postgres image. I'd like to try self hosting some subset of Supabase on Kubernetes, while still using the CNPG operator to manage my database, but haven't been able to figure it out.
Would be great to have some option on how to merge the supabase postgres image with Kubernetes operators like CNPG or Zalando, so a self-hosted Supabase, or really any Kubernetes Postgres implementattion, can take advantage of both the high availability management goodies of these operators, and also the baseline extensions and other functionality of this image.
Beta Was this translation helpful? Give feedback.
All reactions