feat: stockage des médias en PostgreSQL (alternative au S3) (#482)

* feat: add PostgreSQL database storage backend for media files

Add a new `db_storage` app that provides a custom Django Storage backend
storing files directly in PostgreSQL. This offers a persistent storage
alternative for PaaS deployments (Scalingo, Heroku, etc.) where the
filesystem is ephemeral and S3 is not available.

Activated via SF_USE_DB_STORAGE=1. Priority: S3 > DB Storage > FileSystem.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add S3 to database storage migration command

Adds `migrate_s3_to_db` management command that:
- Downloads all files from S3 bucket and stores them as StoredFile entries
- Updates hardcoded S3 URLs in Wagtail Revision content_json
- Scans URLField/CharField and RichTextField on all models for S3 URLs
- Supports --dry-run, --skip-files, and --skip-urls options

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add DB storage documentation and update README

Add comprehensive documentation for the PostgreSQL media storage feature
including activation guide, S3 migration steps, and architecture overview.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve ruff E501 and black formatting issues

Fix line length violations (>119 chars) in test_migrate_s3.py by
extracting S3 env dict to module constant. Remove unused imports
and fix black formatting in migrate_s3_to_db.py.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unused BytesIO import in storage.py

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: conditionner db_storage app et URL à SF_USE_DB_STORAGE

L'app db_storage et sa route /db-storage/ ne sont ajoutées que si
SF_USE_DB_STORAGE=True, suivant le même pattern que SF_USE_WHITENOISE
et PROCONNECT_ACTIVATED.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: ruff F405 noqa in settings_test, revert getattr MEDIA_ROOT

- Add noqa: F405 for INSTALLED_APPS references from star import
- Revert getattr(settings, "MEDIA_ROOT", "") to settings.MEDIA_ROOT
  since Django provides a default empty string in global_settings

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add 1 GB disclaimer and DB-to-S3 reverse migration

- Add warning that DB storage is not recommended beyond 1 GB of media
  in settings.py, .env.example, and docs/db-storage.md
- Add migrate_db_to_s3 management command to transfer stored files
  from PostgreSQL back to an S3 bucket (with --dry-run support and
  skip-if-exists logic)
- Add 5 tests for the new command

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: apply black formatting to migrate_db_to_s3 files

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: clarify storage selection order and add VPS use case

Replace misleading "Priority" wording with "Selection order" to
distinguish code precedence from recommendation. Add VPS to
filesystem use cases in documentation table.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Attempt fix

* fix: handle Revision.content as JSONField in migrate_s3_to_db

Wagtail 3+ changed Revision.content_json (TextField) to
Revision.content (JSONField/dict). The previous "Attempt fix"
renamed the field but still called .replace() on a dict, which
would fail at runtime.

This fix serializes the JSON content to string for URL
search/replace, then parses back. Also adds a regression test.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix issue that prevented Images admin view to show

* Changed order of routes so that the db_storage works on a dev machine

* Allow the use of MinIO for local testing of S3

* Fix migrate_db_to_s3 command

* Necessary so that the migration script works

* Add the missing migrate_db_to_files script

* Update release number

* Update scalingo deployment file

* Updates after review

---------

Co-authored-by: Chaïb Martinez <chaibax@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Author: 3 people

.env.example

@@ -15,6 +15,11 @@ USE_DOCKER=0
 USE_UV=0
 # SF_USE_WHITENOISE: Set 1 or True to use Whitenoise
 SF_USE_WHITENOISE=0
+# SF_USE_DB_STORAGE: Set 1 or True to store media files in PostgreSQL
+# instead of the filesystem. Useful for PaaS with ephemeral filesystems.
+# /!\ Not recommended beyond 1 GB of media — prefer S3 for larger volumes.
+# Selection order: S3_HOST wins if set, then SF_USE_DB_STORAGE, then filesystem (default)
+SF_USE_DB_STORAGE=0
 
 DATABASE_NAME=djdb
 DATABASE_USER=dju
@@ -31,6 +36,9 @@ S3_BUCKET_NAME=
 S3_BUCKET_REGION=eu-west-3
 # S3_LOCATION: If the S3 bucket is shared, add a unique folder name
 S3_LOCATION=
+# S3_PUBLIC_HOST: If the S3 bucket has a different public address (for example when using MinIO), specify it
+# This disables the signature on the URLs, so the bucket must allow public read access.
+S3_PUBLIC_HOST=
 
 # (Optional) FORCE_SCRIPT_NAME: used to allow to the site to be served under a sub-path
 FORCE_SCRIPT_NAME=

.gitignore

@@ -145,6 +145,9 @@ dmypy.json
 .VSCode
 .vscode/
 
+# AI stuff
+.claude/
+
 # Project-specific stuff
 config.json
 cron.json

ONBOARDING.md

@@ -113,6 +113,48 @@ psql -c "CREATE USER sitesconformes WITH CREATEDB LOGIN PASSWORD 'votre_mot_de_p
 psql -c "CREATE DATABASE sitesconformes OWNER sitesconformes;" -U postgres
 ```
 
+### Utilisation de MinIO
+
+[MinIO](https://min.io/) permet de simuler un stockage objet compatible S3 en local, utile pour tester la configuration de production sans avoir besoin d'un vrai bucket S3.
+
+#### Lancer MinIO
+
+```sh
+docker run -d \
+  --name minio \
+  -p 9000:9000 \
+  -p 9001:9001 \
+  -v ~/minio-data:/data \
+  -e MINIO_ROOT_USER=admin \
+  -e MINIO_ROOT_PASSWORD=password123 \
+  quay.io/minio/minio server /data --console-address ":9001"
+```
+
+#### Créer le bucket
+
+Accéder à la console MinIO sur <http://localhost:9001> (identifiants : `admin` / `password123`), puis créer un bucket (par exemple `sc-local`).
+
+Pour éviter l'utilisation d'URLs signées (plus simple en local), rendre le bucket public : _Buckets → sc-local → Anonymous → Add Access Rule → Prefix `/`, Access `readonly`_.
+
+#### Variables d'environnement
+
+Ajouter les variables suivantes dans le fichier `.env` :
+
+```sh
+S3_HOST=host.docker.internal:9000
+S3_PUBLIC_HOST=localhost:9000
+S3_PROTOCOL=http
+S3_KEY_ID=admin
+S3_KEY_SECRET=password123
+S3_BUCKET_NAME=sc-local
+S3_BUCKET_REGION=
+S3_LOCATION=medias/
+```
+
+> **Note :** C'est la variable `S3_HOST` qui active le stockage S3 dans l'application. Sans elle, les médias seront stockés sur le système de fichiers local, quelle que soit la configuration MinIO.
+
+Alternativement, il est également possible de passer par un stockage des fichiers directement dans la base PostgreSQL, cf. [documentation](./docs/db-storage.md)
+
 ## Fonctionnement depuis un sous-répertoire
 
 Lorsque la variable `FORCE_SCRIPT_NAME` est configurée, le site tourne dans un sous-répertoire, fonctionnalité qui n’est pas gérée par le serveur de développement de base de Django (`runserver`).

README.md

@@ -48,6 +48,7 @@ Sites Conformes est développé en utilisant le framework [Django](https://www.d
 - **[django-dsfr](https://github.com/numerique-gouv/django-dsfr)** : Permet d’utiliser facilement le [système de design de l’État](https://www.systeme-de-design.gouv.fr/) dans des templates Django.
 - **events** : Similaire à `blog`, mais permet de gérer des événements et des pages de calendrier, ainsi que les exports iCal correspondants.
 - **forms** : implémentation du [module de création de formulaire](https://docs.wagtail.org/en/stable/reference/contrib/forms/index.html) de Wagtail, par exemple pour les pages de contact. Volontairement assez limité (suffisant pour un formulaire de contact mais pas beaucoup plus), pour les cas complexes il vaut mieux privilégier l’intégration de [Démarches simplifiées](https://www.demarches-simplifiees.fr) ou de [Grist](https://grist.numerique.gouv.fr/).
+- **db_storage** : stockage des médias en base de données PostgreSQL, alternative au S3 pour les PaaS avec filesystem éphémère (cf. [documentation](./docs/db-storage.md))
 - **proconnect** : permet la connexion via [ProConnect](https://www.proconnect.gouv.fr/)
 
 ### Structure du dépôt

config/settings.py

@@ -58,6 +58,12 @@ def getenv_bool(key: str, default: bool):
 # Allow enabling WhiteNoise via an environment variable (disabled by default)
 SF_USE_WHITENOISE = getenv_bool("SF_USE_WHITENOISE", False)
 
+# Allow storing media files in PostgreSQL instead of the filesystem (disabled by default)
+# Useful for PaaS deployments with ephemeral filesystems (Scalingo, Heroku, etc.)
+# /!\ Not recommended beyond 1 GB of media — prefer S3 for larger volumes.
+# Selection order: S3_HOST wins if set, then SF_USE_DB_STORAGE, then filesystem (default)
+SF_USE_DB_STORAGE = getenv_bool("SF_USE_DB_STORAGE", False)
+
 INTERNAL_IPS = [
     "127.0.0.1",
 ]
@@ -107,6 +113,9 @@ def getenv_bool(key: str, default: bool):
     "wagtail.admin",
 ]
 
+if SF_USE_DB_STORAGE:
+    INSTALLED_APPS.insert(-1, "db_storage")
+
 if SF_USE_WHITENOISE:
     INSTALLED_APPS.insert(0, "whitenoise.runserver_nostatic")
 
@@ -272,22 +281,47 @@ def show_toolbar(request):
 # https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html
 
 if os.getenv("S3_HOST"):
-    endpoint_url = f"{os.getenv('S3_PROTOCOL', 'https')}://{os.getenv('S3_HOST')}"
+    protocol = os.getenv("S3_PROTOCOL", "https")
+    endpoint_url = f"{protocol}://{os.getenv('S3_HOST')}"
+    bucket_name = os.getenv("S3_BUCKET_NAME", "set-bucket-name")
+    public_host = os.getenv("S3_PUBLIC_HOST", "")
+
+    options = {
+        "bucket_name": bucket_name,
+        "access_key": os.getenv("S3_KEY_ID", "123"),
+        "secret_key": os.getenv("S3_KEY_SECRET", "secret"),
+        "endpoint_url": endpoint_url,
+        "region_name": os.getenv("S3_BUCKET_REGION", "fr"),
+        "file_overwrite": False,
+        "location": os.getenv("S3_LOCATION", ""),
+    }
+
+    if public_host:
+        # Presigned URLs bind the signature to the endpoint hostname, so when the
+        # internal endpoint differs from the public one, signing must be disabled.
+        # The bucket is appended to custom_domain to produce path-style URLs for MinIO.
+        options["custom_domain"] = f"{public_host}/{bucket_name}"
+        options["url_protocol"] = f"{protocol}:"
+        options["querystring_auth"] = False
+        public_endpoint = f"{protocol}://{public_host}"
+    else:
+        public_endpoint = endpoint_url
 
     STORAGES["default"] = {
         "BACKEND": "storages.backends.s3.S3Storage",
-        "OPTIONS": {
-            "bucket_name": os.getenv("S3_BUCKET_NAME", "set-bucket-name"),
-            "access_key": os.getenv("S3_KEY_ID", "123"),
-            "secret_key": os.getenv("S3_KEY_SECRET", "secret"),
-            "endpoint_url": endpoint_url,
-            "region_name": os.getenv("S3_BUCKET_REGION", "fr"),
-            "file_overwrite": False,
-            "location": os.getenv("S3_LOCATION", ""),
-        },
+        "OPTIONS": options,
     }
 
-    MEDIA_URL = f"{endpoint_url}/"
+    MEDIA_URL = f"{public_endpoint}/"
+elif SF_USE_DB_STORAGE:
+    STORAGES["default"] = {
+        "BACKEND": "db_storage.storage.DatabaseStorage",
+    }
+    MEDIA_URL = os.getenv("MEDIA_URL", "db-storage/")
+    MEDIA_ROOT = os.path.join(BASE_DIR, os.getenv("MEDIA_ROOT", ""))
+
+    if FORCE_SCRIPT_NAME and not MEDIA_URL.startswith(FORCE_SCRIPT_NAME):
+        MEDIA_URL = f"{FORCE_SCRIPT_NAME}/{MEDIA_URL}"
 else:
     STORAGES["default"] = {
         "BACKEND": "django.core.files.storage.FileSystemStorage",

config/settings_test.py

@@ -1,5 +1,10 @@
 from config.settings import *  # NOSONAR # noqa: F401,F403
 
+# Enable db_storage for testing its functionality
+SF_USE_DB_STORAGE = True
+if "db_storage" not in INSTALLED_APPS:  # noqa: F405
+    INSTALLED_APPS.insert(-1, "db_storage")  # noqa: F405
+
 WHITENOISE_AUTOREFRESH = True
 WHITENOISE_MANIFEST_STRICT = False
 

config/urls.py

@@ -22,7 +22,14 @@
         "robots.txt",
         TemplateView.as_view(template_name="robots.txt", content_type="text/plain"),
     ),
-] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
+]
+
+if settings.SF_USE_DB_STORAGE:
+    urlpatterns += [
+        path("db-storage/", include("db_storage.urls")),
+    ]
+
+urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
 
 # Only add this on a dev machine, outside of tests
 if not settings.TESTING and settings.DEBUG and "localhost" in settings.HOST_URL:

db_storage/__init__.py

@@ -0,0 +1,6 @@
+from django.apps import AppConfig
+
+
+class DbStorageConfig(AppConfig):
+    default_auto_field = "django.db.models.BigAutoField"
+    name = "db_storage"