Skip to content

Commit 2696a67

Browse files
ZimengXiongZimengXiong
committed
feat: opt-in s3 image storage with migration and progressive image hydration
1 parent cb3caa9 commit 2696a67

16 files changed

Lines changed: 2948 additions & 435 deletions

‎README.md‎

Lines changed: 58 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -121,7 +121,7 @@ Notes:
121121
# Installation
122122

123123
> [!CAUTION]
124-
> This is a pre-release feature and production-readiness depends on deployment controls:
124+
> This is a BETA deployment and production-readiness depends on deployment controls:
125125
> use TLS, trusted reverse proxy, fixed secrets, backups, and endpoint rate limits.
126126
127127
> [!CAUTION]
@@ -182,11 +182,12 @@ docker compose up -d
182182

183183
When running ExcaliDash behind Traefik, Nginx, or another reverse proxy, configure both containers so that API + WebSocket calls resolve correctly:
184184

185-
| Variable | Purpose |
186-
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
187-
| `FRONTEND_URL` | Backend allowed origin(s). Must match the public URL users access (for example `https://excalidash.example.com`). Supports comma-separated values for multiple addresses. |
188-
| `TRUST_PROXY` | Set to `1` when traffic passes through one trusted reverse-proxy hop (for example frontend nginx -> backend) and headers are sanitized. |
189-
| `BACKEND_URL` | Frontend container-to-backend target used by Nginx. Override when backend host differs from default service DNS/host. |
185+
| Variable | Purpose |
186+
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
187+
| `FRONTEND_URL` | Backend allowed origin(s). Must match the public URL users access (for example `https://excalidash.example.com`). Supports comma-separated values for multiple addresses. |
188+
| `TRUST_PROXY` | Set to `1` when traffic passes through one trusted reverse-proxy hop (for example frontend nginx -> backend) and headers are sanitized. |
189+
| `BACKEND_URL` | Frontend container-to-backend target used by Nginx. Override when backend host differs from default service DNS/host. |
190+
| `ENFORCE_HTTPS_REDIRECT` | When `FRONTEND_URL` uses `https://`, the backend automatically redirects plain-HTTP requests to HTTPS. Set to `false` if your outer gateway already enforces HTTPS and you want to disable the built-in redirect (avoids redirect loops when `X-Forwarded-Proto` is not forwarded). Default: `true`. |
190191

191192
```yaml
192193
# docker-compose.yml example
@@ -198,6 +199,9 @@ backend:
198199
- TRUST_PROXY=1
199200
# Or multiple URLs (comma-separated) for local + network access
200201
# - FRONTEND_URL=http://localhost:6767,http://192.168.1.100:6767,http://nas.local:6767
202+
# If your outer gateway enforces HTTPS and X-Forwarded-Proto is not forwarded,
203+
# disable the built-in redirect to prevent redirect loops:
204+
# - ENFORCE_HTTPS_REDIRECT=false
201205
frontend:
202206
environment:
203207
# For standard Docker Compose (default)
@@ -273,16 +277,39 @@ backend:
273277
- OIDC_CLIENT_ID=your-client-id
274278
# Optional for public clients; required for confidential clients
275279
# - OIDC_CLIENT_SECRET=your-client-secret
280+
# Optional token endpoint auth override (useful for some IdPs/HS setups)
281+
# - OIDC_TOKEN_ENDPOINT_AUTH_METHOD=client_secret_post
282+
# Optional override when your IdP client is configured for a non-default ID token alg
283+
# - OIDC_ID_TOKEN_SIGNED_RESPONSE_ALG=HS256
276284
- OIDC_REDIRECT_URI=https://excalidash.example.com/api/auth/oidc/callback
277285
- OIDC_SCOPES=openid profile email
278286
```
279287

288+
Quick preflight check (recommended before starting backend):
289+
290+
```bash
291+
cd backend
292+
npm run oidc:doctor
293+
```
294+
295+
Provider-specific env templates for existing IdPs:
296+
297+
- `backend/.env.oidc.keycloak.example`
298+
- `backend/.env.oidc.authentik.example`
299+
300+
Copy one to `backend/.env`, update issuer/client/redirect values, then run `npm run oidc:doctor`.
301+
280302
Notes:
281303

282304
| Topic | Notes |
283305
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
284306
| OIDC-only (`oidc_enforced`) | You typically do not use local bootstrap admin registration; first admin can be created through your IdP depending on config. |
285307
| Reverse proxy | Set `FRONTEND_URL` and `TRUST_PROXY` correctly or auth + websockets may fail. |
308+
| ID token algorithm | ExcaliDash defaults to `RS256`. If your IdP client is explicitly configured for another signed ID-token algorithm such as `HS256`, set `OIDC_ID_TOKEN_SIGNED_RESPONSE_ALG` to match that exact client setting. `none` is not allowed, and `HS*` requires `OIDC_CLIENT_SECRET`. |
309+
| Keycloak issuer format | Use realm issuer URL: `https://<keycloak-host>/realms/<realm>`. |
310+
| Authentik issuer format | Use provider issuer URL: `https://<authentik-host>/application/o/<provider-slug>/`. |
311+
| Authentik `email_verified` | If Authentik does not emit `email_verified=true`, either add the scope mapping or set `OIDC_REQUIRE_EMAIL_VERIFIED=false`. |
312+
| Redirect URI | Must be exact callback: `https://<excalidash-host>/api/auth/oidc/callback`. |
286313

287314
</details>
288315

@@ -336,19 +363,35 @@ docker compose -f docker-compose.oidc.yml down
336363

337364
Base values are documented in `backend/.env.example`. Common ones to care about:
338365

339-
| Variable | Default / Example | Description |
340-
| -------------- | ------------------------- | ----------------------------------------------------------------------------------- |
341-
| `DATABASE_URL` | `file:/app/prisma/dev.db` | SQLite file or external DB URL. |
342-
| `FRONTEND_URL` | `http://localhost:6767` | Allowed frontend origin(s), comma-separated for multiple entries. |
343-
| `TRUST_PROXY` | `false` | `false`, `true`, or hop count (for example `1`). |
344-
| `JWT_SECRET` | `change-this-secret...` | Recommended in production so sessions remain stable across restarts and migrations. |
345-
| `CSRF_SECRET` | `change-this-secret` | Recommended in production so CSRF validation remains stable across restarts. |
346-
| `AUTH_MODE` | `local` | `local`, `hybrid`, `oidc_enforced`. |
366+
| Variable | Default / Example | Description |
367+
| ------------------------ | ------------------------- | ----------------------------------------------------------------------------------- |
368+
| `DATABASE_URL` | `file:/app/prisma/dev.db` | SQLite file or external DB URL. |
369+
| `FRONTEND_URL` | `http://localhost:6767` | Allowed frontend origin(s), comma-separated for multiple entries. |
370+
| `TRUST_PROXY` | `false` | `false`, `true`, or hop count (for example `1`). |
371+
| `JWT_SECRET` | `change-this-secret...` | Recommended in production so sessions remain stable across restarts and migrations. |
372+
| `CSRF_SECRET` | `change-this-secret` | Recommended in production so CSRF validation remains stable across restarts. |
373+
| `AUTH_MODE` | `local` | `local`, `hybrid`, `oidc_enforced`. |
374+
| `ENFORCE_HTTPS_REDIRECT` | `true` | Set to `false` to disable the built-in HTTP→HTTPS redirect when your outer gateway handles it. |
375+
| `ENABLE_S3_IMAGE_STORAGE` | `false` | Opt-in: externalize embedded drawing image payloads to S3-compatible object storage. |
376+
| `S3_ENDPOINT` | `http://localhost:8333` | S3-compatible endpoint URL (for example SeaweedFS, MinIO, or cloud S3 endpoint). |
377+
| `S3_BUCKET` | `excalidash-images-test` | Bucket used for externalized drawing image blobs. |
378+
| `S3_ACCESS_KEY_ID` / `S3_SECRET_ACCESS_KEY` | _required when enabled_ | Credentials for object storage access. |
379+
| `S3_FORCE_PATH_STYLE` | `true` | Keep `true` for most self-hosted S3-compatible services (including local SeaweedFS). |
380+
| `S3_KEY_PREFIX` | `excalidash-images` | Prefix for object keys inside the configured bucket. |
381+
382+
When enabling external image storage on an existing instance, run:
383+
384+
```bash
385+
cd backend
386+
npm run storage:migrate-images
387+
```
347388

348389
</details>
349390

350391
# Development
351392

393+
For contributor workflow, `make dev` starts the app in local single-user mode so you can reproduce editor bugs without going through login/onboarding. Use `make dev-auth` if you need to test local auth or OIDC flows from your `backend/.env`.
394+
352395
<details>
353396
<summary>Clone the Repository</summary>
354397

@@ -460,7 +503,7 @@ Common flags:
460503
</details>
461504

462505
# Credits
463-
506+
If you find ExcaliDash useful, please consider [sponsoring](https://github.com/sponsors/ZimengXiong)
464507
- Example designs from:
465508
- https://github.com/Prakash-sa/system-design-ultimatum/tree/main
466509
- https://github.com/kitsteam/excalidraw-examples/tree/main

‎backend/.env.example‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,17 @@ CSRF_SECRET=change-this-secret-in-production
3232
# Optional GitHub token to increase API rate limits (public repo scopes are fine).
3333
# UPDATE_CHECK_GITHUB_TOKEN=
3434

35+
# Optional: externalize drawing image payloads to S3-compatible object storage.
36+
# This is opt-in and disabled by default.
37+
# ENABLE_S3_IMAGE_STORAGE=false
38+
# S3_ENDPOINT=http://localhost:8333
39+
# S3_REGION=us-east-1
40+
# S3_BUCKET=excalidash-images-test
41+
# S3_ACCESS_KEY_ID=change-me
42+
# S3_SECRET_ACCESS_KEY=change-me
43+
# S3_FORCE_PATH_STYLE=true
44+
# S3_KEY_PREFIX=excalidash-images
45+
3546
# One-time bootstrap setup code (first admin registration)
3647
# BOOTSTRAP_SETUP_CODE_TTL_MS=900000
3748
# BOOTSTRAP_SETUP_CODE_MAX_ATTEMPTS=10

0 commit comments

Comments
 (0)