Skip to content

Commit a084505

Browse files
baijumclaude
baijum
and
claude
committed
docs: add server contract documenting platform-to-workflow interface
Defines the contract between bootstrap-server.sh and app deploy/preview workflows: directory layout, lifecycle, assumptions, Caddyfile generation, per-app credentials, and infrastructure scripts reference. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 5256992 commit a084505

3 files changed

Lines changed: 175 additions & 0 deletions

File tree

‎docs/ecosystem.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,8 @@ Each application gets:
8080
- Its own storage bucket
8181
- Its own container(s)
8282

83+
For details on how the server is structured and how app workflows interact with it, see [Server Contract](server-contract.md).
84+
8385
## Core Philosophy
8486

8587
### 1. Repository-Driven Deployment

‎docs/self-hosting.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,3 +132,5 @@ After deployment, your server runs:
132132
- Celery background workers
133133

134134
All managed through GitHub — push code to deploy updates.
135+
136+
For the full directory layout, lifecycle sequence, and workflow assumptions, see [Server Contract](server-contract.md).

‎docs/server-contract.md‎

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
# Server Contract
2+
3+
This document defines the contract between the platform infrastructure (bootstrapped by `bootstrap-server.sh`) and the app deployment workflows (`deploy.yml`, `preview.yml`). It explains the directory layout, lifecycle, and assumptions that connect them.
4+
5+
## Server Directory Layout
6+
7+
```
8+
/opt/platform/ # Platform root (created by bootstrap-server.sh)
9+
docker-compose.yml # Platform services (postgres, redis, minio, caddy, loki, promtail, grafana)
10+
.env # Platform credentials (POSTGRES_PASSWORD, MINIO_ROOT_*, GRAFANA_ADMIN_PASSWORD, ACME_EMAIL)
11+
.bootstrapped # Timestamp marker from last bootstrap run
12+
Caddyfile # Global Caddyfile — imports /etc/caddy/apps/*.caddy
13+
caddy-apps/ # Per-app Caddyfile fragments (written by deploy/preview workflows)
14+
<app-name>.caddy # Production route for an app
15+
<app-name>-pr-<N>.caddy # Preview route for a PR
16+
ops.caddy # Grafana route (created by bootstrap)
17+
credentials/ # Per-app credential files (created by create-app-credentials.sh)
18+
<app-name>.env # DB_USER, DB_PASSWORD, S3_ACCESS_KEY, S3_SECRET_KEY
19+
infrastructure/ # Ops scripts (copied from platform repo by bootstrap)
20+
backup-postgres.sh
21+
restore-postgres.sh
22+
check-alerts.sh
23+
update-images.sh
24+
create-app-credentials.sh
25+
usage-report.sh
26+
loki-config.yml # Loki configuration
27+
promtail-config.yml # Promtail configuration
28+
grafana/ # Grafana provisioning files and dashboards
29+
provisioning/datasources/
30+
provisioning/dashboards/
31+
dashboards/
32+
33+
/opt/apps/ # Application root (created by bootstrap-server.sh)
34+
<app-name>/ # Cloned app repo (created manually by admin)
35+
deploy/
36+
.env # App runtime config (created manually from env.template)
37+
docker-compose.yml # App services (joins towlion network)
38+
<app-name>-pr-<N>/ # Preview clone (created/destroyed by preview.yml)
39+
40+
/data/ # Persistent data volumes (created by bootstrap-server.sh)
41+
postgres/ # PostgreSQL data directory
42+
redis/ # Redis data directory
43+
minio/ # MinIO object storage
44+
caddy/data/ # Caddy TLS certificates
45+
caddy/config/ # Caddy config state
46+
loki/ # Loki log storage
47+
grafana/ # Grafana state
48+
backups/postgres/ # pg_dump backup files (7-day retention)
49+
```
50+
51+
## Bootstrap to Deploy Lifecycle
52+
53+
1. **Bootstrap the server** — Run `sudo bash infrastructure/bootstrap-server.sh` on a fresh Debian 12 machine. This creates the directory layout above, installs Docker, creates the `deploy` user, generates platform credentials, starts the 7 platform services, copies infrastructure scripts, and installs cron jobs.
54+
55+
2. **Configure DNS** — Point app domains and `*.preview.<domain>` to the server IP.
56+
57+
3. **Clone the app repo** — SSH in as `deploy` and clone the app to `/opt/apps/<name>/`.
58+
59+
4. **Create `deploy/.env`** — Copy `deploy/env.template` and fill in values (DATABASE_URL, S3 credentials, etc.).
60+
61+
5. **Provision per-app credentials (optional)** — Run `create-app-credentials.sh <name>` to create an isolated PostgreSQL user and MinIO bucket. Credentials are written to `/opt/platform/credentials/<name>.env`.
62+
63+
6. **Configure GitHub secrets** — Set `SERVER_HOST`, `SERVER_USER`, `SERVER_SSH_KEY`, and `APP_DOMAIN` on the app repo. Add `PREVIEW_DOMAIN` for preview environments.
64+
65+
7. **Push to main** — Triggers `deploy.yml`:
66+
- SSHes into the server as `deploy`
67+
- `cd /opt/apps/<name> && git pull origin main`
68+
- Creates the app database if it doesn't exist (via platform postgres)
69+
- Sources per-app credentials from `/opt/platform/credentials/<name>.env` (if present) and updates `deploy/.env` with isolated DB/S3 values
70+
- `docker compose -p <name> -f deploy/docker-compose.yml up -d --build`
71+
- Runs Alembic migrations inside the app container
72+
- Writes a Caddyfile to `/opt/platform/caddy-apps/<name>.caddy`
73+
- Reloads Caddy to pick up the new route
74+
75+
## App Workflow Server Assumptions
76+
77+
The `deploy.yml` and `preview.yml` workflows SSH into the server and depend on the following structure being in place:
78+
79+
| Path / Resource | Purpose | Created By |
80+
|---|---|---|
81+
| `/opt/platform/docker-compose.yml` | Platform services (postgres, redis, caddy, etc.) | `bootstrap-server.sh` |
82+
| `/opt/platform/.env` | `POSTGRES_PASSWORD` for database operations | `bootstrap-server.sh` |
83+
| `/opt/platform/caddy-apps/` | Writable directory for per-app Caddyfile fragments | `bootstrap-server.sh` |
84+
| `/opt/platform/credentials/<name>.env` | Per-app DB/S3 credentials (optional) | `create-app-credentials.sh` |
85+
| `/opt/apps/<name>/` | Cloned app repo with `deploy/.env` configured | Admin (manual) |
86+
| `towlion` Docker network | Shared network connecting platform services and app containers | `bootstrap-server.sh` |
87+
| `deploy` user | SSH user with Docker group membership | `bootstrap-server.sh` |
88+
89+
If any of these are missing, the workflow will fail. The bootstrap script is idempotent and can be re-run to restore missing structure.
90+
91+
## Infrastructure Scripts Reference
92+
93+
All scripts live in the platform repo under `infrastructure/` and are copied to `/opt/platform/infrastructure/` during bootstrap.
94+
95+
| Script | Purpose | Invocation |
96+
|---|---|---|
97+
| `bootstrap-server.sh` | Transform fresh Debian 12 into running platform | Manual (`sudo bash`) |
98+
| `verify-server.sh` | Read-only health check of server state | Manual (`bash`) |
99+
| `create-app-credentials.sh` | Provision per-app PostgreSQL user + MinIO bucket | Manual (`bash <script> <app-name>`) |
100+
| `backup-postgres.sh` | Per-database `pg_dump` with 7-day retention | Cron: daily at 02:00 |
101+
| `restore-postgres.sh` | Restore a database from backup | Manual (`bash <script>`) |
102+
| `check-alerts.sh` | Check container health, disk, memory; create GitHub Issues | Cron: every 5 minutes |
103+
| `update-images.sh` | Pull latest Docker images and recreate containers | Cron: weekly Sunday at 03:00 |
104+
| `usage-report.sh` | Generate 6-section resource usage report | Manual (`bash`) |
105+
106+
## Caddyfile Generation
107+
108+
The platform Caddyfile at `/opt/platform/Caddyfile` contains a single import directive:
109+
110+
```
111+
{
112+
email {$ACME_EMAIL:admin@localhost}
113+
}
114+
115+
import /etc/caddy/apps/*.caddy
116+
```
117+
118+
The `caddy-apps/` directory is bind-mounted into the Caddy container at `/etc/caddy/apps/`. App workflows write per-app `.caddy` files into this directory.
119+
120+
**Production** (`deploy.yml`) writes `/opt/platform/caddy-apps/<name>.caddy`:
121+
122+
```
123+
app.example.com {
124+
reverse_proxy <name>-app-1:8000
125+
}
126+
```
127+
128+
**Preview** (`preview.yml`) writes `/opt/platform/caddy-apps/<name>-pr-<N>.caddy`:
129+
130+
```
131+
pr-<N>.preview.example.com {
132+
reverse_proxy <name>-pr-<N>-app-1:8000
133+
}
134+
```
135+
136+
After writing the file, both workflows reload Caddy:
137+
138+
```bash
139+
docker compose -f /opt/platform/docker-compose.yml exec -T caddy caddy reload --config /etc/caddy/Caddyfile
140+
```
141+
142+
Preview cleanup removes the `.caddy` file and reloads Caddy again.
143+
144+
## Per-App Credentials
145+
146+
By default, apps connect to PostgreSQL as the `postgres` superuser (credentials from `deploy/.env`). For credential isolation, run:
147+
148+
```bash
149+
bash /opt/platform/infrastructure/create-app-credentials.sh <app-name>
150+
```
151+
152+
This creates:
153+
154+
- **PostgreSQL**: A dedicated user (`<app_name>_user`) with access restricted to `<app_name>_db`
155+
- **MinIO**: A dedicated user (`<app-name>-user`) with a scoped policy limiting access to the `<app-name>-uploads` bucket
156+
- **Credentials file**: `/opt/platform/credentials/<app-name>.env` containing `DB_USER`, `DB_PASSWORD`, `S3_ACCESS_KEY`, `S3_SECRET_KEY` (mode 600, owned by `deploy`)
157+
158+
On subsequent deploys, `deploy.yml` checks for this credentials file and, if found, updates `deploy/.env` with the per-app values via `sed`:
159+
160+
```bash
161+
CREDENTIALS_FILE="/opt/platform/credentials/${APP_NAME}.env"
162+
if [ -f "$CREDENTIALS_FILE" ]; then
163+
source "$CREDENTIALS_FILE"
164+
sed -i "s|^DATABASE_URL=.*|DATABASE_URL=postgresql://${DB_USER}:${DB_PASSWORD}@postgres:5432/${APP_DB}|" deploy/.env
165+
sed -i "s|^S3_ACCESS_KEY=.*|S3_ACCESS_KEY=${S3_ACCESS_KEY}|" deploy/.env
166+
sed -i "s|^S3_SECRET_KEY=.*|S3_SECRET_KEY=${S3_SECRET_KEY}|" deploy/.env
167+
sed -i "s|^S3_BUCKET=.*|S3_BUCKET=${APP_NAME}-uploads|" deploy/.env
168+
fi
169+
```
170+
171+
If no credentials file exists, the workflow falls back to whatever is already in `deploy/.env` and logs a warning.

0 commit comments

Comments
 (0)