Skip to content

Commit 0818cac

Browse files
committed
clean up docs
1 parent 23a4c9a commit 0818cac

8 files changed

Lines changed: 177 additions & 83 deletions

File tree

docs/api-reference.md

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,4 +42,28 @@ If you run the app directly in local development (no reverse proxy header inject
4242

4343
- `ALL /webhook/*`
4444

45-
Configured paths can include multiple segments (for example `/webhook/sample/01J0Z8X3GWBD9117Q9H4M2KCFP`).
45+
Configured paths must start with `/webhook/`. Typical format used by the UI:
46+
47+
```text
48+
/webhook/{routeName}/{ULID}
49+
```
50+
51+
Multiple path segments are supported (for example `/webhook/sample/01J0Z8X3GWBD9117Q9H4M2KCFP`).
52+
53+
## Data Model
54+
55+
Primary tables:
56+
- `ingress`
57+
- `egress`
58+
- `templates`
59+
- `route_configs`
60+
61+
Default local database file: `gateherald.db`
62+
63+
## Migration And Seeding Behavior
64+
65+
On app startup:
66+
1. Migration scripts run.
67+
2. Previously applied migrations are skipped using an internal script-run tracking table.
68+
3. Templates and route configs are loaded into runtime memory.
69+
4. Route config validation runs before server start.

docs/docker.md

Lines changed: 22 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@
44

55
Two containers share an internal Docker bridge network:
66

7-
- **`backend`** Node.js app in API-only mode (`SERVE_UI=false`, `FRONTEND_ONLY_API=true`). Publishes port 3000 for external webhook ingress.
8-
- **`frontend`** nginx serving the static UI and proxying admin API requests to the backend over the internal network. No published port for the backend API means direct external access to `/api/*` is blocked at the network level.
7+
- **`backend`** Node.js app in API-only mode (`SERVE_UI=false`, `FRONTEND_ONLY_API=true`). Publishes port 3000 for external webhook ingress.
8+
- **`frontend`** nginx serving the static UI and proxying admin API requests to the backend over the internal network. No published port for the backend API means direct external access to `/api/*` is blocked at the network level.
99

1010
Traffic flow:
1111
- Webhook providers → `backend:3000` (published)
@@ -15,14 +15,14 @@ Traffic flow:
1515

1616
The following files are referenced throughout these steps:
1717

18-
- `Dockerfile` backend container image
19-
- `docker-compose.yml` service definitions
20-
- `deploy/docker/nginx.conf` nginx config for the frontend container
21-
- `.dockerignore` excludes `.env.*` files, the database, and other unnecessary paths from the build context so secrets are not baked into the image
18+
- `Dockerfile` backend container image
19+
- `docker-compose.yml` service definitions
20+
- `deploy/docker/nginx.conf` nginx config for the frontend container
21+
- `.dockerignore` excludes `.env.*` files, the database, and other unnecessary paths from the build context so secrets are not baked into the image
2222

2323
All are included in the repo under their respective paths.
2424

25-
## Step 1 Build The UI CSS
25+
## Step 1) Build The UI CSS
2626

2727
The frontend container serves static files. Build the CSS before starting the stack:
2828

@@ -33,7 +33,7 @@ npm run build:css
3333

3434
This outputs `ui/dist/styles.css`, which the nginx container will serve directly.
3535

36-
## Step 2 Configure The nginx Auth Snippet
36+
## Step 2) Configure The nginx Auth Snippet
3737

3838
The frontend nginx config includes an auth snippet at `/etc/nginx/snippets/gateherald-admin-auth.conf`, mounted from `deploy/nginx/snippets/`. Choose one:
3939

@@ -52,7 +52,7 @@ htpasswd -c deploy/docker/.htpasswd-gateherald <username>
5252

5353
The `docker-compose.yml` mounts this file into the frontend container at `/etc/nginx/.htpasswd-gateherald`.
5454

55-
## Step 3 Set The Shared Secret
55+
## Step 3) Set The Shared Secret
5656

5757
`ADMIN_PROXY_SHARED_SECRET` is the shared credential between the nginx frontend and the Node backend. The frontend injects it as a request header; the backend verifies it before accepting any admin API call.
5858

@@ -64,11 +64,11 @@ node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"
6464

6565
Set `ADMIN_PROXY_SHARED_SECRET` to this value in your `.env.production` file (see Step 4). That is the only place it needs to go.
6666

67-
`deploy/docker/nginx.conf` uses `${ADMIN_PROXY_SHARED_SECRET}` as a placeholder. At startup, the nginx container reads the value from the environment and substitutes it before the config is loaded the secret is never written into any file in the repository.
67+
`deploy/docker/nginx.conf` uses `${ADMIN_PROXY_SHARED_SECRET}` as a placeholder. At startup, the nginx container reads the value from the environment and substitutes it before the config is loaded; the secret is never written into any file in the repository.
6868

6969
The frontend nginx container only receives `ADMIN_PROXY_SHARED_SECRET` from the environment. Other backend secrets (`DB_PASSWORD`, `API_TOKEN`, etc.) are passed only to the backend service via `env_file: .env.production` and are not exposed to the nginx container.
7070

71-
## Step 4 Configure Backend Environment
71+
## Step 4) Configure Backend Environment
7272

7373
Create a `.env.production` file in the project root (or pass variables directly to Compose). A minimal production config:
7474

@@ -85,15 +85,15 @@ Set `ALLOWED_ORIGINS` to the external URL where the frontend nginx is reachable.
8585

8686
For SQLite the default database location is used; it will be persisted via the named volume defined in `docker-compose.yml`. If using Postgres or MySQL instead, add the `DATABASE_URL` / `DB_*` vars from `.env.production.example`.
8787

88-
## Step 5 Run Migrations
88+
## Step 5) Run Migrations
8989

9090
Run database migrations once before starting the app for the first time, or after any upgrade that includes new migrations:
9191

9292
```bash
9393
docker compose run --rm backend node scripts/db-migrate.js
9494
```
9595

96-
## Step 6 Start The Stack
96+
## Step 6) Start The Stack
9797

9898
```bash
9999
docker compose up -d
@@ -108,17 +108,17 @@ To follow logs:
108108
docker compose logs -f
109109
```
110110

111-
## Step 7 TLS
111+
## Step 7) TLS
112112

113113
The `docker-compose.yml` exposes the frontend on HTTP port 8080. For HTTPS, put a TLS-terminating reverse proxy (another nginx, Caddy, Traefik, etc.) in front of port 8080 and set the appropriate `X-Forwarded-Proto` header. The app already reads this header for cookie `Secure` flag decisions.
114114

115-
Do not expose the backend container's port 3000 through TLS termination intended for admin users keep that path for webhook ingress only.
115+
Do not expose the backend container's port 3000 through TLS termination intended for admin users; keep that path for webhook ingress only.
116116

117-
## Optional Postgres/MySQL Database Container
117+
## Optional - Postgres/MySQL Database Container
118118

119119
By default the backend uses SQLite, persisted via a named Docker volume. This is sufficient for a single-node deployment. If you want a proper relational database with concurrent write support and a fully independent data lifecycle, add a `db` service to the Compose stack.
120120

121-
### 1 Install The Driver
121+
### 1) Install The Driver
122122

123123
The Postgres or MySQL driver must be present in the image. Add it as a production dependency before building:
124124

@@ -130,13 +130,13 @@ npm install pg pg-hstore
130130
npm install mysql2
131131
```
132132

133-
### 2 Add The db Service To docker-compose.yml
133+
### 2) Add The db Service To docker-compose.yml
134134

135135
Add a `db` service on the same internal network, with its own named volume. Update `backend` to depend on it:
136136

137137
```yaml
138138
volumes:
139-
gateherald-data: # remove or repurpose no longer used for SQLite
139+
gateherald-data: # remove or repurpose; no longer used for SQLite
140140
gateherald-db: # postgres data directory
141141

142142
services:
@@ -151,7 +151,7 @@ services:
151151
volumes:
152152
- gateherald-db:/var/lib/postgresql/data
153153
restart: unless-stopped
154-
# No 'ports' not reachable from outside Docker
154+
# No 'ports' - not reachable from outside Docker
155155

156156
backend:
157157
build: .
@@ -167,7 +167,7 @@ services:
167167
168168
Do not publish the `db` container's port. It only needs to be reachable from `backend` on the internal network.
169169

170-
### 3 Update Backend Environment
170+
### 3) Update Backend Environment
171171

172172
Remove the SQLite volume mount from `backend` and add database connection vars to `.env.production`:
173173

@@ -183,7 +183,7 @@ DB_SSL=false
183183

184184
`DB_HOST=db` resolves to the `db` container via Docker's internal DNS. `DB_SSL=false` is appropriate here because the connection stays on the internal bridge network; enable it if your setup routes through a TLS-capable proxy.
185185

186-
### 4 Run Migrations
186+
### 4) Run Migrations
187187

188188
Wait for Postgres to be ready before migrating. A simple one-off approach:
189189

docs/getting-started.md

Lines changed: 0 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -96,38 +96,6 @@ The UI files also support static hosting with relative paths:
9696
- `./template-builder.html`
9797
- `./dist/styles.css`
9898

99-
## Route Format
100-
101-
Configured route paths must start with:
102-
103-
```text
104-
/webhook/
105-
```
106-
107-
Typical format used by the UI:
108-
109-
```text
110-
/webhook/{routeName}/{ULID}
111-
```
112-
113-
## Data Model
114-
115-
Primary tables:
116-
- `ingress`
117-
- `egress`
118-
- `templates`
119-
- `route_configs`
120-
121-
Default local database file:
122-
- `gateherald.db`
123-
124-
## Migration And Seeding Behavior
125-
126-
On app startup:
127-
1. Migration scripts run
128-
2. Previously applied migrations are skipped using an internal script-run tracking table
129-
3. Templates and route configs are loaded into runtime memory
130-
4. Route config validation runs before server start
13199

132100
Seeder scripts are applied only when explicitly running:
133101

docs/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,6 @@ It includes:
3232
- [API Reference](api-reference.md)
3333
- [Split-Host Deployment](split-host-deployment.md)
3434
- [Docker Deployment](docker.md)
35-
- [Nginx Config Notes](nginx.md)
35+
- [Nginx, Auth & Hardening](nginx.md)
3636
- [Database In Production](database-production.md)
3737
- [Troubleshooting](troubleshooting.md)

docs/nginx.md

Lines changed: 106 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,112 @@
1-
# Nginx Config Notes
1+
# Nginx, Auth & Hardening
22

33
## OIDC Cutover Guidelines
44

5-
Use these steps to switch from Basic Auth to OIDC without changing protected route definitions.
6-
7-
1. Keep location blocks unchanged and only swap the auth include target content:
8-
- From: `deploy/nginx/snippets/gateherald-admin-auth-basic.conf`
9-
- To: `deploy/nginx/snippets/gateherald-admin-auth-oidc.conf`
10-
2. Enable `/oauth2/*` location on the internal UI host and route it to oauth2-proxy.
11-
3. Configure oauth2-proxy with your IdP issuer URL, client ID/secret, redirect URL, and session cookie settings.
12-
4. Keep external host restrictions in place (CIDR allowlist or stronger network controls). OIDC on internal host does not replace edge network restrictions.
13-
5. Verify behavior:
14-
- Unauthenticated requests to `/ui/*` and admin APIs redirect to IdP.
15-
- Authenticated user can load UI and perform template/config CRUD.
16-
- `/webhook/*` remains publicly reachable.
17-
- Admin APIs are unreachable from non-internal networks.
18-
6. Rotate/remove Basic Auth credentials (`/etc/nginx/.htpasswd-gateherald`) after successful cutover.
5+
OIDC auth is handled by [oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/) running alongside nginx. The nginx OIDC snippet uses nginx's `auth_request` module to delegate every protected request to oauth2-proxy before serving it. If oauth2-proxy says the user is not authenticated, nginx redirects to the IdP login page. The user logs in on the IdP's own site (not on the app's login page) and is redirected back once authenticated.
6+
7+
Note: the app has a separate built-in `FRONTEND_AUTH_ENABLED` basic login flow. That is independent of OIDC. When using OIDC, leave `FRONTEND_AUTH_ENABLED=false` (the default); there is no need for both.
8+
9+
### 1) Activate The OIDC Auth Snippet
10+
11+
Copy the OIDC snippet over the active auth file:
12+
13+
```bash
14+
cp deploy/nginx/snippets/gateherald-admin-auth-oidc.conf \
15+
deploy/nginx/snippets/gateherald-admin-auth.conf
16+
```
17+
18+
All protected location blocks already include this file and require no changes. The snippet swaps `auth_basic` for `auth_request /oauth2/auth`, which routes every protected request through oauth2-proxy.
19+
20+
### 2) Uncomment The `/oauth2/` Location Block
21+
22+
In your nginx config (bare-metal: `deploy/nginx/gateherald.conf` or `gateherald-split-hosts.conf` internal host; Docker: `deploy/docker/nginx.conf`), uncomment the oauth2-proxy location:
23+
24+
```nginx
25+
location ^~ /oauth2/ {
26+
proxy_pass http://127.0.0.1:4180;
27+
proxy_set_header Host $host;
28+
proxy_set_header X-Real-IP $remote_addr;
29+
proxy_set_header X-Scheme $scheme;
30+
}
31+
```
32+
33+
### 3) Configure And Run oauth2-proxy
34+
35+
oauth2-proxy must be reachable at `127.0.0.1:4180` on the same host as nginx (or adjust the `proxy_pass` address to match). The `--upstream=static://202` flag means oauth2-proxy only performs auth. It does not proxy the traffic itself (nginx handles that).
36+
37+
Generic OIDC provider:
38+
39+
```bash
40+
oauth2-proxy \
41+
--provider=oidc \
42+
--oidc-issuer-url=https://your-idp.example.com \
43+
--client-id=<client id> \
44+
--client-secret=<client secret> \
45+
--redirect-url=https://gateherald.internal/oauth2/callback \
46+
--email-domain=yourdomain.com \
47+
--cookie-secret=<32-byte random value> \
48+
--upstream=static://202 \
49+
--http-address=127.0.0.1:4180
50+
```
51+
52+
Generate a cookie secret:
53+
54+
```bash
55+
node -e "console.log(require('crypto').randomBytes(32).toString('base64'))"
56+
```
57+
58+
#### Microsoft Entra (Azure AD)
59+
60+
```bash
61+
oauth2-proxy \
62+
--provider=oidc \
63+
--oidc-issuer-url=https://login.microsoftonline.com/<tenant-id>/v2.0 \
64+
--client-id=<app registration client id> \
65+
--client-secret=<app registration client secret> \
66+
--redirect-url=https://gateherald.internal/oauth2/callback \
67+
--email-domain=yourdomain.com \
68+
--cookie-secret=<32-byte random value> \
69+
--upstream=static://202 \
70+
--http-address=127.0.0.1:4180
71+
```
72+
73+
Entra-specific setup:
74+
- Create an app registration in Entra ID.
75+
- Set the redirect URI to `https://gateherald.internal/oauth2/callback` under "Authentication".
76+
- Generate a client secret under "Certificates & secrets".
77+
- `--email-domain` restricts login to users from your org's domain. Set to `*` to allow any authenticated Entra user.
78+
- To restrict access to a specific Entra group, add `--allowed-group=<group object ID>` and grant the app registration `GroupMember.Read.All` permission (or configure the token to include group claims).
79+
80+
### 4) Reload Nginx
81+
82+
```bash
83+
# bare-metal
84+
nginx -s reload
85+
86+
# Docker
87+
docker compose restart frontend
88+
```
89+
90+
### 5) Verify
91+
92+
- Unauthenticated requests to `/ui/*` and admin APIs should redirect to the IdP login page.
93+
- Authenticated user can load the UI and perform template/config CRUD.
94+
- `/webhook/*` remains publicly reachable without auth.
95+
- Admin APIs are unreachable from non-internal networks.
96+
97+
### 6) Clean Up Basic Auth
98+
99+
Once verified, remove the htpasswd file:
100+
101+
```bash
102+
rm deploy/nginx/.htpasswd-gateherald # adjust path to your setup
103+
```
104+
105+
### Notes On The OIDC Snippet
106+
107+
`deploy/nginx/snippets/gateherald-admin-auth-oidc.conf` forwards `X-Forwarded-User` and `X-Forwarded-Email` headers from oauth2-proxy to the upstream backend. The app does not currently consume these headers, but they will be present on proxied requests. Audit them before relying on them for any downstream logic.
108+
109+
Keep external host restrictions (CIDR allowlist or network controls) in place even after enabling OIDC. OIDC on the internal/frontend host does not replace edge network restrictions on the backend host.
19110

20111
## Recommended Post-Cutover Hardening
21112

docs/split-host-deployment.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -45,10 +45,12 @@ When `FRONTEND_ONLY_API=true`, external callers should only reach `/webhook/*`.
4545

4646
For detailed Basic Auth/OIDC cutover and hardening guidance, see `docs/nginx.md`.
4747

48-
## Deployment Flow
48+
## Single Nginx Host
49+
50+
Use `deploy/nginx/gateherald.conf` when the UI and backend run on the same host, with nginx in front:
4951

5052
1. Run backend in API-only mode (`SERVE_UI=false`).
51-
2. Deploy static UI files from `ui/` to `/var/www/gateherald-ui` on your internal proxy host.
53+
2. Deploy static UI files from `ui/` to `/var/www/gateherald-ui` on the same host.
5254
3. Install `deploy/nginx/gateherald.conf` as your site config.
5355
4. Copy the active auth snippet to `/etc/nginx/snippets/gateherald-admin-auth.conf`.
5456
5. Replace `replace_me_with_admin_proxy_shared_secret` in admin API locations with the exact `ADMIN_PROXY_SHARED_SECRET` value.
@@ -71,7 +73,7 @@ Use `deploy/nginx/gateherald-split-hosts.conf` when UI and API/webhook are on di
7173

7274
Rollout checklist:
7375

74-
1. Set backend to API-only mode (`SERVE_UI=false`).
76+
1. Set backend to API-only mode (`SERVE_UI=false` in `.env.production` on the backend host).
7577
2. Deploy UI files to internal host (`/var/www/gateherald-ui`).
7678
3. Apply the internal server block from `deploy/nginx/gateherald-split-hosts.conf` on internal Nginx.
7779
4. Apply the external server block from `deploy/nginx/gateherald-split-hosts.conf` on external Nginx.

docs/troubleshooting.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,7 @@ npm run db:seed
166166

167167
**Problem**: All data (routes, templates, logs) has been deleted.
168168

169-
**Severity**: **CRITICAL** Data loss has occurred.
169+
**Severity**: **CRITICAL** - Data loss has occurred.
170170

171171
**Recovery**:
172172
1. **Immediately stop the application** to prevent further writes.
@@ -580,7 +580,7 @@ npm run db:seed
580580

581581
**Problem**: Modified a template in the UI, but old transformations still happen.
582582

583-
**Root Cause**: Same as routes templates may be cached.
583+
**Root Cause**: Same as routes - templates may be cached.
584584

585585
**Solution**:
586586
1. **Restart the app** to reload all templates:

0 commit comments

Comments
 (0)