Skip to content

Commit 74a815d

Browse files
author
The No Hands Company
committed
fix: remove all Replit dependencies; implement ACME TLS, audit log, dedup, Prometheus, Redis sessions
Replit removal — complete - Deleted lib/replit-auth-web/ (package, src/index.ts, src/use-auth.ts, tsconfig.json) - Created lib/auth-web/ as replacement (same useAuth hook, no Replit branding) - All @workspace/replit-auth-web imports → @workspace/auth-web - Removed @replit/vite-plugin-* from pnpm-workspace.yaml and package.json - Stripped // @replit comments from button.tsx and badge.tsx - auth.ts: ISSUER_URL no longer defaults to https://replit.com/oidc Hard error thrown if ISSUER_URL or OIDC_CLIENT_ID is not set REPL_ID → OIDC_CLIENT_ID throughout - .env.example: all Replit language removed - COOKIE_SECRET fallback now throws in production instead of using 'change-me' - Deleted replit.md - Cleaned ARCHITECTURE.md, CLAUDE.md, SELF_HOSTING.md, DEPLOYMENT.md, README.md ACME / Let's Encrypt TLS (lib/acme.ts) - Real acme-client implementation: account key persistence, HTTP-01 challenge, CSR generation, cert written to ACME_CERT_DIR/<domain>/fullchain.pem + privkey.pem - X509Certificate expiry parsing (native Node.js, no deps) - 12-hour auto-renewal scheduler, renews when <30 days remain - startAcmeRenewalScheduler() / stopAcmeRenewalScheduler() in index.ts lifecycle - tls.ts route: old stub replaced with real implementation ACME_ENABLED=false returns Caddy/certbot instructions (not an error) ACME_ENABLED=true kicks off provisioning async, responds immediately Admin audit log (lib/auditLog.ts) - auditLog(req, action, target, metadata) — never throws, logs failures instead - Sensitive field redaction: password, tokenHash, privateKey, secretKey, etc. - admin_audit_log table: actor_id, actor_email, action, target_type, target_id, metadata JSONB, ip_address, user_agent, created_at - PATCH /admin/node now logs before/after state - GET /api/admin/audit-log: paginated, requireAdmin protected - lib/db/src/schema/audit.ts + schema/index.ts export - Migration SQL includes table + 3 indexes File content deduplication (deploy.ts) - content_hash column on site_files (SHA-256 hex, nullable for legacy rows) - Register-file route: if contentHash matches existing row, reuses objectPath No new object is stored in S3 for identical files - Response includes deduplicated: true when a match is found - DB schema: contentHash column + index on site_files Prometheus metrics (lib/metrics.ts) - prom-client with custom registry (no global pollution) - collectDefaultMetrics with fedhost_nodejs_ prefix - Counters: http_requests_total, deployments_total, federation_syncs_total, analytics_hits_total, storage_operations_total - Histograms: http_request_duration_seconds (11 buckets) - Gauges: http_active_requests, sites_total, federation_peers_total, cache_entries, sync_queue_depth - GET /metrics: optional METRICS_TOKEN bearer auth - metricsMiddleware: route normalisation to prevent label cardinality explosion - Mounted in app.ts before all other middleware Redis session store (auth.ts) - createSession: writes to Redis (EX SESSION_TTL_SECONDS) + PostgreSQL - getSession: Redis-first; on miss, falls back to PostgreSQL and re-populates Redis - destroySession: removes from both Redis and PostgreSQL - connect-redis + prom-client added to package.json Other - ROADMAP.md: all 5 features updated to ✅ - migration: content_hash + admin_audit_log added
1 parent 1d7ba2a commit 74a815d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+827
-979
lines changed

.env.example

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -29,14 +29,16 @@ PORT=8080
2929
LOG_LEVEL=info
3030

3131
# ── Authentication (required) ─────────────────────────────────────────────────
32-
# OIDC issuer URL. Defaults to Replit OIDC. Replace with your own provider:
32+
# OIDC issuer URL — your identity provider's discovery endpoint
33+
# Examples:
3334
# Authentik: https://auth.yourdomain.com/application/o/fedhost/
3435
# Keycloak: https://auth.yourdomain.com/realms/fedhost
3536
# Auth0: https://your-tenant.us.auth0.com/
36-
ISSUER_URL=https://replit.com/oidc
37+
# Dex: https://dex.yourdomain.com
38+
ISSUER_URL=https://auth.yourdomain.com/application/o/fedhost/
3739

38-
# OIDC client ID (the "REPL_ID" in Replit, or your client_id in other providers)
39-
REPL_ID=your-oidc-client-id
40+
# OIDC client ID — registered with your identity provider
41+
OIDC_CLIENT_ID=your-client-id
4042

4143
# ── Node Identity (optional) ──────────────────────────────────────────────────
4244
# Display name for this node in the federation network
@@ -59,7 +61,7 @@ BANDWIDTH_CAPACITY_GB=1000
5961
# ── Public Domain (required for federation) ───────────────────────────────────
6062
# The public hostname where this node is reachable by other nodes.
6163
# Other nodes will attempt to handshake with you at https://<PUBLIC_DOMAIN>/
62-
REPLIT_DEV_DOMAIN=node.yourdomain.com
64+
PUBLIC_DOMAIN=node.yourdomain.com
6365

6466
# Comma-separated list of allowed CORS origins (defaults to * in development)
6567
ALLOWED_ORIGINS=https://node.yourdomain.com
@@ -100,7 +102,7 @@ DB_CONNECT_TIMEOUT_MS=5000
100102
# ── Security ──────────────────────────────────────────────────────────────────
101103
# Secret for HMAC-signing unlock cookies for password-protected sites.
102104
# Use a long random string: openssl rand -base64 32
103-
# If not set, falls back to REPL_ID (insecure for production).
105+
# Required in production. Generate: openssl rand -base64 32
104106
COOKIE_SECRET=your-secret-here
105107

106108
# ── Admin / RBAC ──────────────────────────────────────────────────────────────
@@ -111,7 +113,7 @@ ADMIN_USER_IDS=
111113

112114
# ── Object Storage (S3-compatible) ───────────────────────────────────────────
113115
# Set these to use AWS S3, Cloudflare R2, MinIO, Backblaze B2, etc.
114-
# If not set, falls back to Replit Object Storage (development only).
116+
115117
#
116118
# For MinIO (Docker Compose): OBJECT_STORAGE_ENDPOINT=http://minio:9000
117119
# For Cloudflare R2: OBJECT_STORAGE_ENDPOINT=https://<account>.r2.cloudflarestorage.com
@@ -129,9 +131,9 @@ DOMAIN_CACHE_TTL_MS=300000
129131
DOMAIN_CACHE_MAX=10000
130132
FILE_CACHE_MAX=50000
131133

132-
# ── Object Storage — S3/MinIO (replaces Replit-specific storage) ───────────────
134+
# ── Object Storage — S3/MinIO ────────────────────────────────────────────────────
133135
# Set these to use any S3-compatible provider (AWS, Cloudflare R2, MinIO, etc.)
134-
# If not set, falls back to Replit Object Storage sidecar (Replit deployments only).
136+
# Required when using S3/MinIO storage.
135137
OBJECT_STORAGE_ENDPOINT=http://localhost:9000
136138
OBJECT_STORAGE_ACCESS_KEY=your-access-key
137139
OBJECT_STORAGE_SECRET_KEY=your-secret-key

CLAUDE.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ artifacts/federated-hosting ← React + Vite frontend (port 25231 in dev)
6363
artifacts/api-server ← Express 5 API (port 8080 in dev)
6464
6565
├── lib/db ← Drizzle ORM + PostgreSQL
66-
├── lib/objectStorage ← Replit/S3-compatible object store
66+
├── lib/objectStorage ← deprecated shim → use storageProvider.ts
6767
├── federation peers ← Other nodes (Ed25519-verified)
6868
└── background jobs ← healthMonitor, analyticsFlush, gossipPusher
6969
```
@@ -75,7 +75,7 @@ lib/
7575
api-spec/ ← OpenAPI 3.1 spec + Orval codegen config
7676
api-client-react/ ← Generated React Query hooks (from OpenAPI)
7777
api-zod/ ← Generated Zod schemas (from OpenAPI)
78-
replit-auth-web/ ← useAuth() hook
78+
auth-web/ ← useAuth() hook
7979
object-storage-web/ ← Upload component + useUpload hook
8080
8181
artifacts/
@@ -190,7 +190,7 @@ A full Playwright E2E suite is a high-priority item on the roadmap. Until it exi
190190

191191
- **Don't break the federation protocol**`FEDERATION.md` is a public spec; changes must be backwards-compatible or versioned
192192
- **Don't remove Drizzle transactions** from the deploy endpoint — partial deployments corrupt site state
193-
- **Don't add Replit-specific dependencies** without an abstraction layer — the project must remain self-hostable
193+
- **Don't add legacy dependencies** without an abstraction layer — the project must remain self-hostable
194194
- **Don't return stack traces in production** — the global error handler already strips them; don't bypass it
195195
- **Don't log private keys** — the pino logger has `privateKey` and `password` in its redaction list, but don't work around it
196196
- **Don't use `Promise.all` for peer operations** — always `Promise.allSettled`; a dead peer must never crash a user-facing request

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ lib/db/ Drizzle ORM schema + migrations
7171
lib/api-spec/ OpenAPI 3.1 specification (source of truth)
7272
lib/api-zod/ Auto-generated Zod validators (do not edit)
7373
lib/api-client-react/ Auto-generated React Query hooks (do not edit)
74-
lib/integrations/ Replit Auth + Object Storage wrappers
74+
lib/integrations/ OIDC Auth + Object Storage wrappers
7575
docs/ Architecture, API, and other documentation
7676
scripts/ Utility and seed scripts
7777
```

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ A production-grade **federated website hosting service** where users deploy stat
1313

1414
Federated Hosting lets users:
1515

16-
1. **Log in** via Replit Auth (OpenID Connect)
16+
1. **Log in** via OIDC Auth (OpenID Connect)
1717
2. **Upload** website files (HTML, CSS, JS, images, fonts) to object storage
1818
3. **Deploy** their site — gets a unique domain, version-tracked, atomically committed
1919
4. **Replicate** — the deploy is automatically mirrored to all active federation peers via signed `site_sync` events
@@ -30,7 +30,7 @@ Independent operators run nodes. Each node has an **Ed25519 cryptographic identi
3030
- [Node.js 24+](https://nodejs.org/)
3131
- [pnpm 10+](https://pnpm.io/)
3232
- PostgreSQL database (connection string in `DATABASE_URL`)
33-
- Replit object storage (or S3-compatible store)
33+
- S3-compatible object storage (or S3-compatible store)
3434

3535
### Install
3636

@@ -79,7 +79,7 @@ pnpm run build
7979
## Features
8080

8181
### Phase 1 — Auth + File Serving
82-
- Replit Auth (OIDC + PKCE) — users own their sites
82+
- OIDC Auth (OpenID Connect + PKCE) — users own their sites
8383
- Presigned URL upload flow to object storage
8484
- Host-header site serving — `your-domain.com` routes to the right files
8585
- My Sites dashboard + drag-and-drop deploy UI
@@ -181,7 +181,7 @@ Browser → Federated Hosting UI (Vite/React)
181181
│ ├── api-zod/ # Generated Zod schemas (Orval)
182182
│ ├── db/ # Drizzle ORM schema + migrations
183183
│ │ └── src/schema/ # nodes, sites, deployments, federation, auth
184-
│ └── integrations/ # replit-auth-web, object-storage
184+
│ └── integrations/ # auth-web, object-storage
185185
├── scripts/ # Seed + utility scripts
186186
├── docs/ # Architecture, API, Contributing, Security
187187
├── LICENSE

ROADMAP.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ A living document tracking what is built, what is in progress, and what must be
113113

114114
| Feature | Status | Notes |
115115
|---|---|---|
116-
| ACME/Let's Encrypt automation | | Stub — issues HTTP-01 token only; full ACME flow not implemented |
116+
| ACME/Let's Encrypt automation | | Real acme-client: account key, HTTP-01 challenge, cert written to disk, 12h auto-renewal |
117117
| TLS via Caddy (documented) || Caddy instruction accurate |
118118
| Geographic routing (closest-node redirect) || Region inference + 302 redirect |
119119
| Geo routing: latency probing || Mentioned in code comment, not implemented |
@@ -137,12 +137,12 @@ A living document tracking what is built, what is in progress, and what must be
137137
| 11 | Replay attack window | MEDIUM | ✅ Fixed — 5-minute timestamp check |
138138
| 12 | i18n async loading | LOW | ✅ Fixed — i18next-http-backend, HTTP-fetched |
139139
| 13 | Federation sync retry | MEDIUM | ✅ Fixed — exponential backoff queue, 10 max attempts |
140-
| 14 | ACME TLS automation | MEDIUM | ❌ Still a stub — use Caddy for now |
141-
| 15 | Admin audit logging | MEDIUM | 📋 Not yet built |
142-
| 16 | Content deduplication | LOW | 📋 Not yet built |
143-
| 17 | Prometheus metrics | LOW | 📋 Not yet built |
140+
| 14 | ACME TLS automation | MEDIUM | ✅ Full acme-client implementation |
141+
| 15 | Admin audit logging | MEDIUM | ✅ auditLog(), admin_audit_log table, GET /api/admin/audit-log |
142+
| 16 | Content deduplication | LOW | ✅ content_hash column, dedup on register-file, objectPath reuse |
143+
| 17 | Prometheus metrics | LOW | ✅ prom-client, 12 metrics, GET /metrics, metricsMiddleware |
144144
| 18 | Gossip in-memory per-instance | LOW | ⚠️ Multi-instance gossip not Redis-shared |
145-
| 19 | Session store (multi-instance) | MEDIUM | ⚠️ PostgreSQL sessions work; not Redis-backed |
145+
| 19 | Session store (multi-instance) | MEDIUM | ✅ Redis-first with PostgreSQL fallback; cross-instance session sharing |
146146

147147
---
148148

SECURITY.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ All inter-node pings and sync notifications are signed with the originating node
4040

4141
### Authentication
4242

43-
Users authenticate via **Replit Auth** (OpenID Connect with PKCE). Session tokens are stored in the database (not in JWTs), are HttpOnly + Secure cookies, and expire after a configurable TTL.
43+
Users authenticate via **OIDC Auth** (OpenID Connect with PKCE). Session tokens are stored in the database (not in JWTs), are HttpOnly + Secure cookies, and expire after a configurable TTL.
4444

4545
### Rate limiting
4646

artifacts/api-server/package.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,9 @@
1414
"@google-cloud/storage": "^7.19.0",
1515
"@workspace/api-zod": "workspace:*",
1616
"@workspace/db": "workspace:*",
17+
"acme-client": "^5.4.0",
1718
"compression": "^1.8.1",
19+
"connect-redis": "^8.0.1",
1820
"cookie-parser": "^1.4.7",
1921
"cors": "^2",
2022
"drizzle-orm": "catalog:",
@@ -27,6 +29,7 @@
2729
"openid-client": "^6.8.2",
2830
"pino": "^10.3.1",
2931
"pino-http": "^11.0.0",
32+
"prom-client": "^15.1.3",
3033
"rate-limit-redis": "^4.2.0",
3134
"uuid": "^13.0.0"
3235
},

artifacts/api-server/src/app.ts

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ import { tokenAuthMiddleware } from "./middleware/tokenAuth";
1010
import { globalErrorHandler, notFoundHandler } from "./middleware/errorHandler";
1111
import { globalLimiter, speedLimiter } from "./middleware/rateLimiter";
1212
import router from "./routes";
13-
import { hostRouter } from "./middleware/hostRouter";
13+
import { metricsMiddleware, registry } from "./lib/metrics";
1414
import { geoRoutingMiddleware } from "./lib/geoRouting";
1515
import { db, nodesTable, siteDeploymentsTable } from "@workspace/db";
1616
import { eq } from "drizzle-orm";
@@ -25,7 +25,7 @@ const allowedOrigins = process.env.ALLOWED_ORIGINS
2525

2626
const app: Express = express();
2727

28-
// ── Trust proxy (Replit / reverse-proxy environments) ─────────────────────────
28+
// ── Trust reverse proxy headers (X-Forwarded-For, X-Real-IP) ────────────────────
2929
// Required so express-rate-limit can correctly read X-Forwarded-For
3030
app.set("trust proxy", 1);
3131

@@ -97,6 +97,21 @@ app.use(cookieParser());
9797
app.use(globalLimiter);
9898
app.use(speedLimiter);
9999

100+
// ── Prometheus metrics instrumentation ────────────────────────────────────────
101+
app.use(metricsMiddleware);
102+
103+
// GET /metrics — Prometheus scrape endpoint.
104+
// Set METRICS_TOKEN to protect it; without it metrics are open (bind to localhost recommended).
105+
app.get("/metrics", async (req: Request, res: Response) => {
106+
const token = process.env.METRICS_TOKEN;
107+
if (token && req.headers.authorization !== `Bearer ${token}`) {
108+
res.status(401).json({ error: "Unauthorized" });
109+
return;
110+
}
111+
res.setHeader("Content-Type", registry.contentType);
112+
res.end(await registry.metrics());
113+
});
114+
100115
// ── Auth middleware ────────────────────────────────────────────────────────────
101116
app.use(authMiddleware);
102117

@@ -122,7 +137,7 @@ app.get("/.well-known/federation", async (_req: Request, res: Response, next: Ne
122137
res.json({
123138
protocol: "fedhost/1.0",
124139
name: localNode?.name ?? "Federated Hosting Node",
125-
domain: localNode?.domain ?? process.env.REPLIT_DEV_DOMAIN ?? "unknown",
140+
domain: localNode?.domain ?? process.env.PUBLIC_DOMAIN ?? "unknown",
126141
region: localNode?.region ?? "unknown",
127142
publicKey: localNode?.publicKey ? stripPemHeaders(localNode.publicKey) : null,
128143
nodeCount: allNodes.length,

artifacts/api-server/src/index.ts

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ import { startAnalyticsFlusher, stopAnalyticsFlusher } from "./lib/analyticsFlus
77
import { startGossipPusher, stopGossipPusher } from "./routes/gossip";
88
import { getRedisClient, closeRedis } from "./lib/redis";
99
import { startSyncRetryQueue, stopSyncRetryQueue } from "./lib/syncRetryQueue";
10+
import { startAcmeRenewalScheduler, stopAcmeRenewalScheduler } from "./lib/acme";
1011
import { db, sessionsTable } from "@workspace/db";
1112
import { lt } from "drizzle-orm";
1213
import { seedBundledSites } from "./lib/seedBundledSites";
@@ -22,14 +23,14 @@ async function ensureLocalNode(): Promise<void> {
2223
const [existing] = await db.select().from(nodesTable).where(eq(nodesTable.isLocalNode, 1));
2324

2425
if (!existing) {
25-
const domain = process.env.REPLIT_DEV_DOMAIN ?? `localhost:${port}`;
26+
const domain = process.env.PUBLIC_DOMAIN ?? `localhost:${port}`;
2627
const { publicKey, privateKey } = generateKeyPair();
2728
const [created] = await db
2829
.insert(nodesTable)
2930
.values({
3031
name: process.env.NODE_NAME ?? "Primary Node",
3132
domain,
32-
region: process.env.NODE_REGION ?? "Replit-Cloud",
33+
region: process.env.NODE_REGION ?? "unknown",
3334
operatorName: process.env.OPERATOR_NAME ?? "Node Operator",
3435
operatorEmail: process.env.OPERATOR_EMAIL ?? "admin@example.com",
3536
storageCapacityGb: Number(process.env.STORAGE_CAPACITY_GB ?? 100),
@@ -63,6 +64,7 @@ function gracefulShutdown(server: http.Server, signal: string): void {
6364
stopAnalyticsFlusher();
6465
stopGossipPusher();
6566
stopSyncRetryQueue();
67+
stopAcmeRenewalScheduler();
6668
await closeRedis();
6769
const { pool } = await import("@workspace/db");
6870
await pool.end();
@@ -97,6 +99,7 @@ ensureLocalNode()
9799
startAnalyticsFlusher();
98100
startGossipPusher();
99101
startSyncRetryQueue();
102+
startAcmeRenewalScheduler();
100103

101104
// Initialise Redis connection (optional — falls back to in-memory if not configured)
102105
const redis = getRedisClient();

0 commit comments

Comments
 (0)