Skip to content

Commit 0e07cd0

Browse files
JessyTsuiclaude
andauthored
Add API Playground, tracking refactor, and admin analytics (#118)
* Add API Playground, tracking refactor, admin analytics, and dashboard UX improvements ## API Playground - New /dashboard/playground page with Tavily-inspired UI - Left panel: query input, include answer toggle, collapsible additional fields (API key, max results, ranking mode, include summary, filters) - Right panel: Code view (Python/JS/Shell/Go with syntax highlighting) and Response view (Preview with clickable result cards + JSON with line numbers) - Per-result thumbs up/down feedback stored in playground_feedback table - Code copy uses full API key; display uses masked version ## Tracking Links Refactor - Permanent short_id on retrieval_units (deterministic sha256 hash) - Search no longer writes per-request tracking_links rows (zero DB writes) - Redirect handler dynamically builds target URL from current video data - Legacy tracking_links fallback for old links - query_logs.results_preview stores request-level result snapshots - search_surface field distinguishes API vs playground searches ## Admin Analytics - New /admin/analytics page with impression-aware CTR metrics - Content and creator performance leaderboards with min-impression thresholds - Search quality views: top queries, empty results, position CTR - Playground feedback analysis (explicitly labeled playground-only) ## API Key Management - Key prefix changed from cerul_sk_ to cerul_ (backwards compatible) - Raw API keys stored in database for later viewing - Auto-create Default key on user signup - Prevent deletion of last active key (403) - Tavily-style key display: masked by default, eye icon reveals for 5s, copy full key ## Dashboard UX - Playground added to sidebar with terminal icon - Top nav buttons use opaque backgrounds (fix dark overlay bleed-through) - Create key dialog: rounded corners, higher z-index - Dashboard layout: conditional header rendering - Fixed viewport-height playground layout with internal scrolling ## Migrations - 015: playground_feedback table - 016: retrieval_units.short_id + query_logs.results_preview - 017: query_logs.search_surface - 018: api_keys.raw_key storage ## Documentation - CLAUDE.md: Neon ops, proxy/workerd, architecture notes - docs/tracking-links-refactor.md - docs/admin-analytics-plan.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Harden API key handling and playground flows --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 513c205 commit 0e07cd0

42 files changed

Lines changed: 7349 additions & 413 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CLAUDE.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,50 @@ These are values that rarely change between environments.
2525
When adding a new parameter:
2626
1. Add to **all three** `.env` files if it belongs in `.env`, keeping keys in sync
2727
2. Only add to `config/base.yaml` if it's a pure algorithm/business parameter
28+
29+
## Database (Neon Serverless Postgres)
30+
31+
### Connection behavior
32+
- Neon free tier auto-suspends after inactivity; cold starts take 3-5 seconds.
33+
- Launch tier can disable auto-suspend ("Scale to zero") in Branches → Edit Compute.
34+
- Set `connectionTimeoutMillis: 60_000` in Node.js Pool config to survive cold starts.
35+
- Better Auth's `resetAuthDatabaseState()` calls `pool.end()` on timeout, poisoning all subsequent requests. Restart the Next.js dev server to recover.
36+
37+
### Migrations
38+
- Run with `./scripts/migrate-db.sh`. Use `--from` to target specific migrations.
39+
- **Never do large UPDATE backfills in a single transaction on Neon.** A single-transaction UPDATE over 100k+ rows will hang for hours on serverless compute. Always batch (5,000 rows per commit) using a shell loop with independent transactions.
40+
- Migration files: `db/migrations/NNN_name.sql`. Tracked in `schema_migrations` table.
41+
42+
## Local Development
43+
44+
### Proxy / Network
45+
- The local API runs on Cloudflare Workers dev (`wrangler dev` / `workerd`) which does **not** respect `http_proxy`/`https_proxy` env vars.
46+
- If your network requires a proxy (e.g. in China), enable Clash TUN mode or use `proxychains4` to wrap `wrangler dev`. Otherwise outbound requests to Gemini, Neon, etc. will silently time out.
47+
48+
### Dev startup
49+
- `./rebuild.sh` — install deps, sync `.dev.vars`, run migrations, build.
50+
- `./scripts/dev.sh` — start frontend (Next.js) + API (wrangler dev) together.
51+
- API binds to `API_BASE_URL` from `.env` (default `localhost:8000`). Frontend proxies dashboard/admin API calls through `/api/console/[...path]`.
52+
53+
## Architecture Notes
54+
55+
### API key format
56+
- Prefix: `cerul_` followed by 32 hex chars. Auth middleware pattern: `^cerul_[A-Za-z0-9]{32,}$`.
57+
- Keys are SHA256-hashed before storage. Raw key is never persisted and is shown only once at creation.
58+
- Users create API keys explicitly from the dashboard when needed.
59+
- Users cannot delete their last active key (backend returns 403).
60+
61+
### Tracking links
62+
- Each `retrieval_unit` has a permanent `short_id` (deterministic: `sha256(video_id:unit_type:unit_index)[:12]`), generated at index time.
63+
- Search results return `/v/{short_id}?req={request_id}&rank={rank}` — no per-search DB writes for redirect links.
64+
- Redirect handler looks up `retrieval_units` + `videos` to build the target URL dynamically (never stale).
65+
- Legacy `tracking_links` table still exists for old short links (fallback lookup).
66+
- Click tracking: `tracking_events` records `short_id`, `request_id`, `result_rank` from URL query params.
67+
68+
### Search surface attribution
69+
- `query_logs.search_surface` distinguishes `"api"` vs `"playground"` searches.
70+
- `query_logs.results_preview` (JSONB) stores a lightweight snapshot of returned results per request, used for dashboard history and as the impression denominator for CTR analytics.
71+
72+
### Console proxy
73+
- Frontend dashboard/admin API calls go through Next.js at `/api/console/[...path]`, which authenticates via Better Auth session, signs the request with HMAC, and forwards to the Hono API.
74+
- The dialog/modal z-index must be > 80 (header z-index) to properly overlay the top nav.

0 commit comments

Comments
 (0)