Skip to content

feat(db): migrate from Supabase to Neon Postgres + Drizzle ORM (phase 1) - #313

Merged
thisisouvik merged 3 commits into
mainfrom
feat/phase-1-neon-drizzle
Sep 12, 2026
Merged

feat(db): migrate from Supabase to Neon Postgres + Drizzle ORM (phase 1)#313
thisisouvik merged 3 commits into
mainfrom
feat/phase-1-neon-drizzle

Conversation

@thisisouvik

Copy link
Copy Markdown
Owner

Summary

Phase 1 of the rebuild: Supabase is gone. Postgres now lives on Neon and is accessed through Drizzle ORM; authentication is a first-party SEP-10 wallet session cookie; KYC files go to Vercel Blob. No Supabase package, env var, code path or doc reference remains.

What changed

Database

  • lib/db/schema.ts — single source of truth (18 tables, enums, indexes, relations). users replaces auth.users; profiles keeps the same id.
  • Two fixes the old schema needed anyway: loans.pool_id is nullable (direct marketplace loans never had a pool) and reputation_snapshots.score_breakdown exists (the daily job was already writing it).
  • drizzle/0000_init.sql (generated) + drizzle/0001_functions_and_triggers.sql (triggers, referral functions, the row-locked record_loan_funding). RLS/auth.uid() are gone — authorization is in the app layer.
  • lib/db/client.ts (HTTP driver + WebSocket pool), lib/db/{queries,rows,metadata}.ts.
  • npm run db:generate | db:migrate | db:studio.

Auth

  • Sign-in upserts users + profiles and sets an HttpOnly HS256 JWT cookie (jose, 7 days). Edge proxy verifies locally; requireAuthenticatedUser / requireApiUser / requireTradeVaultAdmin re-read the user row. New POST /api/auth/signout.
  • TRADE_VAULT_ADMIN_EMAILS now also accepts wallet addresses.

Storage

  • KYC documents → private Vercel Blob; admins view via GET /api/admin/kyc/document (streams after the admin check).

Ported

4 server actions · 34 API routes · 19 dashboard pages · 12 lib modules · liquidation keeper · webhook listener · e2e seed script. Liquidity updates are now SQL-side increments (no read-modify-write race); the borrower transaction feed lost an N+1.

Tests & CI

  • New __tests__/helpers/fake-db.ts; 8 test files re-mocked. All 19 pre-existing failures fixed — 659/659 green.
  • Real bugs found on the way: fetchJsonSafe hung on a fetch that ignored its abort signal; amount formatting read navigator.language (server/client locale mismatch → hydration errors); off-chain reputation tiers used the on-chain scale.
  • ci.yml now runs npm test, and applies migrations on main pushes once the DATABASE_URL secret exists.

Setup after merge (required for the app to work)

  1. Create a Neon project → add its pooled connection string as DATABASE_URL on Vercel and as a GitHub repository secret (DATABASE_URL, used by the migrate step and the backup job).
  2. Add SESSION_SECRET (openssl rand -base64 48) on Vercel.
  3. Create a Vercel Blob store → BLOB_READ_WRITE_TOKEN.
  4. Remove the old NEXT_PUBLIC_SUPABASE_*, SUPABASE_SERVICE_ROLE_KEY, SIWS_PASSWORD_SECRET env vars.
  5. Run npm run db:migrate once against the new database (CI does this automatically on the next main push once the secret exists).

Existing Supabase data is not migrated by this PR — this is a fresh testnet database. If you want the old rows carried over, say so and I'll write a one-off import.

Verification

tsc --noEmit ✅ · eslint ✅ (0 errors) · vitest 659/659 ✅ · next build

🤖 Generated with Claude Code

thisisouvik and others added 2 commits September 12, 2026 17:19
Vercel has rejected every deployment since 26 Aug with
"Hobby accounts are limited to daily cron jobs" because vercel.json
scheduled the liquidation and price-oracle crons on `* * * * *`.
Separately the contracts job has been red since 29 Aug because
usdc_lending_pool never compiled its tests.

Vercel
- vercel.json: every cron is now once a day (the Hobby ceiling), and the
  reputation-scoring cron is scheduled too (it had a route but no entry)
- new .github/workflows/keepers.yml runs the liquidation keeper and price
  oracle every 5 minutes by calling the same authenticated endpoints. It is
  a no-op until the KEEPER_BASE_URL and CRON_SECRET repo secrets exist.
- liquidation schedule test, route comment, .env.example and the
  keeper/oracle docs describe the new cadence

Contracts (usdc_lending_pool)
- `ledgers_elapsed` was u64 in a u128 saturating_mul (compile error)
- deposit used token.transfer_from, which needs a prior allowance the
  caller never grants; use transfer under the depositor's require_auth
  like the other contracts do
- tests: add lifetimes to create_token, replace catch_unwind (not
  UnwindSafe with an Env handle) with try_deposit, and raise ledger TTLs
  so the year-long yield tests do not archive contract storage
- commit the generated test snapshots like every other crate

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Name the elided lifetime on usdc_client's return type and prefix the
unused test binding, so `cargo clippy --all-targets -D warnings` passes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Sep 12, 2026

Copy link
Copy Markdown

Deployment failed for project trustlend-stellar with the following error:

Hobby accounts are limited to daily cron jobs. This cron expression (* * * * *) would run more than once per day. Upgrade to the Pro plan to unlock all Cron Jobs features on Vercel.

Learn More: https://vercel.link/3Fpeeb1

@coderabbitai

coderabbitai Bot commented Sep 12, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: e59f7307-0a73-42cc-b81c-41e8aad8532e


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…ase 1)

Replace Supabase (Postgres + Auth + Storage + RLS) with Neon Postgres via
Drizzle ORM, a first-party wallet session, and Vercel Blob. No Supabase
code, package, env var or doc reference remains.

Database
- lib/db/schema.ts is the single source of truth: 18 tables, all enums,
  indexes and relations. `users` replaces auth.users (one row per wallet);
  `profiles` keeps the same id. loans.pool_id is now nullable (direct
  marketplace loans never had a pool) and reputation_snapshots gains
  score_breakdown, which the daily recalculation was already writing.
- drizzle/0000_init.sql (generated) + drizzle/0001_functions_and_triggers.sql
  (updated_at triggers, reputation snapshot sync, referral code generation,
  record_referral / qualify_referral / settle_referral_payout, and the
  row-locked record_loan_funding for partial fills). auth.uid() checks are
  gone; authorization lives in the app layer.
- lib/db/client.ts: HTTP driver (getDb) for requests, WebSocket pool
  (getPooledDb) for transactions/scripts; both null when DATABASE_URL is
  unset so pages render empty states instead of crashing.
- lib/db/{queries,rows,metadata}.ts: shared reads and snake_case row mappers
  so the dashboard JSX did not have to change shape.
- npm run db:generate / db:migrate / db:studio (drizzle-kit).

Auth
- SEP-10 sign-in now upserts users + profiles and sets an HttpOnly
  HS256 JWT cookie (lib/auth/session-token.ts, `jose`, 7 days). The edge
  proxy verifies it locally; requireAuthenticatedUser / requireApiUser /
  requireTradeVaultAdmin re-read the user row. New POST /api/auth/signout.
- SessionUser replaces the Supabase user object (walletAddress, role,
  fullName, email). TRADE_VAULT_ADMIN_EMAILS accepts wallet addresses too.
- SIWS_PASSWORD_SECRET and the service-role key are gone; SESSION_SECRET
  is new.

Storage
- KYC uploads go to Vercel Blob as private files; admins view them through
  GET /api/admin/kyc/document (streams after the admin check).

Ported
- 4 server actions, 34 API routes, 19 dashboard pages, 12 lib modules, the
  liquidation keeper, the webhook listener and the e2e seed script (now
  plain `pg`). Read-modify-write liquidity updates became SQL-side
  increments; the borrower transactions feed lost its N+1 query.

Tests & CI
- __tests__/helpers/fake-db.ts fakes the Drizzle chain; 8 test files
  re-mocked on it. Also fixed the pre-existing failures: off-chain tier
  thresholds (scoring wrongly used the on-chain scale; /api/reputation
  too), the keeper tests' missing grace-period mock, a leaked mock queue,
  a swallowed HTTPS error in parse-deployment-url, a time-dependent
  analytics test, an invalid-base64 SEP-31 fixture, and formatXlmPrecise's
  contradictory expectation. 659/659 pass.
- Real bugs found on the way: fetchJsonSafe hung when a fetch ignored its
  abort signal (now raced against the deadline) and amount formatting read
  navigator.language (server/client locale mismatch → hydration errors;
  now always en-US).
- ci.yml runs `npm test` and applies migrations on main pushes when the
  DATABASE_URL secret exists. Supabase placeholders removed from CI, CSP,
  .env.example and the docs (getting-started, auth-siws, disaster-recovery,
  README).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@thisisouvik
thisisouvik force-pushed the feat/phase-1-neon-drizzle branch from 653e718 to bf49732 Compare September 12, 2026 16:09
@vercel

vercel Bot commented Sep 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
trustlend-stellar Ready Ready Preview Sep 12, 2026 4:10pm UTC

Comment on lines +24 to +53
name: Trigger cron endpoints
runs-on: ubuntu-latest
timeout-minutes: 5
env:
KEEPER_BASE_URL: ${{ secrets.KEEPER_BASE_URL }}
CRON_SECRET: ${{ secrets.CRON_SECRET }}
steps:
- name: Skip when secrets are not configured
id: gate
run: |
if [ -z "$KEEPER_BASE_URL" ] || [ -z "$CRON_SECRET" ]; then
echo "KEEPER_BASE_URL / CRON_SECRET not set — nothing to do."
echo "configured=false" >> "$GITHUB_OUTPUT"
else
echo "configured=true" >> "$GITHUB_OUTPUT"
fi

- name: Liquidation keeper
if: steps.gate.outputs.configured == 'true'
run: |
curl --fail-with-body --silent --show-error --max-time 120 \
-X POST "$KEEPER_BASE_URL/api/cron/liquidation" \
-H "Authorization: Bearer $CRON_SECRET"

- name: Price oracle
if: steps.gate.outputs.configured == 'true'
run: |
curl --fail-with-body --silent --show-error --max-time 120 \
-X POST "$KEEPER_BASE_URL/api/cron/price-oracle" \
-H "Authorization: Bearer $CRON_SECRET"
@thisisouvik
thisisouvik merged commit 63e0904 into main Sep 12, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants