|
| 1 | +# Backend Architecture |
| 2 | + |
| 3 | +This document is the canonical reference for backend service architecture. It is |
| 4 | +referenced by `backend/src/services/indexerService.ts` and |
| 5 | +`backend/src/services/soroban-indexer.service.ts` as the authoritative source for |
| 6 | +indexer ownership, SSE broadcast flow, and keeper-key authorization. |
| 7 | + |
| 8 | +For the full project-wide architecture (event type data flows, pause/resume |
| 9 | +timing, environment variables, and operational runbook) see |
| 10 | +[`docs/ARCHITECTURE.md`](../../docs/ARCHITECTURE.md). |
| 11 | + |
| 12 | +--- |
| 13 | + |
| 14 | +## Indexer Ownership Model |
| 15 | + |
| 16 | +Three files with overlapping names handle indexing and indexer management. Only |
| 17 | +one of them is the source of truth for stream state. |
| 18 | + |
| 19 | +| File | Role | Status | |
| 20 | +|------|------|--------| |
| 21 | +| `src/workers/soroban-event-worker.ts` (`SorobanEventWorker`) | **Source-of-truth indexer.** Polls Soroban RPC, decodes XDR events, persists `Stream` / `StreamEvent` rows, advances the `IndexerState` cursor, and broadcasts SSE updates. | **Active / source of truth.** Started by `src/workers/index.ts`. | |
| 22 | +| `src/services/soroban-indexer.service.ts` (`SorobanIndexerService`) | **Legacy indexer being phased out.** A simpler duplicate poller that writes to the same DB rows and races with the worker on the same `Stream` / `StreamEvent` records (issue #801). | **Legacy — do not extend.** Removal tracked with functional consolidation (issue #801). Started directly from `src/index.ts`. | |
| 23 | +| `src/services/indexerService.ts` | **Not an indexer at all.** Admin control-plane helpers (`getIndexerStatus`, `resetIndexer`, `replayFromLedger`) that read/reset the shared `IndexerState` cursor row and trigger the worker's poll loop. | **Active.** Name is misleading; kept alongside the legacy indexer above. | |
| 24 | + |
| 25 | +### Key Rules |
| 26 | + |
| 27 | +1. **When debugging indexing, read `src/workers/soroban-event-worker.ts` |
| 28 | + first.** It is the only file that persists canonical stream state. |
| 29 | +2. **Do not add new behavior to `soroban-indexer.service.ts`.** It exists only |
| 30 | + for backwards compatibility while the double-indexer race (issue #801) is |
| 31 | + consolidated. Mirror any changes in `SorobanEventWorker` instead. |
| 32 | +3. **`indexerService.ts` is control-plane only** — it never reads the chain; it |
| 33 | + manages the shared cursor and triggers replays. |
| 34 | + |
| 35 | +### The Dual-Indexer Race |
| 36 | + |
| 37 | +Both `SorobanEventWorker` and `SorobanIndexerService` poll the same Soroban RPC |
| 38 | +for the same contract events and write to the same `Stream` and `StreamEvent` |
| 39 | +rows. Because they run on independent timers, they can race: |
| 40 | + |
| 41 | +- Both may process the same ledger simultaneously. |
| 42 | +- Both write to the same `Stream` row (upsert), so the last writer wins — |
| 43 | + usually harmless for immutable fields but problematic for additive mutations |
| 44 | + like `withdrawnAmount` (issue #808). |
| 45 | +- `StreamEvent` dedup via `@@unique([transactionHash, eventType])` prevents |
| 46 | + duplicate event rows, but does **not** protect stream state mutations. |
| 47 | + |
| 48 | +**Mitigation:** Do not extend the legacy indexer. The consolidation (issue #801) |
| 49 | +will remove `SorobanIndexerService` entirely. |
| 50 | + |
| 51 | +### Naming Convention Plan |
| 52 | + |
| 53 | +The team convention is kebab-case with a `.service.ts` suffix. Once functional |
| 54 | +consolidation lands: |
| 55 | + |
| 56 | +| Current Name | Expected Future Name | |
| 57 | +|---|---| |
| 58 | +| `indexerService.ts` | `indexer.service.ts` | |
| 59 | +| `soroban-indexer.service.ts` | *(removed)* | |
| 60 | + |
| 61 | +--- |
| 62 | + |
| 63 | +## SSE Broadcast Flow |
| 64 | + |
| 65 | +The SSE (Server-Sent Events) subsystem delivers real-time contract event |
| 66 | +notifications to connected frontend clients. The full SSE architecture (scaling, |
| 67 | +memory, security) is documented in |
| 68 | +[`docs/SSE_ARCHITECTURE.md`](./SSE_ARCHITECTURE.md). |
| 69 | + |
| 70 | +### End-to-End Path |
| 71 | + |
| 72 | +``` |
| 73 | +Soroban RPC |
| 74 | + │ SorobanEventWorker polls for new contract events |
| 75 | + ▼ |
| 76 | +SorobanEventWorker (src/workers/soroban-event-worker.ts) |
| 77 | + │ decode XDR → upsert Stream → insert StreamEvent |
| 78 | + ▼ |
| 79 | +PostgreSQL (via Prisma) |
| 80 | + │ Stream + StreamEvent rows updated |
| 81 | + ▼ |
| 82 | +SSE broadcast (src/services/sse.service.ts) |
| 83 | + │ sseService.broadcastToStream(streamId, event, data) |
| 84 | + │ sseService.broadcastToUser(publicKey, event, data) |
| 85 | + │ sseService.broadcastToAdmin(event, data) |
| 86 | + │ |
| 87 | + ├──► [Single instance] Direct write to in-memory client registry |
| 88 | + │ |
| 89 | + └──► [Multi-instance] Redis Pub/Sub |
| 90 | + │ publish to sse:stream:<id>, sse:user:<address> |
| 91 | + ▼ |
| 92 | + All backend instances subscribe |
| 93 | + │ rebroadcast to local connected clients |
| 94 | + ▼ |
| 95 | + Frontend (useStreamEvents hook) |
| 96 | +``` |
| 97 | + |
| 98 | +### Broadcast Channels |
| 99 | + |
| 100 | +The worker uses three broadcast entry points depending on the event: |
| 101 | + |
| 102 | +| Method | When Used | Target Audience | |
| 103 | +|--------|-----------|-----------------| |
| 104 | +| `sseService.broadcastToStream(streamId, event, data)` | Stream lifecycle events (created, topped_up, withdrawn, cancelled, completed, paused, resumed) | Clients subscribed to that specific stream ID or `*` | |
| 105 | +| `sseService.broadcastToUser(publicKey, event, data)` | (Reserved for user-scoped events) | Clients subscribed to `user:<publicKey>` or `*` | |
| 106 | +| `sseService.broadcastToAdmin(event, data)` | Protocol-level events (fee_collected, fee_config_updated, admin_transferred) | The admin user identified by `ADMIN_PUBLIC_KEY` env var | |
| 107 | + |
| 108 | +### Multi-Instance Fanout |
| 109 | + |
| 110 | +When `REDIS_URL` is configured, broadcasts go through Redis Pub/Sub instead of |
| 111 | +direct in-memory writes: |
| 112 | + |
| 113 | +1. The originating instance publishes `{ event, data }` to |
| 114 | + `sse:stream:<id>` or `sse:user:<address>`. |
| 115 | +2. Every backend instance subscribes via `psubscribe('sse:stream:*', |
| 116 | + 'sse:user:*')` and rebroadcasts to its own local clients. |
| 117 | +3. This means events reach all connected clients regardless of which backend |
| 118 | + instance they are connected to. |
| 119 | + |
| 120 | +### Client Connection Limits |
| 121 | + |
| 122 | +| Limit | Default | Env Var | |
| 123 | +|-------|---------|---------| |
| 124 | +| Max SSE connections per server | 10,000 | `MAX_SSE_CONNECTIONS` | |
| 125 | +| Max connections per IP | 5 | Hardcoded | |
| 126 | +| Max connections per authenticated user | 10 | Hardcoded | |
| 127 | + |
| 128 | +Slow clients (write buffer ≥ 64 KB) are automatically dropped to protect |
| 129 | +throughput for healthy clients. |
| 130 | + |
| 131 | +--- |
| 132 | + |
| 133 | +## Keeper-Key Authorization Model |
| 134 | + |
| 135 | +FlowFi splits transaction signing into two categories: custodial (server-signed) |
| 136 | +and non-custodial (wallet-signed). The signing key determines who is responsible |
| 137 | +for the transaction. |
| 138 | + |
| 139 | +### Action Signing Matrix |
| 140 | + |
| 141 | +| Action | Signer | Mechanism | |
| 142 | +|--------|--------|-----------| |
| 143 | +| **Top-up** | Server (custodial) | Backend submits the transaction using `KEEPER_SECRET_KEY`. The frontend sends only the stream ID and amount. | |
| 144 | +| **Withdraw** | Wallet (non-custodial) | Frontend builds and signs the transaction via the connected wallet (Freighter). The backend simulate endpoint exists for fee estimation only. | |
| 145 | +| **Pause / Resume** | Wallet (non-custodial) | Same as withdraw — frontend-signed. Backend simulate endpoints exist for fee estimation but do not submit. | |
| 146 | +| **Create stream** | Wallet (non-custodial) | Frontend signs via wallet and submits directly to the Soroban RPC. | |
| 147 | + |
| 148 | +### The `KEEPER_SECRET_KEY` |
| 149 | + |
| 150 | +- Stored as an environment variable on the backend. |
| 151 | +- Loaded by `src/services/sorobanService.ts` via |
| 152 | + `process.env.KEEPER_SECRET_KEY`. |
| 153 | +- Used **exclusively** by the top-up flow. The `topUpStream` function builds |
| 154 | + the transaction, signs it with the keeper keypair, and submits it to the |
| 155 | + Soroban RPC. |
| 156 | +- If `KEEPER_SECRET_KEY` is not configured, `topUpStream` throws |
| 157 | + `'KEEPER_SECRET_KEY not configured'` and the request returns HTTP 500. |
| 158 | +- The cancel endpoint (`src/controllers/stream/cancel.ts`) also reads |
| 159 | + `KEEPER_SECRET_KEY` but only to check whether the server wallet is configured; |
| 160 | + the actual cancel transaction is wallet-signed by the sender. |
| 161 | + |
| 162 | +### Security Boundary |
| 163 | + |
| 164 | +> **Do not wire pause/resume/withdraw to a server-side submit path.** Only |
| 165 | +> `top-up` is intentionally custodial. All other mutating actions must be |
| 166 | +> wallet-signed by the user to preserve the non-custodial security model. |
| 167 | +
|
| 168 | +The keeper key is a server-side secret and is never exposed to the frontend. |
| 169 | +It lives exclusively in the backend's environment configuration. |
| 170 | + |
| 171 | +--- |
| 172 | + |
| 173 | +## Database Models |
| 174 | + |
| 175 | +For a quick reference of the models involved in indexing: |
| 176 | + |
| 177 | +| Model | Key Fields | Purpose | |
| 178 | +|-------|------------|---------| |
| 179 | +| `User` | `publicKey` | Stellar wallet addresses | |
| 180 | +| `Stream` | `streamId`, `sender`, `recipient`, `ratePerSecond`, `depositedAmount`, `withdrawnAmount`, `isActive` | Mirrors on-chain stream state | |
| 181 | +| `StreamEvent` | `streamId`, `eventType`, `transactionHash`, `ledgerSequence`, `timestamp` | Indexed on-chain events; unique on `(transactionHash, eventType)` | |
| 182 | +| `IndexerState` | `lastLedger`, `lastCursor` | Cursor for last successfully indexed ledger sequence | |
| 183 | + |
| 184 | +--- |
| 185 | + |
| 186 | +## Related Documentation |
| 187 | + |
| 188 | +- [Root Architecture](../../docs/ARCHITECTURE.md) — full project-wide architecture |
| 189 | +- [SSE Architecture](./SSE_ARCHITECTURE.md) — SSE scaling, security, operational runbook |
| 190 | +- [SSE Implementation](./SSE_IMPLEMENTATION.md) — client integration guide |
| 191 | +- [Authentication](./AUTHENTICATION.md) — SEP-10 + JWT auth flow |
0 commit comments