Skip to content

Latest commit

 

History

History
199 lines (146 loc) · 8.72 KB

File metadata and controls

199 lines (146 loc) · 8.72 KB

Architecture

Service mesh

graph TB
    subgraph "UserFacing"
        UD[User Dashboard<br/>React 18 + Vite]
        AD[Admin Console<br/>React 18 + Vite]
        BFF[bff<br/>aggregator + auth proxy]
    end

    subgraph "Domain services"
        Auth[auth<br/>JWT, OTP, sessions]
        Esc[escrow<br/>state machine + transitions]
        Wal[wallet<br/>custodial keys + transfers]
        Mil[milestone<br/>milestone state machine]
        Inq[inquiry<br/>dispute lifecycle]
        Comp[compliance<br/>KYC + sanctions]
        Conn[connect<br/>external integrations]
    end

    subgraph "Infrastructure services"
        Lis[listenerEngine<br/>multiChain polling]
        Notif[notification<br/>email + push fanOut]
        Ledg[ledger<br/>canonical accounting]
        Rep[reporting<br/>analytics aggregates]
        Adm[admin<br/>admin API]
    end

    subgraph "Data plane"
        PG[(PostgreSQL<br/>perService schemas)]
        Redis[(Redis<br/>sessions + Socket.IO adapter)]
        Kafka[Kafka<br/>event bus]
    end

    subgraph "External"
        KMS[AWS KMS]
        S3[S3]
        SM[Secrets Manager]
        Mail[Resend / Nodemailer]
        Chains[ETH / BSC / Polygon / Solana / Tron]
    end

    UD --> BFF
    AD --> BFF
    BFF --> Auth
    BFF --> Esc
    BFF --> Wal
    BFF --> Mil
    BFF --> Inq

    Esc --> Kafka
    Wal --> Kafka
    Lis --> Kafka
    Mil --> Kafka
    Inq --> Kafka

    Notif -.subscribes.-> Kafka
    Ledg -.subscribes.-> Kafka
    Esc -.subscribes.-> Kafka
    Wal -.subscribes.-> Kafka

    Wal --> KMS
    Wal --> Chains
    Lis --> Chains

    Esc --> PG
    Wal --> PG
    Lis --> PG
    Auth --> PG
    Mil --> PG
    Inq --> PG
    Ledg --> PG

    Auth --> Redis
    BFF --> Redis

    Notif --> Mail
    Comp --> S3
    Conn --> SM
Loading

12 NestJS services + 1 BFF + 2 frontends. All services share one Postgres cluster (with perService schemas), one Redis cluster, and one Kafka cluster.


PerService responsibilities

bff — BackendForFrontend gateway

Single API surface for both SPAs. Authenticates requests (JWT validation), proxies to domain services, aggregates multiService responses (e.g., "give me escrow status + wallet balance + recent notifications in one response"). Stateless — sessions in Redis, business state in domain services.

auth — Authentication + sessions

JWT issuance (passportJwt), TOTP/OTP for 2FA, password hashing (@node-rs/argon2 + bcrypt for legacy), session management in Redis. Exposes S2SAuth guard (jwtS2s.guard.ts) for interService calls.

escrow — The state machine

The product's core. Owns the Escrow and EscrowTransition entities. Implements the state machine:

agreement → accepted → funded → delivery → inspection → completed
                                                       → disputed
                                                       → cancelled

Every transition is recorded as a row in escrowTransitions (audited, immutable). Platform fee logic (PlatformFeesService.calculateFee) lives here. CronDriven lifecycle automation (escrowExpirationCron, inspectionAutoReleaseCron) drives timeBased transitions.

wallet — Custodial key management + onChain transfers

The most securityCritical service. Owns perUser custodial wallet addresses across EVM (ETH/BSC/Polygon — single key, multiChain), Solana, and Tron. KMS encrypts private keys; no plaintext ever touches application memory persistently. Hot/funding/cold key tiering. PerChain executors (evmExecutor.service, solanaExecutor.service, tronExecutor.service) handle the chainSpecific transfer semantics.

listenerEngine — MultiChain event detection

Polls 5 chains for ERC20 / SPL / TRC20 Transfer events. PerChain confirmation buffer (Ethereum at 12 blocks, BSC at 15, etc.) protects against reorg. PostgreSQL checkpoint state per chain. Replay mode for catchUp after restart. RPC failover across multiple endpoints per chain. Publishes deposit.detected events to Kafka for the escrow service to consume.

milestone — Milestone state machine

For milestoneBased escrows (multiStep deliveries), runs an inner state machine: created → funded → delivered → approved → disputed. 2-24 milestones per escrow. Owns the Milestone entity.

inquiry — Dispute resolution

Lifecycle of adminMediated disputes. Subscribes to escrow.disputed events, opens an inquiry, walks through evidence collection, ends with an admin forceClose that routes the funds to the awarded party.

compliance — KYC + sanctions

Sumsub integration for KYC, OFAC sanctions list checks, geoBlocking (geoipLite). Blocks escrow creation if either party fails compliance.

connect — External integrations

Tronscan/Etherscan/Solscan API clients for transaction verification, thirdParty rate feeds, external blockchain data providers.

notification — Email + push fanOut

Subscribes to most Kafka topics. Renders Handlebars email templates, sends via Resend (primary) / Nodemailer (fallback). WebPush via Socket.IO. PerUser notification preferences in DB.

ledger — Canonical accounting

Subscribes to wallet.transfer.completed + escrow.transitioned events. Maintains a doubleEntry ledger of every stateChanging operation. The source of truth for "what does this account balance look like right now" and "give me the audit trail for escrow X".

reporting — Analytics aggregates

CronDriven aggregations across the ledger. Powers admin analytics dashboards: GMV by chain, dispute rate, average resolution time, etc.

admin — Admin API

Privileged operations: forceClose escrow, override KYC status, refund accidentallyLocked funds, rotate platform fee. All admin actions are logged to a separate adminAudit schema.


Data plane

PostgreSQL multiSchema

One physical cluster, schemas named per service (escrow, wallet, listenerEngine, etc.). Each service has its own Prisma client (@prisma/client generated separately per service). CrossService joins are forbidden — data shared between services flows through Kafka events.

TradeOff: easier ops (one Postgres to monitor, back up, upgrade) at the cost of weaker service isolation (a misconfigured migration in one service could in theory affect another via shared resources). Mitigated by perSchema permissions and CI checks on Prisma schema changes.

Kafka topics (excerpt)

Topic Producer Consumers
deposit.detected listenerEngine escrow
escrow.transitioned escrow notification, ledger, reporting
wallet.transfer.requested escrow wallet
wallet.transfer.completed wallet ledger, escrow, notification
inquiry.created inquiry notification, admin
kyc.completed compliance escrow

Outbox pattern (TECHSTACK §6) ensures DB writes + Kafka publishes are atomic perService.

Redis

Two main uses:

  1. Session store (@nestjs/throttler, JWT refresh tokens, OTP codes) — authService controls TTLs.
  2. Socket.IO adapter (@socket.io/redis-adapter) for horizontal scaling of realTime push.

External dependencies

  • AWS KMS — walletService uses customerManaged keys for envelopeLess direct encryption of private keys. PerOperation encryption context binds the encrypt/decrypt call to a specific (service, user, depositId) tuple — audit trail in CloudTrail.
  • AWS S3 — compliance documents (KYC scans), inquiry evidence (dispute attachments).
  • AWS Secrets Manager — external API keys (Tronscan, Etherscan, Resend, etc.) stored centrally, rotated periodically.
  • Resend + Nodemailer + Handlebars templates — transactional email.
  • Chains — Ethereum, BSC, Polygon (via Alchemy + private RPC failover), Solana (via Helius + Solana mainnet beta RPC), Tron (TronGrid + private node).

CrossCutting concerns

Observability

  • Structured logs (Pino) per service, aggregated to a central log store
  • OpenTelemetry tracing — request ID flows through BFF → domain services → DB queries
  • Prometheus metrics per service: request count, error rate, p50/p99 latency, Kafka consumer lag
  • Sentry for unhandled errors

Deployment

  • Each service is a separate Docker image, separate Kubernetes deployment
  • Helm charts per service with environmentSpecific values
  • Database migrations run via migrateMongo-style migration tooling, applied before container restart
  • BlueGreen deployment for the BFF; rolling for domain services

Security perimeter

  • Application boundary: BFF + authService. JWT validation, rate limiting (NestJS Throttler), helmet, CORS.
  • Custody boundary: walletService. KMS keys, noCache decrypt, hot/cold tiering, withdrawLimit alerts.
  • Network boundary: VPC isolation. Only BFF and listenerEngine talk to the outside world (BFF for HTTP, listenerEngine for RPC).