Skip to content

system-design.md

Latest commit

 

History

History
273 lines (194 loc) · 8.2 KB

system-design.md

File metadata and controls

273 lines (194 loc) · 8.2 KB
Raw

System Design: Real-Time Vocabulary Quiz (Server-Side, Python/FastAPI)

Scope

This document describes the technical design for a real-time vocabulary quiz feature that supports:

  • Joining a quiz by unique quiz_id
  • Real-time answer submission
  • Server-authoritative score updates
  • Live leaderboard updates

Targets (from PRD):

  • Concurrent users per quiz session: >= 100
  • Score update latency: < 300ms (server receive -> server emit)
  • Leaderboard update latency: < 500ms (server receive -> server emit)
  • Availability during active quizzes: 99.9%
  • Score consistency: 100% (server-authoritative scoring)

Architecture Diagram

flowchart LR
  subgraph clientApps [ClientApps]
    WebClient[WebClient_or_TestClient]
  end

  subgraph realtimeLayer [RealTimeCommunicationLayer]
    WsApi[FastAPI_WebSocket_API]
  end

  subgraph backend [Backend]
    QuizEngine[QuizEngine_Service]
    SessionStore[SessionStateStore]
    QuestionBank[QuestionBank_Predefined]
  end

  subgraph dataLayer [DataLayer]
    Redis[(Redis_optional)]
    Postgres[(Postgres_optional)]
  end

  WebClient <-->|WebSocket_events| WsApi
  WsApi --> QuizEngine
  QuizEngine --> QuestionBank
  QuizEngine --> SessionStore
  SessionStore --> Redis
  SessionStore --> Postgres
Loading

Components and Responsibilities

WebClient_or_TestClient

  • Connects via WebSocket, joins a quiz via quiz_id
  • Submits answers during an active quiz
  • Renders live leaderboard updates without page refresh
  • On reconnect, re-joins and accepts a server snapshot to resync

FastAPI_WebSocket_API (Real-time Communication Layer)

  • Manages WebSocket connections and maps sockets to a quiz_id room
  • Validates incoming event payloads and routes to QuizEngine
  • Broadcasts server events (score updates, leaderboard updates, resync snapshots)
  • Handles disconnects and implements heartbeats (ping/pong)
  • Implementation note: the WebSocket API layer is organized into:
    • DTO schema (ws_models.py), registry-based dispatcher (ws_dispatcher.py), typed handlers (ws_handlers.py), and a thin transport loop (ws.py)

QuizEngine_Service (Server-Authoritative Scoring)

  • Single source of truth for scoring and leaderboard ordering
  • Enforces deterministic scoring rules and tie-breakers
  • Enforces idempotency (prevents double scoring on retries/reconnects)
  • Produces versioned state updates (monotonic session version)

QuestionBank_Predefined

  • Provides predefined quiz questions available at session start (per PRD)
  • Challenge scope: static JSON / in-memory fixtures

SessionStateStore

A storage abstraction for quiz state operations:

  • Get or create session by quiz_id
  • Upsert participant
  • Update score
  • Read leaderboard snapshot / topN
  • Provide snapshot for join/reconnect (resync)

Implementations:

  • InMemorySessionStateStore (MVP, single-node)
  • RedisSessionStateStore (optional scale-ready mode)

Redis (Optional, Scale-ready Mode)

  • Shared state for horizontally scaled deployments
  • Efficient leaderboard operations (Sorted Sets) and fast increments
  • Optional pub/sub to propagate updates across instances

Postgres (Optional, Future)

  • Not required by PRD (persistent profiles/sessions out of scope)
  • Future: audits, analytics, replay, enterprise reporting

Key Data Model (Minimal)

QuizSession

  • quiz_id: str
  • status: lobby | active | ended
  • version: int (monotonic)
  • participants: map[user_id -> Participant]
  • leaderboard: list[LeaderboardEntry] (cached topN)

Participant

  • user_id: str
  • display_name: str
  • score: int
  • last_client_seq: int (idempotency guard)

LeaderboardEntry

  • user_id: str
  • display_name: str
  • score: int
  • rank: int

AnswerSubmission (event payload)

  • quiz_id: str
  • user_id: str
  • question_id: str
  • answer: str
  • client_seq: int

Real-time API (WebSocket Events)

Client -> Server

  • join_quiz: { quiz_id, user_id, display_name }
  • submit_answer: { question_id, answer, client_seq }
  • ping: { ts } (optional)

Server -> Client

  • joined: { quiz_id, version, participant_score, leaderboard }
  • score_updated: { quiz_id, version, user_id, score, correct }
  • leaderboard_updated: { quiz_id, version, leaderboard }
  • state_resync: { quiz_id, version, participant_score, leaderboard, reason }
  • error: { code, message }

Data Flow: Join -> Answer -> Leaderboard

sequenceDiagram
  participant C as Client
  participant WS as FastAPI_WS_API
  participant QE as QuizEngine
  participant SS as SessionStateStore

  C->>WS: connect(WebSocket)
  C->>WS: join_quiz{quiz_id,user_id,display_name}
  WS->>QE: join_quiz(...)
  QE->>SS: upsert_participant(quiz_id,user_id,display_name)
  QE->>SS: get_snapshot(quiz_id,user_id)
  QE-->>WS: joined{version,participant_score,leaderboard}
  WS-->>C: joined{version,participant_score,leaderboard}

  C->>WS: submit_answer{question_id,answer,client_seq}
  WS->>QE: submit_answer(...)
  QE->>QE: validate_payload_and_question()
  QE->>QE: idempotency_check(user_id,client_seq)
  QE->>QE: score_answer_deterministically()
  QE->>SS: update_score_and_version(quiz_id,user_id,delta)
  QE->>SS: get_leaderboard_topN(quiz_id)
  QE-->>WS: score_updated{version,user_id,score}
  QE-->>WS: leaderboard_updated{version,leaderboard}
  WS-->>C: score_updated{...}
  WS-->>C: leaderboard_updated{...}
Loading

Consistency, Fairness, and Ordering

Server-authoritative scoring

  • Clients display scores computed by server only.
  • Ensures “100% score consistency” across clients.

Deterministic rules

  • Answer evaluation uses deterministic normalization (e.g., trim + casefold).
  • Leaderboard tie-break is deterministic (score desc, then user_id lexicographic ascending).

Idempotency

  • Each user includes client_seq that increments per submission.
  • Server stores last_client_seq and ignores submissions where client_seq <= last_client_seq.

Versioned session state

  • Session has monotonic version incremented on each accepted scoring mutation.
  • Clients apply only events with version greater than their current version; otherwise they request/receive state_resync.

Performance Notes (Meeting Latency Targets)

  • For >= 100 users/session, recomputing topN by sorting in-memory on each update is acceptable for MVP.
  • For larger scale, use Redis Sorted Sets:
    • ZINCRBY for score increments
    • ZREVRANGE for topN retrieval
  • Broadcast only topN + the user’s score (not full participant list) to reduce payload size.
  • Coalesce bursts (if multiple updates in quick succession, broadcast the latest state version).

Reliability Notes

  • Heartbeats detect dead sockets quickly and cleanly.
  • Reconnect path: client reconnects -> join_quiz -> server returns joined/state_resync snapshot.
  • Graceful errors:
    • invalid quiz_id
    • unknown question_id
    • malformed payloads
    • duplicate/out-of-order client_seq

Technologies and Tools (and Why)

Client

  • Minimal web client or test harness
  • Why: challenge focuses on the real-time engine; a thin client validates correctness and latency

Real-time layer

  • WebSocket
  • Why: push-based low-latency updates without polling

Backend

  • Python 3.x + FastAPI
  • Why: rapid iteration, async-friendly, strong request validation (Pydantic), straightforward WebSocket support

ASGI server

  • Uvicorn
  • Why: stable ASGI server with good WebSocket performance

State / scaling (optional for MVP)

  • Redis
  • Why: shared state for horizontal scaling and efficient leaderboard operations

Persistence (future)

  • Postgres
  • Why: audit/history/analytics if requirements expand (not required for challenge)

Testing and verification

  • pytest + pytest-asyncio
  • Why: validate scoring determinism, idempotency, reconnect/resync behaviors

Load / latency testing

  • Locust (Python) or k6 (JS)
  • Why: simulate >= 100 concurrent users and measure p95 latency against PRD targets

Observability

  • Structured logging (stdlib logging or structlog)
    • Implemented: standard library logging with JSON output (see server/src/quiz_server/observability/logging.py)
  • Metrics (optional): Prometheus-compatible counters/histograms
  • Why: measurable latency and operational confidence; aligns with “monitoring and observability” requirement