Skip to content

Latest commit

 

History

History
122 lines (84 loc) · 5.35 KB

File metadata and controls

122 lines (84 loc) · 5.35 KB

Realtime Sync Engine banner

Node.js TypeScript strict ws Vitest MIT License

Realtime Sync Engine

A focused, production-style implementation of a real-time data synchronization layer: reconnection with exponential backoff, per-key throttled broadcast, and multi-client fan-out over WebSockets.

Overview

Most "real-time" demos just relay every upstream message straight to the browser and call it done. That breaks down fast: upstream feeds can emit dozens of updates per second, connections drop, and naive re-broadcast either floods clients or falls silently out of sync when the connection dies.

This project isolates the three problems that actually matter in a real-time sync layer, each with its own tested module:

  1. Reconnection — the upstream connection will drop. It needs to come back on its own, without hammering the server (src/backoff.ts, src/upstream-client.ts).
  2. Throttling — a fast upstream feed shouldn't translate into a flood of messages downstream. Clients need the freshest value, not every intermediate tick (src/throttled-broadcaster.ts).
  3. Fan-out — one upstream connection, many downstream clients, without duplicating upstream load per client (src/server.ts).

As a concrete example, the server connects to Binance's public trade stream (no API key required) and re-broadcasts throttled price updates to any number of connected browser clients over a local WebSocket, with a minimal live dashboard included.

Architecture

Pipeline: Binance public WS -> UpstreamClient -> ThrottledBroadcaster -> Local WebSocketServer -> Browser dashboard

Live dashboard

Screenshot del dashboard en vivo mostrando el precio de BTC/USDT actualizándose

Core Features

  • Automatic reconnection with exponential backoff and jitter
  • Per-key throttled broadcast (trailing-edge, always delivers the latest value)
  • Multi-client fan-out from a single upstream connection
  • Zero external dependencies for the core logic beyond ws
  • Fully unit tested with fake timers — no network calls in tests
  • Minimal live dashboard served from the same process
  • TypeScript, strict mode

Why this design

The reconnection logic and the throttling logic are deliberately separate, framework-agnostic modules (backoff.ts, throttled-broadcaster.ts) with no dependency on ws or HTTP. They're pure enough to unit test with fake timers and no real sockets, and they'd work the same way whether the upstream is a crypto exchange, a sports data provider, or an internal service — the pattern is the point, not the specific data source.

Project Structure

src/
├── backoff.ts               # exponential backoff calculation (pure function)
├── throttled-broadcaster.ts # per-key throttled broadcast (trailing-edge)
├── upstream-client.ts       # WebSocket client with auto-reconnect
├── types.ts
└── server.ts                # wires everything together + serves the dashboard

test/
├── backoff.test.ts
└── throttled-broadcaster.test.ts

public/
└── index.html                # minimal live dashboard

Running locally

npm install
npm run dev

Then open http://localhost:8080 — you should see a live BTC/USDT price ticking in, throttled to one update every 200ms regardless of how fast the upstream feed sends trades.

Configuration

Environment variables (all optional, sane defaults included):

Variable Default Description
PORT 8080 Local HTTP/WebSocket server port
UPSTREAM_URL Binance BTC/USDT trade stream Any WebSocket URL emitting JSON messages
BROADCAST_INTERVAL_MS 200 Minimum time between broadcasts per symbol

Tests

npm test

Both the backoff calculation and the throttled broadcaster are tested in isolation using fake timers — no real network connections or sleeps involved, so the suite runs in well under a second.

Tech Stack

  • Node.js
  • TypeScript
  • ws
  • Vitest
  • ESLint

License

MIT — see LICENSE for details.

Author

Nicolás Pedernera

Systems Engineer — Universidad de Buenos Aires, 2024

Focused on backend engineering, fintech, cryptocurrency, blockchain infrastructure, and AI systems.