CEX Broker

A high-performance gRPC-based cryptocurrency exchange broker service that provides unified access to multiple centralized exchanges (CEX) through the CCXT library. Built with TypeScript, Bun, and designed for reliable trading operations with policy enforcement, real-time streaming, and zero-knowledge proof integration.

🚀 Features

• Multi-Exchange Support: Unified API to any CEX supported by CCXT (100+ exchanges)
• gRPC Interface: High-performance RPC communication with type safety
• Real-time Streaming: Live orderbook, trades, ticker, OHLCV, balance, and order updates
• Policy Enforcement: Configurable trading and withdrawal limits with real-time policy updates
• IP Authentication: Security through IP whitelisting
• Zero-Knowledge Proofs: Optional Verity integration for privacy-preserving operations
• Secondary Broker Support: Multiple API keys per exchange for load balancing and redundancy
• Real-time Policy Updates: Hot-reload policy changes without server restart
• Type Safety: Full TypeScript support with generated protobuf types
• Comprehensive Logging: Built-in logging with tslog
• CLI Support: Command-line interface for easy management
• Deposit Address Management: Fetch deposit addresses for supported networks
• Advanced Order Management: Create, fetch, and cancel orders with full details

📋 Prerequisites

• Bun (v1.2.17 or higher)
• API keys for supported exchanges (e.g., Binance, Bybit, etc.)
• Optional: Verity prover URL for zero-knowledge proof integration

🛠️ Installation

1. Clone the repository:

   git clone <repository-url>
   cd cex-broker

2. Install dependencies:

   bun install

3. Generate protobuf types:

   bun run proto-gen

⚙️ Configuration

Environment Variables

The broker loads configuration from environment variables with the CEX_BROKER_ prefix:

# Server Configuration
PORT_NUM=8086

# Primary Exchange API Keys (format: CEX_BROKER_<EXCHANGE>_API_KEY/SECRET)
CEX_BROKER_BINANCE_API_KEY=your_binance_api_key
CEX_BROKER_BINANCE_API_SECRET=your_binance_api_secret
CEX_BROKER_BYBIT_API_KEY=your_bybit_api_key
CEX_BROKER_BYBIT_API_SECRET=your_bybit_api_secret
CEX_BROKER_KRAKEN_API_KEY=your_kraken_api_key
CEX_BROKER_KRAKEN_API_SECRET=your_kraken_api_secret

# Secondary Exchange API Keys (for load balancing and redundancy)
CEX_BROKER_BINANCE_API_KEY_1=your_secondary_binance_api_key
CEX_BROKER_BINANCE_API_SECRET_1=your_secondary_binance_api_secret
CEX_BROKER_BINANCE_API_KEY_2=your_tertiary_binance_api_key
CEX_BROKER_BINANCE_API_SECRET_2=your_tertiary_binance_api_secret

# OpenTelemetry Metrics (Optional)
# Send metrics via OTLP to a collector
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
OTEL_SERVICE_NAME=cex-broker
# Or use CEX_BROKER_OTEL_* (default port 4318). Legacy: CEX_BROKER_CLICKHOUSE_* also supported.
# CEX_BROKER_OTEL_HOST=otel-collector
# CEX_BROKER_OTEL_PORT=4318
# CEX_BROKER_OTEL_PROTOCOL=http

Note: Only configure API keys for exchanges you plan to use. The system will automatically detect and initialize configured exchanges.

Research / Backtest (ClickHouse Path B)

Archive subscribe streams (OHLCV, orderbook, trades, ticker) to ClickHouse via the archive forwarder, visualize candles in the browser, run Python backtests, and optionally feed Hummingbot from the same warehouse.

• Overview: research/README.md
• Full guide: docs/research-backtest.md

Quick start:

docker network create fiet-sandbox || true
docker compose -f docker/clickhouse-research.compose.yml up -d
bun run start-archive-forwarder   # if not using compose forwarder service
SYMBOLS=BTC/USDT,BNB/USDT,DOGE/USDT bun run start-archive-watch
CLICKHOUSE_PORT=8123 bun run start-candle-viewer   # http://localhost:8091

Dev watchers: dev:candle-viewer, dev:archive-forwarder, dev:archive-watch (see research/README.md).

Key env vars: CEX_BROKER_ARCHIVE_ENABLED=true, CEX_BROKER_ARCHIVE_FORWARDER_URL, CEX_BROKER_ARCHIVE_DEAD_LETTER_PATH, and CEX_BROKER_DEPLOYMENT_ID. The archive is disabled for every enable value except the exact string true. Production durability requires the dead-letter file to reside on persistent writable storage or a mounted volume; a container-local ephemeral path is not durable.

Wallet-authenticated exchanges

Some exchanges (for example Hyperliquid, Vertex, Paradex, and Derive) authenticate with an on-chain wallet instead of exchange-issued API keys. The broker keeps the same API_KEY / API_SECRET interface and maps credentials internally based on each exchange's CCXT requiredCredentials:

• CEX_BROKER_<EXCHANGE>_API_KEY → wallet address (0x…)
• CEX_BROKER_<EXCHANGE>_API_SECRET → private key (0x… hex)

Example:

CEX_BROKER_HYPERLIQUID_API_KEY=0x1234567890abcdef1234567890abcdef12345678
CEX_BROKER_HYPERLIQUID_API_SECRET=0xabcdef1234567890abcdef1234567890abcdef1234567890abcdef1234567890

gRPC metadata uses the same parity interface: api-key carries the wallet address and api-secret carries the private key.

Detection is automatic from CCXT requiredCredentials. A dex: true flag does not imply wallet auth; exchanges such as WOOFi Pro and Modetrade still use API keys.

Treat API_SECRET values for wallet exchanges as signing keys with the same operational security as API secrets.

Spot vs perp (marketType)

The broker defaults to spot everywhere unless a request explicitly opts into perps/futures. This overrides exchange-level CCXT defaults (for example Hyperliquid's internal defaultType: swap).

Pass marketType in action payloads (string map) or subscribe options:

marketType	Meaning
omitted / spot	Spot markets and spot balances
swap or perp	Perpetuals (resolves symbols like ETH/USDC:USDC on Hyperliquid)
future	Dated futures where supported

Examples:

// CreateOrder payload
{
  "fromToken": "ETH",
  "toToken": "USDC",
  "amount": "1",
  "price": "2500",
  "marketType": "swap",
  "params": { "slippage": "0.05" }
}

// FetchBalances payload
{ "marketType": "swap", "balanceType": "total" }

Policy markets support optional suffixes:

• HYPERLIQUID:ETH/USDC@swap — perp only
• HYPERLIQUID:ETH/USDC@spot — spot only
• HYPERLIQUID:ETH/USDC:USDC — explicit unified perp symbol
• BINANCEUSDM:ETH/USDT — use the futures exchange id directly

For split futures exchanges (binanceusdm, krakenfutures, kucoinfutures), register the futures cex id separately. Fund movement between spot and futures wallets uses Action.Call or exchange-specific transfer actions.

Perp configuration actions

Two capability-gated actions complement Action.Call:

Action	Value	Requires CCXT	Purpose
GetPerpConfigState	14	fetchPositions	Read positions and per-symbol leverage/margin mode
SetPerpConfigState	15	setLeverage	Set leverage (and margin mode) for a symbol

GetPerpConfigState payload: optional symbol, optional params (JSON).

SetPerpConfigState payload: symbol, leverage, optional marginMode (cross | isolated), optional params.

Exchanges without the required capability return gRPC UNIMPLEMENTED. Use Action.Call for other perp operations (transfer, addMargin, closePosition, etc.).

Metrics (OpenTelemetry): Metrics are exported via OTLP. If neither OTEL_EXPORTER_OTLP_ENDPOINT nor CEX_BROKER_OTEL_HOST (or legacy CEX_BROKER_CLICKHOUSE_HOST) is set, metrics are disabled. When enabled, the broker sends metrics to the configured OTLP endpoint (e.g. an OpenTelemetry Collector).

Policy Configuration

Configure trading policies in policy/policy.json.

• Full reference: see POLICY.md (supported options, matching rules, reload behaviour, and troubleshooting)
• Example policy: policy/policy.json
• Treasury corridor example: policy/policy.binance-mexc-usdc-bep20.example.json permits only USDC over the normalized BNB/BSC/BEP20 network between Binance and MEXC; use explicit ceremony-time overrides for non-default acquired-asset transfers.

{
  "withdraw": {
    "rule": [
      {
        "exchange": "BINANCE",
        "network": "BEP20",
        "whitelist": ["0x9d467fa9062b6e9b1a46e26007ad82db116c67cb"]
      },
      {
        "exchange": "BINANCE",
        "network": "ARBITRUM",
        "whitelist": ["0x9d467fa9062b6e9b1a46e26007ad82db116c67cb"]
      }
    ]
  },
  "deposit": {},
  "order": {
    "rule": {
      "markets": [
        "BINANCE:ARB/USDT",
        "BYBIT:ARB/USDC",
        "BINANCE:ETH/USDT",
        "BINANCE:BTC/ETH"
      ],
      "limits": [
        { "from": "USDT", "to": "ETH", "min": 1, "max": 100000 }
      ]
    }
  }
}

🚀 Usage

Starting the Server

# Using the CLI (recommended)
bun run start-broker --policy policy/policy.json --port 8086 --whitelist 127.0.0.1 192.168.1.100 --verityProverUrl http://localhost:8080

# Development mode
bun run start

# Production build
bun run build:ts
bun run ./build/index.js

CLI Options

cex-broker --help

Options:
  -p, --policy <path>                    Policy JSON file (required)
  --port <number>                        Port number (default: 8086)
  -w, --whitelist <addresses...>         IPv4 address whitelist (space-separated list)
  -vu, --verityProverUrl <url>           Verity Prover URL for zero-knowledge proofs

Available Scripts

# Start the server
bun run start

# Start broker server (development)
bun run start-broker

# Start broker server with Verity
bun run start-broker-server-with-verity

# Build for production
bun run build
bun run build:ts

# Run tests
bun test

# Generate protobuf types
bun run proto-gen

# Format code
bun run format

# Lint code
bun run lint
bun run lint:fix

# Check code (format + lint)
bun run check
bun run check:fix

📡 API Reference

The service exposes a gRPC interface with two main methods:

ExecuteAction

Execute trading operations on supported exchanges.

Request:

message ActionRequest {
  Action action = 1;                        // The action to perform
  map<string, string> payload = 2;          // Parameters for the action
  string cex = 3;                           // CEX identifier (e.g., "binance", "bybit")
  string symbol = 4;                        // Trading pair symbol if needed
}

Response:

message ActionResponse {
  string result = 2;                        // JSON string of the result data or ZK proof
}

Available Actions:

• NoAction (0): No operation
• Deposit (1): Confirm deposit transaction
• Withdraw (2): Withdraw funds
• CreateOrder (3): Create a new order
• GetOrderDetails (4): Get order information
• CancelOrder (5): Cancel an existing order
• FetchBalances (6): Get account balances. Supports balanceType: "free", "used", "total" (defaults to "total").
• FetchDepositAddresses (7): Get deposit addresses for a token/network
• FetchTicker (8): Get ticker information
• FetchCurrency (9): Get currency metadata (networks, fees, etc.) for a symbol
• Call (10): Generic method invocation on the underlying broker instance. Provide functionName, optional args array, and optional params object.

Order Book Call Methods

Call also supports broker-defined order-book methods for HB strategy compatibility. These methods use the method payload field and return JSON in ActionResponse.result.

// Discover order-book capability
const capabilityRequest = {
  action: 10, // Call
  cex: "mexc",
  symbol: "ARB/USDT",
  payload: {
    method: "fetch_order_book_capability",
    depthLimit: "100",
    constructionMode: "sampled_top_n_snapshot"
  }
};

// Fetch current top-N order-book snapshot
const snapshotRequest = {
  action: 10, // Call
  cex: "binance",
  symbol: "BTC/USDT",
  payload: {
    method: "fetch_order_book_snapshot",
    depthLimit: "100"
  }
};

// Request historical sampled snapshots
const historicalRequest = {
  action: 10, // Call
  cex: "mexc",
  symbol: "ARB/USDT",
  payload: {
    method: "fetch_historical_order_book_snapshots",
    start: "2026-06-02T00:00:00Z",
    end: "2026-06-02T00:01:00Z",
    cadence: "1s",
    depthLimit: "100",
    constructionMode: "sampled_top_n_snapshot"
  }
};

Current snapshot responses include top-level bids and asks arrays plus metadata:

{
  "bids": [[100.0, 1.0]],
  "asks": [[101.0, 2.0]],
  "timestamp": 1760000000000,
  "receivedTimestamp": 1760000000100,
  "exchange": "binance",
  "symbol": "BTC/USDT",
  "sequence": 123,
  "depthLimit": 100
}

If historical sampled top-N depth is unavailable, the broker returns a typed unsupported result instead of a gRPC transport failure:

{
  "exchange": "mexc",
  "symbol": "ARB/USDT",
  "unsupported": true,
  "unsupportedReason": "historical_order_book_provider_unsupported"
}

Capability responses are conservative: current snapshot and live stream support reflect available broker/provider methods, historical sampled top-N support is only true when implemented for the requested parameters, and exact L2 reconstruction remains false until a validated snapshot-plus-delta reconstruction path exists.

Example Usage:

// Fetch total balances (default)
const totalBalancesRequest = {
  action: 6, // FetchBalances
  payload: { type: "spot" }, // default is spot if omitted
  cex: "binance"
};

// Fetch free balances
const freeBalancesRequest = {
  action: 6, // FetchBalances
  payload: { balanceType: "free", type: "spot" },
  cex: "binance"
};

// Fetch used balances
const usedBalancesRequest = {
  action: 6, // FetchBalances
  payload: { balanceType: "used", type: "spot" },
  cex: "binance"
};

// Create order
const orderRequest = {
  action: 3, // CreateOrder
  payload: {
    orderType: "limit",
    amount: "0.001",
    fromToken: "BTC",
    toToken: "USDT",
    price: "50000"
  },
  cex: "binance",
  symbol: "BTC/USDT"
};

// Fetch deposit addresses
const depositAddressRequest = {
  action: 7, // FetchDepositAddresses
  payload: {
    chain: "BEP20"
  },
  cex: "binance",
  symbol: "USDT"
};

// Fetch currency metadata
const fetchCurrencyRequest = {
  action: 9, // FetchCurrency
  payload: {},
  cex: "binance",
  symbol: "USDT"
};

Subscribe (Streaming)

Real-time streaming of market data and account updates.

Request:

message SubscribeRequest {
  string cex = 1;                          // CEX identifier
  string symbol = 2;                        // Trading pair symbol
  SubscriptionType type = 3;                // Type of subscription
  map<string, string> options = 4;          // Additional options (e.g., timeframe)
}

Response Stream:

message SubscribeResponse {
  string data = 1;                         // JSON string of the streaming data
  int64 timestamp = 2;                     // Unix timestamp
  string symbol = 3;                       // Trading pair symbol
  SubscriptionType type = 4;               // Type of subscription
}

Available Subscription Types:

• NO_ACTION (0): Compatibility default; resolved to ORDERBOOK
• ORDERBOOK (1): Real-time order book updates
• TRADES (2): Live trade feed
• TICKER (3): Ticker information updates
• OHLCV (4): Candlestick data (configurable timeframe)
• BALANCE (5): Account balance updates
• ORDERS (6): Order status updates

For backward compatibility, omitted, NO_ACTION, or invalid subscription type values are resolved to ORDERBOOK.

For Binance spot account streams, BALANCE and ORDERS use Binance's WebSocket API user-data subscription (userDataStream.subscribe.signature). They use the broker account selected by request metadata and do not rely on the retired Spot listenKey REST lifecycle.

Example Usage:

// Subscribe to orderbook updates
const orderbookRequest = {
  cex: "binance",
  symbol: "BTC/USDT",
  type: 1, // ORDERBOOK
  options: {
    depthLimit: "100"
  }
};

// Subscribe to OHLCV with custom timeframe
const ohlcvRequest = {
  cex: "binance",
  symbol: "BTC/USDT",
  type: 4, // OHLCV
  options: {
    timeframe: "1h"
  }
};

🔒 Security

IP Authentication

All API calls require IP authentication. Configure allowed IPs via CLI or broker initialization:

# Via CLI
cex-broker --policy policy.json --whitelist 127.0.0.1 192.168.1.100

# Via code
const config = {
  port: 8086,
  whitelistIps: [
    "127.0.0.1", // localhost
    "::1",       // IPv6 localhost
    "192.168.1.100", // Your allowed IP
  ]
};

Secondary Broker Support

For high-availability, load balancing and compartmentalized capital management, you can configure multiple API keys per exchange:

# Primary keys
CEX_BROKER_BINANCE_API_KEY=primary_key
CEX_BROKER_BINANCE_API_SECRET=primary_secret

# Secondary keys (numbered)
CEX_BROKER_BINANCE_API_KEY_1=secondary_key_1
CEX_BROKER_BINANCE_API_SECRET_1=secondary_secret_1
CEX_BROKER_BINANCE_API_KEY_2=secondary_key_2
CEX_BROKER_BINANCE_API_SECRET_2=secondary_secret_2

To use secondary brokers, include the use-secondary-key metadata in your gRPC calls:

const metadata = new grpc.Metadata();
metadata.set('use-secondary-key', '1'); // Use secondary broker 1
metadata.set('use-secondary-key', '2'); // Use secondary broker 2

Routed Withdraws Via Master

The broker can now orchestrate a routed withdraw for exchanges that require:

1. moving funds from a sub-account to a master account
2. executing the external withdraw from the master account

Current support:

• binance: supported
• other exchanges: not yet supported; the broker falls back to a normal direct withdraw

Configurer guide

From the operator's perspective, the simplest setup is:

1. configure the master account as the primary account for that exchange
2. configure each sub-account as a numbered secondary account
3. keep using normal withdraw policy rules in policy.json
4. set routed-withdraw fields in the request payload when you want this behavior

Minimal Binance example:

# Master account
CEX_BROKER_BINANCE_API_KEY=master_key
CEX_BROKER_BINANCE_API_SECRET=master_secret

# Optional but recommended for clarity
CEX_BROKER_BINANCE_ROLE=master

# Sub-account used as the source of funds
CEX_BROKER_BINANCE_API_KEY_1=subaccount_key_1
CEX_BROKER_BINANCE_API_SECRET_1=subaccount_secret_1

# Optional but recommended for clarity
CEX_BROKER_BINANCE_ROLE_1=subaccount

Example withdraw payload:

{
  "recipientAddress": "0x1234...",
  "amount": "25",
  "chain": "ARBITRUM",
  "routeViaMaster": "true",
  "sourceAccount": "secondary:1",
  "masterAccount": "primary"
}

Recommended operational model:

• make primary the exchange master account
• use secondary:N for sub-accounts that hold funds or trade independently
• only set routeViaMaster=true when the exchange requires master-executed withdrawals

Meaning of sourceAccount and masterAccount

• sourceAccount: the account that currently holds the funds
• masterAccount: the account that should perform the final external withdraw

Accepted selector values:

• primary
• secondary:1, secondary:2, ...
• current for sourceAccount, which means "whatever account was selected by metadata or by default"

Purpose of role, uid, email, and subAccountId

These fields are not all required today, and for the current Binance implementation they are mostly future-proofing rather than something you must configure immediately.

• role: broker-level intent. This is the useful one today. It marks an account as master or subaccount and makes configuration easier to audit.
• email: exchange-specific sub-account identifier. Some exchanges identify sub-accounts by email rather than by API key relationship.
• subAccountId: exchange-specific sub-account identifier used by APIs that require an explicit account id.
• uid: exchange-specific account/user identifier required by some transfer APIs.

If you enforce the convention that primary is always the master account, then yes, uid, email, and subAccountId are redundant for the current Binance flow.

They still exist for two reasons:

• to support future exchange adapters where API keys alone are not enough to identify the transfer source or destination
• to make the broker config model stable now instead of redesigning it later per exchange

Practical rule:

• for Binance today, configure primary as master and secondary:N as sub-accounts; role is optional but recommended
• you do not need uid, email, or subAccountId unless a future exchange adapter requires them

Does deposit require the same setup?

Usually no.

Deposits are different from withdrawals:

• withdrawals may require a master account to authorize the external transfer
• deposits usually just require fetching the deposit address for the account or sub-account you want to receive funds

So the normal approach is:

• deposit directly to the intended target account or sub-account
• only do an internal transfer afterward if funds landed in the wrong internal account for your workflow

Zero-Knowledge Proof Integration

Enable privacy-preserving proof over CEX data with Verity zkTLS integration:

# Start with Verity integration
cex-broker --policy policy.json --verityProverUrl http://localhost:8080

When Verity is enabled, responses include zero-knowledge proofs instead of raw data:

// With Verity enabled
const response = await client.ExecuteAction(request, metadata);
// response.result contains ZK proof instead of raw data

API Key Management

• Store API keys securely in environment variables
• Use read-only API keys when possible
• Regularly rotate API keys
• Monitor API usage and set appropriate rate limits
• Use secondary brokers for redundancy and load distribution

📊 OpenTelemetry Metrics

The broker exports metrics via OpenTelemetry (OTLP) for monitoring and analytics. Metrics are collected for:

• ExecuteAction requests: Request counts, success/failure rates, latency histograms
• Subscribe streams: Subscription counts, duration, error rates
• Action-specific metrics: Tagged by action type, CEX, and symbol

Metrics (OTLP)

The following metrics are exported as OTLP counters and histograms:

• execute_action_requests_total (counter): Total ExecuteAction requests
• execute_action_success_total (counter): Successful ExecuteAction requests
• execute_action_errors_total (counter): Failed ExecuteAction requests
• execute_action_duration_ms (histogram): ExecuteAction latency
• cex_market_action_executions_total (counter): CreateOrder/GetOrderDetails execution telemetry events, tagged by action, exchange, account label, symbol, side, order type, status, and result
• cex_market_action_requested_quantity (histogram): Requested base quantity when known
• cex_market_action_requested_notional (histogram): Requested notional from payload amount * price when known
• cex_market_action_executed_base_quantity (histogram): Executed base quantity reported by the exchange
• cex_market_action_executed_quote_quantity (histogram): Executed quote quantity/cost reported by the exchange
• cex_market_action_average_execution_price (histogram): Exchange-reported or derived average execution price
• cex_market_action_filled_amount (histogram): Filled amount reported by the exchange
• cex_market_action_remaining_amount (histogram): Remaining amount reported by the exchange
• cex_market_action_fee_amount (histogram): Fee amount when provided by the exchange
• cex_market_action_fee_rate (histogram): Fee rate when provided by the exchange
• subscribe_requests_total (counter): Total Subscribe requests
• subscribe_errors_total (counter): Failed Subscribe requests
• subscribe_duration_ms (histogram): Subscribe stream duration

General request metrics include attributes such as action, cex, symbol, error_type, and service. Market-action execution metrics intentionally use low-cardinality attributes only: action, cex, account, symbol, side, order_type, status, result, and service.

Market Action Accounting Telemetry

Every successful CreateOrder response, successful GetOrderDetails response, rejected order response, and failed create-order attempt emits a structured log event named cex_market_action_execution. The event includes the low-cardinality metric attributes above plus join identifiers and accounting values:

• Join identifiers: orderId, clientOrderId, idempotencyId, makerActionId
• Execution values: requested quantity/notional, executed base quantity, executed quote quantity/cost, average execution price, filled amount, remaining amount, fee amount, fee currency, fee rate
• Timing: exchange timestamp when present and broker observed timestamp

Use metrics for aggregations and alerts. For the durable execution audit trail, the broker archives every order lifecycle event to broker_execution.order_events (and pre-order top-of-book to broker_execution.market_metadata_snapshots) through the archive forwarder — the same HTTP /archive → ClickHouse path used for market_data.*. Set CEX_BROKER_ARCHIVE_ENABLED=true, an explicit HTTP(S) CEX_BROKER_ARCHIVE_FORWARDER_URL, and a writable durable JSONL path in CEX_BROKER_ARCHIVE_DEAD_LETTER_PATH; startup fails if either required sink configuration is missing or invalid. In production, that path must be on persistent writable storage or a mounted volume rather than the container's ephemeral filesystem. Queue shedding and rows that remain undeliverable during shutdown are written to that loss journal with their original {table,row} payload before being discarded. Setting CEX_BROKER_ARCHIVE_OTEL_LOGS_ENABLED=true additionally mirrors execution rows to OTel logs for observability, but OTel is never the archive sink of record. Analysts join Maker action rows to broker_execution.order_events using maker_action_id, idempotency_id, client_order_id, or the exchange order_id, then compare Maker propAMM execution price against average_execution_price and fees. Failed CreateOrder rows keep bounded exchange error detail in error_message; their telemetry-shaped payload_json, metrics, and ordinary telemetry logs remain redacted. The broker does not emit raw exchange payloads, API keys, secrets, or credentials in telemetry fields.

Telemetry Test Harness

Order telemetry tests use test/order-telemetry-fixtures.ts to run the real gRPC server with mocked CCXT exchanges. The fixture can simulate create-order responses, order-detail responses, partial fills, rejected orders, failed create-order calls, and fee/no-fee exchange payloads without live credentials.

Run only the focused telemetry suite:

bun test test/order-telemetry.test.ts

To extend coverage for another exchange response shape, add a mocked CCXT order object to createOrderExchangeFixture usage in test/order-telemetry.test.ts and assert the captured CapturingOtelMetrics calls.

Setting Up Metrics

1. Run an OTLP receiver (e.g. OpenTelemetry Collector):

   • Default endpoint: http://localhost:4318/v1/metrics
   • To store in ClickHouse or other backends, use the appropriate exporter in the collector pipeline (e.g. ClickHouse exporter).

2. Configure the broker:

   • Set OTEL_EXPORTER_OTLP_ENDPOINT (e.g. http://localhost:4318) or use CEX_BROKER_OTEL_HOST (and optional CEX_BROKER_OTEL_PORT, CEX_BROKER_OTEL_PROTOCOL). Legacy CEX_BROKER_CLICKHOUSE_* env vars are also supported.

3. Metrics are pushed periodically to the configured endpoint; no database schema is created by the broker (handled by the collector/backend).

Local or Docker: OTLP → ClickHouse (no Prometheus)
Use the included OpenTelemetry + ClickHouse stack so metrics go only to ClickHouse (no Prometheus exporter):

• Docker: docker compose -f docker-compose.otel.yaml up -d, then set OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 and start the broker.
• Config: otel/collector-config.yaml (OTLP receiver → batch → ClickHouse exporter). See otel/README.md for full instructions (Docker and local).

🏗️ Architecture

Project Structure

cex-broker/
├── src/                    # Source code
│   ├── cli.ts             # CLI entry point
│   ├── client.dev.ts      # Development client
│   ├── commands/          # CLI commands
│   │   └── start-broker.ts # Broker startup command
│   ├── handlers/          # RPC dispatch (execute-action, subscribe)
│   ├── helpers/           # Domain utilities (shared/, grpc/, order-book, …)
│   │   ├── index.ts       # Broker pool and policy helpers
│   │   ├── shared/        # Cross-cutting guards and errors
│   │   ├── grpc/          # Payload validation and status mapping
│   │   └── logger.ts      # Logging configuration
│   ├── index.ts           # Main broker class
│   ├── proto/             # Generated protobuf types
│   │   ├── cex_broker/    # Generated broker types
│   │   ├── node.proto     # Service definition
│   │   └── node.ts        # Type exports
│   ├── server.ts          # gRPC wiring only (delegates to handlers/)
│   └── types.ts           # TypeScript type definitions
├── proto/                 # Protocol buffer definitions
│   ├── cexBroker/         # Legacy generated types
│   └── node.proto         # Service definition
├── policy/                # Policy configuration
│   └── policy.json        # Trading and withdrawal rules
├── scripts/               # Build scripts
│   └── patch-protobufjs.js # Protobuf patching script
├── test/                  # Test files
├── patches/               # Dependency patches
├── examples/              # Example usage
│   └── kraken-orderbook-demo.ts
├── build.ts               # Build configuration
├── proto-gen.sh           # Proto generation script
├── test-setup.ts          # Test setup
├── tsconfig.json          # TypeScript configuration
├── biome.json             # Code formatting/linting
├── bunfig.toml            # Bun configuration
└── package.json           # Dependencies and scripts

Core Components

• CEXBroker: Main broker class that manages exchange connections and policy enforcement
• Policy System: Real-time policy validation and enforcement
• gRPC Server: High-performance RPC interface with streaming support
• CCXT Integration: Unified access to 100+ cryptocurrency exchanges
• Verity Integration: Zero-knowledge proof generation for privacy
• Secondary Broker Management: Load balancing and redundancy support

🧪 Development

Adding New Exchanges

The broker automatically supports all exchanges available in CCXT. To add a new exchange:

1. Add your API credentials to environment variables:

   CEX_BROKER_<EXCHANGE>_API_KEY=your_api_key
   CEX_BROKER_<EXCHANGE>_API_SECRET=your_api_secret

2. Update policy configuration if needed for the new exchange

3. The broker will automatically detect and initialize the exchange

Using Secondary Brokers

Secondary brokers provide redundancy and load balancing:

1. Configure secondary API keys:

   CEX_BROKER_BINANCE_API_KEY_1=secondary_key_1
   CEX_BROKER_BINANCE_API_SECRET_1=secondary_secret_1

2. Use secondary brokers in your gRPC calls:

   const metadata = new grpc.Metadata();
   metadata.set('use-secondary-key', '1'); // Use secondary broker

Querying Supported Networks

To understand which networks each exchange supports for deposits and withdrawals, you can query the exchange's currency information:

import ccxt from 'ccxt';

// Initialize the exchange (no API keys needed for public data)
const exchange = new ccxt.binance(); // or any other exchange like ccxt.bybit()

// Fetch all currencies and their network information
const currencies = await exchange.fetchCurrencies();

// Example: Check USDT networks on Binance
const usdtInfo = currencies['USDT'];
console.log("USDT Networks on Binance:");
console.log(usdtInfo?.networks);

// Example output:
// {
//   'BEP20': {id: 'BSC', network: 'BSC', active: true, deposit: true, withdraw: true, fee: 1.0},
//   'ETH': {id: 'ETH', network: 'ETH', active: true, deposit: true, withdraw: true, fee: 15.0},
//   'TRC20': {id: 'TRX', network: 'TRX', active: true, deposit: true, withdraw: true, fee: 1.0}
// }

// Check all available currencies
for (const [currency, info] of Object.entries(currencies)) {
  if ('networks' in info) {
    console.log(`\n${currency} networks:`);
    for (const [network, networkInfo] of Object.entries(info.networks)) {
      console.log(`  ${network}:`, networkInfo);
    }
  }
}

Common Network Identifiers:

• BEP20 / BSC: Binance Smart Chain
• ETH / ERC20: Ethereum
• TRC20: Tron
• ARBITRUM: Arbitrum One
• POLYGON: Polygon
• AVALANCHE: Avalanche C-Chain
• OPTIMISM: Optimism

Using this information in your policy:

{
  "withdraw": {
    "rule": [
      {
        "exchange": "BINANCE",
        "network": "BEP20",
        "whitelist": ["0x9d467fa9062b6e9b1a46e26007ad82db116c67cb"]
      },
      {
        "exchange": "BINANCE",
        "network": "ARBITRUM",
        "whitelist": ["0x9d467fa9062b6e9b1a46e26007ad82db116c67cb"]
      }
    ]
  }
}

Testing

# Run all tests
bun test

# Run tests with watch mode
bun test --watch

# Run tests with coverage
bun test --coverage

Code Quality

# Format code
bun run format

# Lint code
bun run lint

# Check code (format + lint)
bun run check

📦 Dependencies

Core Dependencies

• @opentelemetry/*: OpenTelemetry API, SDK metrics, OTLP HTTP exporter for metrics
• @grpc/grpc-js: gRPC server implementation
• @grpc/proto-loader: Protocol buffer loading
• @usherlabs/ccxt: Enhanced CCXT library with Verity support
• commander: CLI framework
• joi: Configuration validation
• tslog: TypeScript logging

Development Dependencies

• @biomejs/biome: Code formatting and linting
• @types/bun: Bun type definitions
• bun-plugin-dts: TypeScript declaration generation
• bun-types: Additional Bun types
• husky: Git hooks

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes
4. Add tests for new functionality
5. Ensure all tests pass (bun test)
6. Run code quality checks (bun run check)
7. Commit your changes (git commit -m 'Add amazing feature')
8. Push to the branch (git push origin feature/amazing-feature)
9. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🆘 Support

For issues and questions:

• Open an issue on the repository
• Contact the development team
• Check the CCXT documentation for exchange-specific information

🙏 Acknowledgments

• CCXT for providing unified access to cryptocurrency exchanges
• Bun for the fast JavaScript runtime
• gRPC for high-performance RPC communication
• Verity for zero-knowledge proof integration

---

Built with ❤️ by Usher Labs