Add Comprehensive Quant Dev Market Making Guide with 9 modules covering Reg NMS/SHO, ITCH/OUCH/FIX protocols, C++ Optimization, and Strategies

Author: shreejitverma

Market Making/Quant_Dev_Market_Making_Guide/01_Regulatory_Framework_NMS_SHO.md

@@ -0,0 +1,208 @@
+# Nasdaq TotalView-ITCH 5.0: Technical Guide for Quantitative Developers
+
+**Protocol Version:** 5.0  
+**Transport:** UDP/IP (Multicast) / TCP (Snapshot/Recovery)  
+**Encoding:** Binary (Big-Endian)  
+**Focus:** Ultra-low latency, full market depth (MBO - Market by Order)
+
+---
+
+## 1. Protocol Overview
+
+Nasdaq TotalView-ITCH is the proprietary data feed that provides full order depth for Nasdaq-listed securities. Unlike the SIP (consolidated feed), ITCH allows you to build the **full limit order book** (Level 3 data) by broadcasting every individual order message.
+
+### Key Characteristics
+*   **Direct Feed:** Bypasses the SIP for lower latency (~microsecond scale).
+*   **Market by Order (MBO):** You see every individual order, not just price levels. This allows for queue position estimation and advanced microstructure signals.
+*   **Binary Encoding:** Fixed-length messages (mostly) for efficient parsing.
+*   **Big-Endian:** Network byte order. On x86 (Little-Endian) systems, you must byteswap `ntohs` / `ntohl`.
+*   **Nanosecond Timestamps:** High-precision timing for event sequencing.
+
+---
+
+## 2. Data Types
+
+| Type | Size | Description |
+| :--- | :--- | :--- |
+| **Integer** | 2, 4, 6, 8 bytes | Unsigned integers (Big-Endian). |
+| **Price (4)** | 4 bytes | Integer. Divide by 10,000 to get decimal price (Fixed Point 4). |
+| **Price (8)** | 8 bytes | Integer. Divide by 100,000,000 (Fixed Point 8). Used for high-priced assets. |
+| **Alpha** | Variable | Left-justified ASCII string, padded with spaces. |
+| **Timestamp** | 6 bytes | Nanoseconds past midnight (ET). |
+
+**Note on Timestamps:** ITCH uses a split timestamp format in some contexts, but usually provides a standard nanosecond offset from midnight.
+
+---
+
+## 3. Message Header Structure
+
+Every ITCH message starts with a standard header or is framed within a transport packet (MoldUDP64). At the application payload level, messages are distinguished by a **1-byte Message Type**.
+
+| Offset | Length | Name | Type | Description |
+| :--- | :--- | :--- | :--- | :--- |
+| 0 | 1 | Message Type | Alpha | Identifies the message (e.g., 'A' for Add Order). |
+| 1 | Variable | Payload | Mix | Message-specific data. |
+
+*Note: In raw UDP capture (MoldUDP64), there is a session header before the message stream.*
+
+---
+
+## 4. Critical Message Types
+
+### 4.1 System Event Message ('S')
+Indicates the state of the matching engine.
+*   **'O':** Start of Messages.
+*   **'S':** Start of System Hours.
+*   **'Q':** Start of Market Hours (9:30 AM ET) – Open.
+*   **'M':** End of Market Hours (4:00 PM ET) – Close.
+*   **'E':** End of System Hours.
+*   **'C':** End of Messages.
+
+**Quant Logic:** Use 'Q' and 'M' to trigger trading strategies vs. maintenance mode. Use 'S' to reset books.
+
+### 4.2 Stock Directory Message ('R')
+Sent at the start of the day for every tradable symbol. **Crucial for mapping.**
+*   **Stock Locate (2 bytes):** An integer ID assigned to the symbol for the day.
+*   **Stock (8 bytes):** The ticker symbol (e.g., "AAPL    ").
+*   **Market Category:** NYSE, Nasdaq, Amex, etc.
+
+**Optimization Tip:** Do **not** use string comparisons ("AAPL") in your hot path. Map "AAPL" to `Stock Locate ID` (e.g., 1234) at startup. All subsequent order messages use the `Stock Locate ID`. This allows O(1) array indexing for book lookups.
+
+### 4.3 Trading Action Message ('H')
+Indicates halts and pauses.
+*   **Trading State:**
+    *   'H': Halted (Reg NMS, News, etc.).
+    *   'P': Paused (LULD volatility pause).
+    *   'T': Trading Resumed.
+
+**Risk Check:** Your system **must** immediately block proprietary orders if it receives an 'H' or 'P' for a symbol you trade.
+
+### 4.4 Add Order Message ('A' and 'F')
+Adds a new visible order to the book.
+*   **'A':** No MPID attribution (Anonymous).
+*   **'F':** With MPID attribution (e.g., "GSCO").
+
+**Fields:**
+*   **Order Reference Number (8 bytes):** Unique ID for this order. **Key for tracking.**
+*   **Buy/Sell Indicator:** 'B' or 'S'.
+*   **Shares (4 bytes):** Quantity.
+*   **Stock Locate (2 bytes):** Symbol ID.
+*   **Price (4 bytes):** Limit price.
+
+**Book Logic:** Insert node `(Price, Time, OrderRef)` into your Order Book data structure.
+
+### 4.5 Order Executed Message ('E')
+An order on the book was executed (fully or partially).
+*   **Order Reference Number:** Matches the 'A' message.
+*   **Executed Shares:** Number of shares traded.
+*   **Match Number:** Unique Trade ID.
+
+**Book Logic:** Find the order by `Order Reference Number`. Decrement its size by `Executed Shares`. If size becomes 0, remove it.
+**Note:** The price is implied from the original 'A' message.
+
+### 4.6 Order Executed with Price Message ('C')
+An order on the book was executed at a price **different** from its display price (e.g., price improvement/slippage due to cross).
+*   **Execution Price:** The actual trade price.
+*   **Printable:** 'Y' or 'N' (whether it prints to the tape).
+
+**Book Logic:** Same as 'E' (reduce size), but record the trade at the *Execution Price* for volume/signal analysis.
+
+### 4.7 Order Cancel ('X') vs. Order Delete ('D')
+*   **Order Cancel ('X'):** Partial reduction.
+    *   **Canceled Shares:** Amount to remove.
+    *   **Logic:** Decrement size. If 0, remove.
+*   **Order Delete ('D'):** Full removal.
+    *   **Logic:** Immediately remove order from book.
+
+### 4.8 Order Replace Message ('U')
+Efficiency mechanism. Replaces an existing order with a new one (new Order ID, potentially new size/price).
+*   **Original Order Reference Number:** Old order.
+*   **New Order Reference Number:** New order.
+*   **Shares:** New quantity.
+*   **Price:** New price.
+
+**Book Logic:** Atomic "Delete Old" + "Add New".
+**Important:** The new order loses queue priority (new timestamp).
+
+### 4.9 Net Order Imbalance Indicator ('I') - NOII
+Broadcast before the Open (9:28-9:30) and Close (3:50-4:00) to indicate auction state.
+*   **Paired Shares:** Shares that can match at current Reference Price.
+*   **Imbalance Shares:** Excess buy/sell interest.
+*   **Imbalance Direction:** 'B' (Buy side imbalance), 'S' (Sell side), 'N' (No imbalance).
+*   **Far Price / Near Price / Current Reference Price:** Auction price scenarios.
+
+**Quant Strategy:** NOII data is the primary signal for "Opening" and "Closing" auction arbitrage strategies (MOC/LOC orders).
+
+---
+
+## 5. Building the Order Book (Developer Logic)
+
+To reconstitute the limit order book from ITCH:
+
+1.  **Initialize:** Create an array of OrderBooks, indexed by `Stock Locate ID`.
+    ```cpp
+    std::vector<OrderBook> all_books(MAX_STOCK_LOCATE_ID);
+    ```
+
+2.  **Mapping:** On 'R' (Directory), map Symbol String -> ID.
+    ```cpp
+    map_symbol_to_id["AAPL"] = msg.stock_locate;
+    ```
+
+3.  **State Management:**
+    *   **Map:** `std::unordered_map<uint64_t, Order*> order_map;` (Order Ref -> Order Node).
+    *   **Tree/List:** A sorted structure (e.g., Red-Black Tree or Skip List) for Price Levels.
+
+4.  **Processing Loop:**
+    *   **'A' (Add):**
+        *   Create `Order` object.
+        *   Add to `order_map[ref_num]`.
+        *   Insert into Price Level in `OrderBook`.
+    *   **'E'/'X' (Modify):**
+        *   Lookup `Order*` in `order_map[ref_num]`.
+        *   Modify size.
+        *   If size == 0, delete from Map and Price Level.
+    *   **'D' (Delete):**
+        *   Lookup and remove immediately.
+    *   **'U' (Replace):**
+        *   Treat as Delete(Old) + Add(New).
+
+### Code Snippet: Parsing (C++)
+
+```cpp
+struct ItchHeader {
+    uint16_t length; // Little endian usually if read from network layer wrapper
+    char msg_type;
+};
+
+struct AddOrderMsg {
+    uint32_t timestamp_nanos; // Simplified
+    uint64_t order_ref;
+    uint8_t  buy_sell_flag;
+    uint32_t shares;
+    uint16_t stock_locate;
+    uint32_t price_4;
+};
+
+void process_packet(const char* buffer) {
+    char msg_type = buffer[0];
+    switch (msg_type) {
+        case 'A': {
+            // Cast or memcpy to safe struct
+            // BSWAP (ntohl) fields if necessary
+            // update_book(stock_locate, side, price, size, ref);
+            break;
+        }
+        case 'E': {
+            // handle execution
+            break;
+        }
+        // ... handle others
+    }
+}
+```
+
+## 6. Performance Considerations
+1.  **Zero-Copy:** Do not copy the buffer into a struct. Cast the pointer (if alignment allows) or use `memcpy` for safe unaligned access.
+2.  **Pre-allocation:** Pre-allocate Order nodes in a memory pool to avoid `new`/`delete` latency spikes during high message rates.
+3.  **Instruction Cache:** Keep the 'Add', 'Exec', 'Cancel' handlers hot in cache. These make up 95% of traffic.

Market Making/Quant_Dev_Market_Making_Guide/02_Market_Data_Protocol_ITCH_5_0.md

@@ -0,0 +1,162 @@
+# Quantitative Models & Strategies in Market Making
+
+**Focus:** Mathematical frameworks for quoting, inventory management, and alpha signals.
+**Prerequisites:** Probability theory, stochastic calculus basics, convex optimization.
+
+---
+
+## 1. The Market Maker's Problem
+
+The fundamental goal is to maximize terminal wealth utility while managing inventory risk. This is often modeled as a stochastic optimal control problem.
+
+### 1.1 The Avellaneda-Stoikov Model (2008)
+This is the seminal paper for high-frequency market making.
+
+**Core Assumptions:**
+*   **Mid-price ($S_t$)** follows a geometric Brownian motion or arithmetic Brownian motion:
+    $$dS_t = \sigma dW_t$$
+*   **Arrival Rates ($\lambda$)**: The probability of a limit order being filled follows a Poisson process with intensity decaying exponentially with distance ($\delta$) from the mid-price:
+    $$\lambda(\delta) = A e^{-k\delta}$$
+    Where:
+    *   $\delta$: Spread (Distance from mid-price)
+    *   $A$: Base arrival intensity
+    *   $k$: Order book liquidity parameter (higher $k$ = thinner book)
+
+**The Objective Function:**
+Maximize expected exponential utility of terminal wealth:
+$$u(w) = -e^{-\gamma w}$$
+Where $\gamma$ is the risk aversion parameter.
+
+**The Solution (Optimal Quotes):**
+The optimal bid ($r_b^*$) and ask ($r_a^*$) prices are:
+
+$$r_a^* = S_t + \frac{1}{2}\delta^* + (2q - 1)\frac{\gamma \sigma^2 (T-t)}{2}$$
+$$r_b^* = S_t - \frac{1}{2}\delta^* + (2q + 1)\frac{\gamma \sigma^2 (T-t)}{2}$$
+
+**Key Interpretations:**
+1.  **Reservation Price ($r^*$)**: The price at which the MM is indifferent between buying and selling.
+    $$r^*(S, q, t) = S_t - q \gamma \sigma^2 (T-t)$$
+    *   If inventory $q > 0$ (Long): Reservation price shifts *down*. You are eager to sell, reluctant to buy.
+    *   If inventory $q < 0$ (Short): Reservation price shifts *up*.
+    *   **Term:** $q \gamma \sigma^2 (T-t)$ is the inventory risk premium.
+
+2.  **Optimal Spread ($\delta^*$)**:
+    $$\delta^* = \frac{2}{\gamma} \ln(1 + \frac{\gamma}{k})$$
+    *   The spread is independent of inventory in the standard model.
+    *   It depends on risk aversion ($\gamma$) and market liquidity ($k$).
+
+---
+
+## 2. Inventory Management Strategies
+
+Managing $q$ (inventory) is the single most critical risk task.
+
+### 2.1 Skewing (Asymmetric Quoting)
+As derived above, quotes should be centered around the Reservation Price, not the Mid-Price.
+
+**Logic:**
+*   **Current Inventory:** +1000 shares (Long Limit Reached).
+*   **Action:**
+    *   **Ask:** Aggressive. $S_t$ or even $S_t - \epsilon$. We *need* to sell.
+    *   **Bid:** Passive. $S_t - 5\text{ticks}$. We do *not* want to buy more.
+*   **Effect:** Increases probability of Ask fill, decreases probability of Bid fill.
+
+### 2.2 Damping Factor
+In practice, linear inventory penalties can be too volatile. Firms often use a sigmoid or cubic dampener:
+$$\text{Skew} = \alpha \cdot \tanh(\beta \cdot \frac{q}{Q_{max}})$$
+
+### 2.3 Position Limits & Liquidation
+*   **Soft Limit:** Begin skewing aggressively.
+*   **Hard Limit:** Block new opening orders.
+*   **Liquidation Limit:** Send Immediate-or-Cancel (IOC) market orders to dump inventory if it exceeds critical thresholds (Stop Loss).
+
+---
+
+## 3. Alpha Signals (Short-Term Predictors)
+
+Market making is not just passive. You need "Micro-Alpha" to avoid adverse selection (Toxic Flow).
+
+### 3.1 Order Book Imbalance (OBI)
+The ratio of volume at the Best Bid ($V_b$) vs. Best Ask ($V_a$).
+
+$$OBI = \frac{V_b - V_a}{V_b + V_a}$$
+
+*   **Logic:** If $OBI \to +1$ (Huge Bid, Tiny Ask), price is likely to tick up.
+*   **Action:** Skew quotes upward. Lean on the bid.
+
+### 3.2 Order Flow Toxicity (VPIN)
+Volume-Synchronized Probability of Informed Trading.
+*   Measures order flow imbalance relative to volume.
+*   High VPIN $\implies$ High probability of informed trader presence.
+*   **Action:** Widen spreads.
+
+### 3.3 Cross-Asset Correlations (Lead-Lag)
+*   **Scenario:** SPY (S&P 500 ETF) is the most liquid instrument.
+*   **Signal:** SPY ticks up.
+*   **Effect:** Less liquid constituents (e.g., AAPL, MSFT) will likely tick up 5-50ms later.
+*   **Action:** Immediately cancel/re-price AAPL asks upwards before latency arbitrageurs hit them.
+
+---
+
+## 4. Backtesting Market Making Strategies
+
+Backtesting limit order strategies is notoriously difficult due to "fill simulation."
+
+### 4.1 Assumptions & Pitfalls
+1.  **Queue Position:** You cannot assume you are at the front of the queue.
+    *   *Conservative:* Assume you are at the back (Last in, First Out - LIFO logic for fills).
+    *   *Realistic:* Estimate queue depletion.
+2.  **Market Impact:** Your orders affect the market.
+    *   If you post a huge bid, others might front-run you ("Pennying").
+3.  **Latency:** You see a price $S_t$, but by the time your order reaches the exchange, the price is $S_{t+\Delta}$.
+
+### 4.2 Simulator Design
+*   **Input:** Tick-by-tick data (MBO preferred).
+*   **State:** Reconstructed Order Book.
+*   **Matching Engine:** Replicates exchange matching logic (Price-Time Priority).
+*   **Latency Model:** Adds random jitter + constant delay to all order actions.
+
+---
+
+## 5. Practical Exercise: Python Prototype
+
+```python
+import numpy as np
+
+def calculate_optimal_quotes(mid_price, inventory, volatility, risk_aversion, time_horizon):
+    """
+    Avellaneda-Stoikov Reservation Price & Spread Calculation
+    """
+    # 1. Reservation Price
+    # r* = s - q * gamma * sigma^2 * (T - t)
+    reservation_price = mid_price - (inventory * risk_aversion * (volatility**2) * time_horizon)
+    
+    # 2. Optimal Half-Spread (Simplified approximation)
+    # Assumes k (liquidity parameter) is constant derived from market data
+    k = 1.5  # Example value fitted from trade intensity
+    spread = (2 / risk_aversion) * np.log(1 + (risk_aversion / k))
+    
+    half_spread = spread / 2
+    
+    optimal_bid = reservation_price - half_spread
+    optimal_ask = reservation_price + half_spread
+    
+    return optimal_bid, optimal_ask
+
+# Example
+current_price = 100.00
+current_inventory = 500  # Long 500 shares
+sigma = 2.0  # Daily vol
+gamma = 0.1  # Risk aversion
+T = 1.0     # End of day (normalized)
+
+bid, ask = calculate_optimal_quotes(current_price, current_inventory, sigma, gamma, T)
+
+print(f"Mid Price: {current_price}")
+print(f"Inventory: {current_inventory}")
+print(f"Optimal Bid: {bid:.2f}")
+print(f"Optimal Ask: {ask:.2f}")
+print(f"Skew: {(bid+ask)/2 - current_price:.4f}") 
+# Expect negative skew (lower prices) to shed long inventory
+```
+