Add option for immediate execution with quotes in backtesting

[faysou]

Pull Request

NautilusTrader prioritizes correctness and reliability, please follow existing patterns for validation and testing.

• I have reviewed the CONTRIBUTING.md and followed the established practices

Summary

Add option for immediate execution with quotes inside bid ask spread

This PR adds a new allow_immediate_quote_execution configuration option to the backtest engine that enables strategies to receive quotes and submit orders before the exchange processes the quote data. This feature allows orders to be filled on the same tick in discrete-time backtests (e.g., minute, hourly, or daily data) and extends BestPriceFillModel to support execution within the bid-ask spread.

Problem

In discrete-time backtests with sparse data (e.g., quotes every minute or hour), the default execution flow processes quotes through the exchange first, then notifies strategies. When a strategy submits an order in response to a quote tick, the order is queued and only processed after the exchange has already processed that quote. This means the order cannot be filled until the next quote arrives, causing a one-tick delay that may not accurately reflect realistic trading behavior where orders can be filled immediately when market conditions are favorable.

Additionally, BestPriceFillModel previously only matched orders at the best bid or ask prices, which prevented orders with prices inside the spread from being filled, even though such fills are realistic in many market scenarios.

Solution

Introduced a new allow_immediate_quote_execution configuration flag that changes the execution order when enabled: strategies receive quotes first (through the data engine), can submit orders immediately, and these orders are then available when the exchange processes the quote. This allows orders to be filled on the same tick when the fill model determines they are matchable. The feature also extends BestPriceFillModel to allow execution within the bid-ask spread for limit orders.

Implementation details

Backtest engine configuration

• Added allow_immediate_quote_execution: bool parameter to BacktestEngineConfig (default False) that controls whether strategies can receive quotes and submit orders before exchange processing.

• When allow_immediate_quote_execution=True, the backtest engine automatically disables the message queue (use_message_queue=False) to allow immediate order acceptance and processing.

• Updated BacktestEngine.add_venue() to compute effective_use_message_queue based on the config flag, ensuring that immediate execution takes precedence over the explicit use_message_queue parameter.

Data engine changes

• Added allow_immediate_quote_execution: bool parameter to DataEngine.__init__() to receive the configuration flag.

• Modified DataEngine._handle_spread_quote() to process quotes through the data engine first (notifying strategies) before sending to the exchange when immediate execution is enabled. This ensures strategies can submit orders that will be available when the exchange processes the quote.

• When immediate execution is disabled (default), quotes are sent to the exchange first, then processed by the data engine, maintaining backward-compatible behavior.

Fill model enhancements

• Extended BestPriceFillModel.fill() to allow execution within the bid-ask spread for limit orders.

• For BUY limit orders: if the order price is within or above the spread (bid <= price <= ask), the order is matchable. Orders at or above the ask are filled at the ask (aggressive), while orders within the spread are filled at the order price (optimistic).

• For SELL limit orders: if the order price is within or below the spread (bid <= price <= ask), the order is matchable. Orders at or below the bid are filled at the bid (aggressive), while orders within the spread are filled at the order price (optimistic).

• Orders outside the spread remain passive and are not immediately matchable, consistent with realistic market behavior.

Execution engine improvements

• Refactored code formatting and variable declarations for consistency and readability.

• Improved conditional logic with elif chains where appropriate to reduce nesting and improve code clarity.

Testing infrastructure

• Added comprehensive test suite tests/unit_tests/backtest/test_immediate_quote_execution.py (353 lines) that validates immediate execution behavior.

• Tests verify that with allow_immediate_quote_execution=True, orders fill on the same quote tick they are submitted.

• Tests verify that with allow_immediate_quote_execution=False (default), orders fill on the next quote tick, maintaining backward compatibility.

• Added comparison test that demonstrates the timestamp difference between enabled and disabled modes.

• Enhanced tests/unit_tests/backtest/test_fill_models.py (109 new lines) with additional tests for fill model behavior.

Example notebook updates

• Updated examples/backtest/notebooks/databento_option_greeks.py to demonstrate the new immediate execution feature (49 lines modified).

System kernel integration

• Updated nautilus_trader/system/kernel.py to pass the allow_immediate_quote_execution configuration to the data engine (6 lines modified).

Benefits

• Discrete-time backtest accuracy: Enables more realistic backtesting on sparse data (minute, hourly, daily quotes) where strategies can react immediately to market data.

• Reduced execution delay: Orders can be filled on the same tick when market conditions are favorable, eliminating artificial one-tick delays.

• Spread execution support: BestPriceFillModel now supports execution within the bid-ask spread, providing more realistic fill behavior for limit orders.

• Backward compatibility: Default behavior remains unchanged (allow_immediate_quote_execution=False), ensuring existing backtests continue to work as expected.

• Configurable behavior: Users can opt-in to immediate execution when needed without affecting existing workflows.

• Comprehensive testing: New test suite ensures correct behavior and prevents regressions.

Files changed

• examples/backtest/notebooks/databento_option_greeks.py: Updated example to demonstrate immediate execution feature (49 lines modified).

• nautilus_trader/backtest/config.py: Added allow_immediate_quote_execution configuration parameter (5 lines added).

• nautilus_trader/backtest/engine.pyx: Modified execution flow to support immediate quote execution (240 lines modified).

• nautilus_trader/backtest/models/fill.pyx: Extended BestPriceFillModel to allow execution within bid-ask spread (28 lines modified).

• nautilus_trader/data/engine.pxd: Added parameter declaration to data engine interface (1 line added).

• nautilus_trader/data/engine.pyx: Modified quote processing order when immediate execution is enabled (14 lines modified).

• nautilus_trader/execution/engine.pyx: Code formatting and consistency improvements (51 lines modified).

• nautilus_trader/system/kernel.py: Updated to pass configuration to data engine (6 lines modified).

• tests/unit_tests/backtest/test_fill_models.py: Added new fill model tests (109 lines added).

• tests/unit_tests/backtest/test_immediate_quote_execution.py: New comprehensive test suite for immediate execution feature (353 lines added).

Related Issues/PRs

Type of change

• Bug fix (non-breaking)
• New feature (non-breaking)
• Improvement (non-breaking)
• Breaking change (impacts existing behavior)
• Documentation update
• Maintenance / chore

Breaking change details (if applicable)

Documentation

• Documentation changes follow the style guide (docs/developer_guide/docs.md)

Release notes

• I added a concise entry to RELEASES.md that follows the existing conventions (when applicable)

Testing

Ensure new or changed logic is covered by tests.

• Affected code paths are already covered by the test suite
• I added/updated tests to cover new or changed logic