Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- Notice 2188 ("Up-to-the-second historical data requires additional subscription for the API.") is classified as a data advisory instead of a hard error. TWS sends it and then delivers the historical bars anyway — the account merely lacks the up-to-the-second tail — but `historical_data()` returned `Err` on the notice and discarded the bars that followed, so accounts without the real-time entitlement for a listing got no history at all for those symbols. `DATA_ADVISORY_CODES` widens from `[i32; 2]` to `[i32; 3]`, which is breaking only for code binding the const with an explicit array type (#765).

- The frame reader now validates the 4-byte length prefix before using it, instead of trusting it to size an allocation and a `read_exact`. Nothing bounded it: four garbage bytes were read as a body length of up to 4 GiB, which allocated that much and then blocked until that many bytes arrived — consuming and destroying every real message in between, then yielding one bogus frame with the stream left permanently mis-framed. Because the framing is positional, nothing re-synchronizes it, and a mis-framed protobuf payload still decodes without error (prost skips unrecognized field numbers), so the visible symptom was plausible-looking wrong field values that never recovered. Out-of-range prefixes now raise `Error::InvalidFrame` and drive a reconnect. The cap matches the official client's `Constants.MaxMsgSize` (`EReader.readSingleMessage`, which raises `BAD_LENGTH`).

- A frame that no channel claims is now reported instead of dropped. An unrecognized message id raises a `UNKNOWN_MESSAGE_TYPE_CODE` notice and a warning; a known type with no current subscriber stays at `info`, as before, since that is ordinary steady state. Previously the blocking client logged every such frame at `info` without distinguishing the two, and the async client logged nothing at all — so a desynchronized stream was indistinguishable from an idle one, which is why the incident that prompted this work surfaced data-farm notices and no decode error.
Expand Down
46 changes: 46 additions & 0 deletions src/market_data/historical/async_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -336,6 +336,52 @@ async fn test_historical_data_error_response() {
);
}

/// TWS sends 2188 on the error callback and then delivers the bars: the account
/// may not stream the up-to-the-second tail, but the historical data itself is
/// served. Classified as an error it would end the request before
/// `HistoricalData` arrived, and the caller would see no bars at all.
#[tokio::test]
async fn test_historical_data_up_to_the_second_advisory_is_skipped() {
let message_bus = Arc::new(MessageBusStub::with_ordered_responses(vec![
proto_error_response(
9000,
2188,
"Up-to-the-second historical data requires additional subscription for the API.",
),
proto_response(
IncomingMessages::HistoricalData,
historical_data_response()
.bar(
historical_data_daily_bar("20230315")
.ohlc(185.75, 186.25, 185.50, 186.00)
.volume(1500.0)
.wap(185.85)
.count(150),
)
.encode_proto(),
),
proto_response(
IncomingMessages::HistoricalDataEnd,
historical_data_end_response()
.start_date_str("20230315 09:30:00 UTC")
.end_date_str("20230315 16:00:00 UTC")
.encode_proto(),
),
]));

let client = Client::stubbed(message_bus, server_versions::PROTOBUF_REST_MESSAGES_3);

let result = client
.historical_data(&test_contract(), BarSize::Day)
.duration(Duration::days(1))
.ending(datetime!(2023-03-15 16:00:00 UTC))
.fetch()
.await;

let data = result.expect("advisory 2188 must not abort the request");
assert_eq!(data.bars.len(), 1, "bars sent after the advisory should be returned");
}

#[tokio::test]
async fn test_historical_data_unexpected_response() {
// 1 = TickPrice — wrong type for historical_data.
Expand Down
43 changes: 43 additions & 0 deletions src/market_data/historical/sync_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -509,6 +509,49 @@ fn test_historical_data_error_response() {
assert!(result.is_err(), "Expected error due to error response from server");
}

/// TWS sends 2188 on the error callback and then delivers the bars: the account
/// may not stream the up-to-the-second tail, but the historical data itself is
/// served. Classified as an error it would end the request before
/// `HistoricalData` arrived, and the caller would see no bars at all.
#[test]
fn test_historical_data_up_to_the_second_advisory_is_skipped() {
let (client, _bus) = create_blocking_test_client_with_ordered_proto_responses(vec![
proto_error_response(
9000,
2188,
"Up-to-the-second historical data requires additional subscription for the API.",
),
proto_response(
IncomingMessages::HistoricalData,
historical_data_response()
.bar(
historical_data_daily_bar("20230315")
.ohlc(185.75, 186.25, 185.50, 186.00)
.volume(1500.0)
.wap(185.85)
.count(150),
)
.encode_proto(),
),
proto_response(
IncomingMessages::HistoricalDataEnd,
historical_data_end_response()
.start_date_str("20230315 09:30:00 UTC")
.end_date_str("20230315 16:00:00 UTC")
.encode_proto(),
),
]);

let result = client
.historical_data(&Contract::stock("SPY").build(), BarSize::Day)
.duration(Duration::days(1))
.ending(datetime!(2023-03-15 16:00:00 UTC))
.fetch();

let data = result.expect("advisory 2188 must not abort the request");
assert_eq!(data.bars.len(), 1, "bars sent after the advisory should be returned");
}

#[test]
fn test_historical_data_unexpected_response() {
let message_bus = Arc::new(MessageBusStub::with_responses(vec![
Expand Down
22 changes: 13 additions & 9 deletions src/messages.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1122,12 +1122,14 @@ pub const SYSTEM_MESSAGE_CODES: [i32; 4] = [

/// Data-advisory codes that TWS sends on a request which then proceeds
/// normally. The request is *not* rejected — the advisory announces a
/// fallback (delayed market data) and the requested data follows, so these
/// are informational notices, not errors. Classifying them as errors would
/// fallback (delayed market data, or historical data delivered without its
/// up-to-the-second tail) and the requested data follows, so these are
/// informational notices, not errors. Classifying them as errors would
/// terminate the subscription before its data arrives.
/// - 2188: Up-to-the-second historical data requires additional subscription for the API.
/// - 10089: Requested market data requires additional subscription for API; delayed market data is available.
/// - 10167: Requested market data is not subscribed. Displaying delayed market data.
pub const DATA_ADVISORY_CODES: [i32; 2] = [10089, 10167];
pub const DATA_ADVISORY_CODES: [i32; 3] = [2188, 10089, 10167];

/// Data-farm codes reporting a healthy connection ("…connection is OK").
/// Subset of [`WARNING_CODE_RANGE`]; classified [`ConnectivityStatus::Ok`].
Expand Down Expand Up @@ -1200,7 +1202,7 @@ pub const UNKNOWN_MESSAGE_TYPE_CODE: i32 = -5;
/// 2. [`Warning`](Self::Warning) — 2100..=2169, or code 399 with a `Warning:` line.
/// 3. [`SystemMessage`](Self::SystemMessage) — 1100, 1101, 1102, 1300.
/// 4. [`OrderRejection`](Self::OrderRejection) — 200..=399, excluding the cases above.
/// 5. [`DataAdvisory`](Self::DataAdvisory) — [`DATA_ADVISORY_CODES`] (10089, 10167).
/// 5. [`DataAdvisory`](Self::DataAdvisory) — [`DATA_ADVISORY_CODES`] (2188, 10089, 10167).
/// 6. [`Error`](Self::Error) — everything else.
///
/// Marked `#[non_exhaustive]` so IBKR can introduce new code ranges without a
Expand Down Expand Up @@ -1230,7 +1232,8 @@ pub enum NoticeCategory {
/// Order rejection (codes 200..=399, excluding informational cases by precedence).
OrderRejection,
/// Data advisory ([`DATA_ADVISORY_CODES`]): the request proceeded with a
/// fallback (delayed market data) rather than failing. Informational.
/// fallback (delayed market data, or historical data without its
/// up-to-the-second tail) rather than failing. Informational.
DataAdvisory,
/// Any other error code.
Error,
Expand Down Expand Up @@ -1367,10 +1370,11 @@ impl Notice {

/// Returns `true` if this is a data advisory ([`DATA_ADVISORY_CODES`]).
///
/// Data advisories (codes 10089, 10167) announce that a request proceeded
/// with a fallback — delayed market data instead of real-time — rather
/// than failing. The requested data still follows, so the subscription
/// stays open and the notice is informational, not an error.
/// Data advisories (codes 2188, 10089, 10167) announce that a request
/// proceeded with a fallback — delayed market data instead of real-time,
/// or historical data without its up-to-the-second tail — rather than
/// failing. The requested data still follows, so the subscription stays
/// open and the notice is informational, not an error.
pub fn is_data_advisory(&self) -> bool {
DATA_ADVISORY_CODES.contains(&self.code)
}
Expand Down
Loading