You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Every position/fill event during a backtest clones the entire account object, which carries an append-only events: Vec that grows for the whole run. Since the clone copies that growing vector each time, per-fill cost grows with the number of prior events - O(n²) over the length of the backtest. It's invisible at the call site; it reads as "just cloning an account."
My understanding
BaseAccount (nautilus-model, src/accounts/base.rs) holds events: Vec, append-only (self.events.push(event)), untrimmed by default.
nautilus-portfolio::portfolio::update_position() (src/portfolio.rs, ~2562) calls cache.borrow().account_owned(&account_id) — an owned clone of the whole AccountAny / BaseAccount including that vector - on every position event, where most other accessors (Cache::account_ref) return a borrow.
Impact
Profiling a multi-year 1-minute backtest (~2.46M bars, ~1.1k trades) with samply, nautilus_model::accounts::{base,any}::clone accounted for ~15% of total backtest CPU time, scaling with backtest length.
Existing mitigation and its limit
ExecutionEngineConfig::purge_account_events_interval_mins / purge_account_events_lookback_mins (both default None = off) drive BaseAccount::base_purge_account_events to trim the history, bounding the O(n²) growth. But the clone still happens every fill - purging shrinks the payload, not the copy. For a backtest that never reads account.events(), the whole history is copied for data nothing consumes.
This was my solution and it works fine, but it took quite a bit to become aware of the issue and solve it.
Understood to be by design - the history backs last_event(), Portfolio::statistics(), and live reconciliation. Filing to surface the hidden cost and ask whether:
the default could enable a bounded lookback (unbounded-by-default is a footgun for long backtests), and/or
update_position could borrow instead of taking an owned clone, since it doesn't appear to need ownership past the RefCell borrow.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Summary
Every position/fill event during a backtest clones the entire account object, which carries an append-only events: Vec that grows for the whole run. Since the clone copies that growing vector each time, per-fill cost grows with the number of prior events - O(n²) over the length of the backtest. It's invisible at the call site; it reads as "just cloning an account."
My understanding
BaseAccount (nautilus-model, src/accounts/base.rs) holds events: Vec, append-only (self.events.push(event)), untrimmed by default.
nautilus-portfolio::portfolio::update_position() (src/portfolio.rs, ~2562) calls cache.borrow().account_owned(&account_id) — an owned clone of the whole AccountAny / BaseAccount including that vector - on every position event, where most other accessors (Cache::account_ref) return a borrow.
Impact
Profiling a multi-year 1-minute backtest (~2.46M bars, ~1.1k trades) with samply, nautilus_model::accounts::{base,any}::clone accounted for ~15% of total backtest CPU time, scaling with backtest length.
Existing mitigation and its limit
ExecutionEngineConfig::purge_account_events_interval_mins / purge_account_events_lookback_mins (both default None = off) drive BaseAccount::base_purge_account_events to trim the history, bounding the O(n²) growth. But the clone still happens every fill - purging shrinks the payload, not the copy. For a backtest that never reads account.events(), the whole history is copied for data nothing consumes.
This was my solution and it works fine, but it took quite a bit to become aware of the issue and solve it.
Understood to be by design - the history backs last_event(), Portfolio::statistics(), and live reconciliation. Filing to surface the hidden cost and ask whether:
Beta Was this translation helpful? Give feedback.
All reactions