Skip to content

feat(quantitative): deflated Sharpe ratio command - #7633

Open
ipezygj wants to merge 2 commits into
OpenBB-finance:developfrom
ipezygj:feat/deflated-sharpe-ratio
Open

feat(quantitative): deflated Sharpe ratio command#7633
ipezygj wants to merge 2 commits into
OpenBB-finance:developfrom
ipezygj:feat/deflated-sharpe-ratio

Conversation

@ipezygj

@ipezygj ipezygj commented Aug 8, 2026

Copy link
Copy Markdown

Why

obb.quantitative.performance ships Sharpe, Sortino and Omega — but no way to answer the question that decides whether any of them mean anything after optimization: was this result selected as the best of N attempts, and does it survive that selection? Under pure noise, the expected maximum Sharpe of N zero-skill strategies grows with N; a raw Sharpe from a parameter sweep is a best-of-N number.

What

obb.quantitative.performance.deflated_sharpe_ratio(data=returns, target="close", trials=100) — the Deflated Sharpe Ratio (Bailey & López de Prado, 2014): the probability that the true Sharpe exceeds the expected maximum of trials zero-skill attempts, adjusting for sample length and return skewness/kurtosis. Returns Sharpe, the expected-max bar, the DSR probability, observations and trials.

  • helpers.deflated_sharpe_stats() holds the pure computation; the router command follows the existing omega_ratio pattern (POST, DeflatedSharpeModel, lazy imports, PythonEx/APIEx examples).
  • trials_sr_std (cross-trial Sharpe dispersion) defaults to the null 1/sqrt(n-1); callers who track their optimizer's actual trial spread can pass it through the helper.
  • scipy only — already a dependency of this extension.

Tests

extensions/quantitative/tests/test_deflated_sharpe.py: parity at 1e-7 against the numguard reference implementation on a deterministic series, single-trial-reduces-to-PSR, monotonicity in trial count, input guards. 5/5 passing locally alongside the existing helper tests.

The same math was merged into ffn last week (pmorissette/ffn#311), and equivalents are under review in backtesting.py (kernc/backtesting.py#1388) and jesse (jesse-ai/jesse#611).

Note: I haven't regenerated the static package reference in this PR — happy to run the build step and push it if you'd like it included, or leave it to your release tooling, whichever matches your process.

Adds obb.quantitative.performance.deflated_sharpe_ratio (Bailey & Lopez
de Prado, 2014): the probability that a strategy's true Sharpe exceeds
the expected maximum Sharpe of N zero-skill trials, answering whether a
result selected as the best of N tested variants reflects skill or
selection luck. Adjusts for sample length and return skewness/kurtosis.

- helpers.deflated_sharpe_stats(): pure computation, parity-tested at
  1e-7 against the numguard reference implementation
- DeflatedSharpeModel + POST command in the performance router,
  following the omega_ratio pattern
- trials_sr_std defaults to the null 1/sqrt(n-1); callers who track
  their optimizer's actual cross-trial Sharpe dispersion can pass it
@CLAassistant

CLAassistant commented Aug 8, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

The first guard compared the standard deviation against eps x scale, which is the
residue of a single rounding rather than of the whole sum. Measured over constant
series spanning values 1e-7..1e3 and lengths 3..10000, the residue reaches
1.96 eps x scale, so the original threshold still let a flat series through at
other lengths: it was calibrated on one series and tested on that same series.

n eps x scale keeps a margin of at least 3.9x at every length measured, and a
real series with sigma=1e-12 sits more than ten orders of magnitude above it, so
nothing legitimate is caught. The test now sweeps values x lengths rather than
asserting one point, and was run against the unfixed function, where it fails.
@ipezygj

ipezygj commented Aug 11, 2026

Copy link
Copy Markdown
Author

Pushed a correctness fix to this branch, found by probing my own implementation rather than by review.

A constant return series has no Sharpe ratio, but it did not reach the deflation arithmetic as a nan: the standard deviation of a constant series is floating-point residue rather than an exact zero, so a flat 0.1% series divided out to a Sharpe of ~1e16. That is finite, so it passed every guard, and the deflated Sharpe came out 1.0 — certainty of a real edge, from the one input that carries no information about one. The statistic exists to catch results that are too good to be true, so answering with maximum confidence there is the worst possible failure direction.

The guard now compares the dispersion against the resolution of a float at the scale of the data, scaled by the number of terms summed (n * eps * scale), rather than against zero. Measured over constant series spanning values 1e-7..1e3 and lengths 3..10000, the residue reaches at most 1.96 eps × scale, so this keeps a margin of at least 3.9x at every length, while a real series with sigma=1e-12 sits more than ten orders of magnitude above the threshold — nothing legitimate is caught, and the test asserts that a quiet series still gets a number.

Two notes on the test, since they are the reason this took a second pass:

  • It sweeps values × lengths instead of asserting one series. My first attempt used a single flat series, passed, and still leaked at other lengths — the residue depends on both the value and the length, so a guard calibrated on one point tests only that point.
  • It was run against the unfixed function and fails there, so it tests the guard rather than merely accompanying it.

Happy to split this into a separate commit or reword anything if you would rather review it apart from the feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants