feat(quantitative): deflated Sharpe ratio command - #7633
Conversation
Adds obb.quantitative.performance.deflated_sharpe_ratio (Bailey & Lopez de Prado, 2014): the probability that a strategy's true Sharpe exceeds the expected maximum Sharpe of N zero-skill trials, answering whether a result selected as the best of N tested variants reflects skill or selection luck. Adjusts for sample length and return skewness/kurtosis. - helpers.deflated_sharpe_stats(): pure computation, parity-tested at 1e-7 against the numguard reference implementation - DeflatedSharpeModel + POST command in the performance router, following the omega_ratio pattern - trials_sr_std defaults to the null 1/sqrt(n-1); callers who track their optimizer's actual cross-trial Sharpe dispersion can pass it
The first guard compared the standard deviation against eps x scale, which is the residue of a single rounding rather than of the whole sum. Measured over constant series spanning values 1e-7..1e3 and lengths 3..10000, the residue reaches 1.96 eps x scale, so the original threshold still let a flat series through at other lengths: it was calibrated on one series and tested on that same series. n eps x scale keeps a margin of at least 3.9x at every length measured, and a real series with sigma=1e-12 sits more than ten orders of magnitude above it, so nothing legitimate is caught. The test now sweeps values x lengths rather than asserting one point, and was run against the unfixed function, where it fails.
|
Pushed a correctness fix to this branch, found by probing my own implementation rather than by review. A constant return series has no Sharpe ratio, but it did not reach the deflation arithmetic as a The guard now compares the dispersion against the resolution of a float at the scale of the data, scaled by the number of terms summed ( Two notes on the test, since they are the reason this took a second pass:
Happy to split this into a separate commit or reword anything if you would rather review it apart from the feature. |
Why
obb.quantitative.performanceships Sharpe, Sortino and Omega — but no way to answer the question that decides whether any of them mean anything after optimization: was this result selected as the best of N attempts, and does it survive that selection? Under pure noise, the expected maximum Sharpe of N zero-skill strategies grows with N; a raw Sharpe from a parameter sweep is a best-of-N number.What
obb.quantitative.performance.deflated_sharpe_ratio(data=returns, target="close", trials=100)— the Deflated Sharpe Ratio (Bailey & López de Prado, 2014): the probability that the true Sharpe exceeds the expected maximum oftrialszero-skill attempts, adjusting for sample length and return skewness/kurtosis. Returns Sharpe, the expected-max bar, the DSR probability, observations and trials.helpers.deflated_sharpe_stats()holds the pure computation; the router command follows the existingomega_ratiopattern (POST,DeflatedSharpeModel, lazy imports, PythonEx/APIEx examples).trials_sr_std(cross-trial Sharpe dispersion) defaults to the null1/sqrt(n-1); callers who track their optimizer's actual trial spread can pass it through the helper.Tests
extensions/quantitative/tests/test_deflated_sharpe.py: parity at 1e-7 against the numguard reference implementation on a deterministic series, single-trial-reduces-to-PSR, monotonicity in trial count, input guards. 5/5 passing locally alongside the existing helper tests.The same math was merged into ffn last week (pmorissette/ffn#311), and equivalents are under review in backtesting.py (kernc/backtesting.py#1388) and jesse (jesse-ai/jesse#611).
Note: I haven't regenerated the static package reference in this PR — happy to run the build step and push it if you'd like it included, or leave it to your release tooling, whichever matches your process.