Skip to content

The advertised BCa interval is never BCa in the default pipeline: the acceleration term is silently dropped above 60 grid points #116

Description

@marcohost33-maker

Summary

Found while closing out the relaxation-window unit dependence (PR #115). compute_relaxation_layer computes the jackknife — the input to the BCa acceleration term a — only for short grids:

jk = None
if t_grid.size <= 60:
    jk = _jackknife(winner_fn, t_grid, rel_entropy, theta_hat, None)
cis = bca_ci(samples, theta_hat, jackknife_estimates=jk)

bca_ci then takes the documented a = 0.0 fallback:

if jackknife_estimates is None:
    a = 0.0

The default grid is 80 points, so the branch is never taken in the default pipeline. With a = 0 the interval reduces to a bias-corrected (BC) interval — the "a" of BCa is exactly the acceleration. Nothing in the code, the field name, the docstring or the docs says so.

The bare if t_grid.size <= 60: carries no comment, and there is no CHANGELOG or docs entry for it, so this reads as an unintended consequence of the grid-size default rather than a declared trade-off.

Measured

Instrumenting bca_ci on an amplitude-damped qubit through the default path:

default grid       : span 20.0, n_points 80
jackknife supplied : [False]
=> acceleration a  : 0.0  (BC, not BCa)
reported bca_ci_beta: (1.0270445499117724, 1.0284105734239335)

claim_status: pending.

Why it matters here specifically

This is a labelling/claim problem more than a numerical one — a BC interval is a legitimate estimator, just not the one advertised. But "explicit uncertainty" is one of the three pillars the library states for itself, and the BCa label is on the public surface in at least six places:

  • RelaxationResult.bca_ci_beta (field name) and its # BCa 95% CI comment
  • README.md — "Explicit uncertainty. Bootstrap CIs (BCa) …", the quickstart comment, and the U/G layer table row
  • docs/explanation/no-single-number.md — "Fitted quantities carry BCa bootstrap intervals"
  • docs/tutorials/first-diagnostic-run.md — "bootstrap_B controls the parametric bootstrap used for the BCa confidence …" and "# 95% BCa confidence interval (lo, hi)"

UncertaintyResult.fit_uncertainty (U0) is derived from this interval's half-width, so the uncertainty layer inherits the same label.

Direction of the error is not determined a priori: a corrects for a variance that changes with the parameter value, and dropping it skews both endpoints in whichever direction the skew runs. It is not conservative by construction.

Cause

Almost certainly cost. _jackknife refits the model once per retained sample — 80 extra GLS fits per run on the default grid, on top of bootstrap_B — so the 60-point gate looks like a latency guard that predates the grid default being 80.

Note that PR #115 makes the default span physics-scaled but leaves RELAXATION_N_POINTS = 80 unchanged, so it neither causes nor fixes this.

Suggested direction (not decided)

Roughly in increasing order of work:

  1. Say what is computed. Report which estimator was used (e.g. an interval_method: "BC" | "BCa" field alongside bca_ci_beta, in the same additive style as t_grid_source in fix: scale the default relaxation window to the system's own relaxation time #115) and align the README/docs wording. Cheapest, and restores the honesty invariant immediately.
  2. Raise or drop the gate and accept the cost, possibly with the jackknife computed only for the winning model (already the case) and only when bootstrap_B indicates the caller wants inference quality.
  3. Replace the leave-one-out jackknife with a cheaper acceleration estimate — e.g. the empirical-influence / infinitesimal-jackknife form, which can be obtained from the existing bootstrap replicates without additional fits, and is the standard route when leave-one-out is too expensive.

Option 1 is independent of the others and could land on its own; 2 and 3 change numbers and would need anchor review.

Scope note

Deliberately not bundled into PR #115, which is scoped to the time-grid unit dependence and does not touch the bootstrap path. Whichever option is chosen changes either a public claim or the reported interval, so it warrants its own reviewable change.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions