Skip to content

Add BCa-at-scale section to quantiles user guide#30

Merged
rwilson4 merged 2 commits into
masterfrom
claude/quantile-bca-closed-form
May 7, 2026
Merged

Add BCa-at-scale section to quantiles user guide#30
rwilson4 merged 2 commits into
masterfrom
claude/quantile-bca-closed-form

Conversation

@rwilson4
Copy link
Copy Markdown
Owner

@rwilson4 rwilson4 commented May 7, 2026

Summary

Follow-up to #29. Extends docs/user_guides/quantiles_at_scale.rst with a "BCa intervals at scale" section that closes the BCa loop the previous guide left open.

The previous guide showed how to bypass the bootstrap loop for sample quantiles via the Bin(n+1, q) index trick, but punted on BCa: the jackknife step is O(n) statistic evaluations and dominates at large n. This section derives a closed-form jackknife and a closed-form acceleration that eliminate that step entirely.

What's in the new section

  • Closed-form jv. Removing observation X_i either leaves the leave-one-out quantile at X_{(k')} or shifts it to X_{(k'+1)} depending on whether X_i's rank is above or at-or-below k' = ceil((n-1) q). Two distinct values across the entire jackknife array; no statistic evaluations.

  • Closed-form a_hat. Plugging the two-valued jv into the BCa acceleration sum, the data-dependence δ³ cancels via affine invariance, leaving

    a_hat = (2k' - n) / (6 * sqrt(n * k' * (n - k')))
    

    Depends only on n and q. Falls out: median ⇒ a_hat = 0 (BCa→BC), |a_hat| grows for tail quantiles, a_hat = O(1/sqrt(n)) so BCa→percentile asymptotically.

  • Validation table. Closed-form a_hat agrees with bootstrap_stat.jackknife_values to 6 decimals at n = 200, 1000, 10000, 100000 (verified empirically before drafting).

  • General-principle subsection ties the two-hook BCa pattern back to zero_inflated.rst: same recipe (split theta_star and jv, exploit structure on each axis), different exploitable structure (linearity vs. ranks).

  • Caveats updated to drop the obsolete "BCa is out of scope" paragraph and replace it with a note about matching the quantile convention (method='inverted_cdf') across the bootstrap, observed q_hat, and jackknife.

Test plan

  • cd docs && uv run make html builds cleanly (4 unrelated intersphinx warnings from sandbox).
  • Numerical verification: closed-form jv matches jackknife_values() exactly at n ∈ {200, 1000, 10000, 100000}; closed-form a_hat matches both to 6 decimals.
  • Eyeball the rendered HTML on the deployed site after merge — section structure uses the same ^^^^ underline depth as the rest of the guide, but worth a glance.

https://claude.ai/code/session_01CRkE1qgfLrZz9oxcfA2Msu


Generated by Claude Code

claude added 2 commits May 7, 2026 18:10
Derives the closed-form jackknife for the q-th order-statistic
quantile (two-valued: X_{(k')} for ranks above k', X_{(k'+1)} for
ranks at or below) and the resulting closed-form acceleration

    a_hat = (2k' - n) / (6 * sqrt(n * k' * (n - k'))).

Data dependence cancels via affine invariance, leaving an
expression that depends only on n and q. Numerical agreement with
bootstrap_stat.jackknife_values is exact to floating-point
precision across n = 200 to 100,000.

Pairing this with the index-trick bootstrap and passing both
through the existing theta_star= / jv= hooks on bcanon_interval
gives a BCa pipeline with no resampling and no jackknife, mirroring
the zero-inflated pattern. Updated the caveats section accordingly.

https://claude.ai/code/session_01CRkE1qgfLrZz9oxcfA2Msu
The validation table established correctness; this records the
speedup. At n=200,000, B=500: library default ~5 min (99%+ in the
jackknife), theta_star + closed-form jv ~35 ms — roughly 10,000x.
Pairs naturally with the algebraic argument that BCa decomposes
into two independently-acceleratable steps.

https://claude.ai/code/session_01CRkE1qgfLrZz9oxcfA2Msu
@rwilson4 rwilson4 merged commit dc832a5 into master May 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants