Skip to content

fix(tdigest): interpolate quantiles toward the nearer centroid and normalize the left tail - #250

Merged
tisonkun merged 6 commits into
apache:mainfrom
jaideeppyne:fix/tdigest-quantile-interpolation
Aug 31, 2026
Merged

fix(tdigest): interpolate quantiles toward the nearer centroid and normalize the left tail#250
tisonkun merged 6 commits into
apache:mainfrom
jaideeppyne:fix/tdigest-quantile-interpolation

Conversation

@jaideeppyne

@jaideeppyne jaideeppyne commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Correct the interior quantile interpolation weights so the estimate moves toward the nearer centroid as rank increases.
  • Fix the mirrored tail formulas: subtract from max in the right quantile tail and normalize the left rank tail by total weight.
  • Handle the weight-two right-tail boundary without evaluating an indeterminate 0 / 0 expression.
  • Add focused exact regressions and monotonicity properties for directly updated and merged digests.
  • Document the bug fix in the changelog.

Why

The interior interpolation paired each centroid mean with the distance from that same side of the bracket. Linear interpolation needs the opposite-side distance, so the previous result could move toward the lower centroid as rank increased and make quantiles non-monotonic.

The left and right tail branches contained two independent mirror errors. The right-tail formula also degenerates when the last centroid has weight two: at the only boundary that reaches the formula, both its numerator and denominator are zero. Returning the known maximum at that boundary keeps the handling local and avoids changing interpolation elsewhere.

Validation

  • cargo x check
  • cargo x test
  • cargo x lint
  • Exact interior, tail, and weight-two boundary regressions
  • QuickCheck monotonicity properties for digests built through both update and merge

@tisonkun tisonkun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution! Merging ...

@tisonkun
tisonkun enabled auto-merge (squash) August 31, 2026 09:53
@tisonkun
tisonkun merged commit 7025bf6 into apache:main Aug 31, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants