Skip to content

feat(rerank): observe max_chars_per_doc truncation via operation metrics - #3001

Open
chethanuk wants to merge 2 commits into
volcengine:mainfrom
chethanuk:feat/2880-rerank-truncation-counter
Open

feat(rerank): observe max_chars_per_doc truncation via operation metrics#3001
chethanuk wants to merge 2 commits into
volcengine:mainfrom
chethanuk:feat/2880-rerank-truncation-counter

Conversation

@chethanuk

Copy link
Copy Markdown
Contributor

Summary

Follow-up to the max_chars_per_doc cap (#2880). A too-tight cap silently reshapes reranker input; this PR makes it visible in production.

When the cap actually shortens documents inside _rerank_scores, the retriever emits two operation-scoped telemetry counters — rerank.docs_truncated (documents shortened) and rerank.chars_trimmed (characters removed) — which flow through the existing operation-telemetry summary into the Prometheus bridge, mirroring the existing vector.* counters exactly.

How it works

flowchart LR
    A["_rerank_scores (cap > 0)"] --> B["_record_truncation(documents, cap)"]
    B -- "only if docs_truncated &gt; 0" --> C["telemetry.count(<br/>rerank.docs_truncated,<br/>rerank.chars_trimmed)"]
    C --> D["operation telemetry summary<br/>(new rerank section)"]
    D --> E["Prometheus bridge<br/>curated allowlist, emits when value &gt; 0"]
    E --> F["openviking_rerank_docs_truncated_total<br/>openviking_rerank_chars_trimmed_total"]
    F --> G["PromQL: series ABSENT when nothing truncated<br/>coalesce with '... or vector(0)'"]
Loading

Notes

  • Metrics only — zero behavior change. Emitted only when something is actually trimmed, so a disabled cap (default 0) or an untriggered cap stays silent (no metric noise). Because the bridge emits only when value > 0, an untruncated run leaves the series absent (not 0) — documented, with the or vector(0) PromQL idiom for alerts.
  • Safe when no telemetry is bound: get_current_telemetry() returns a disabled no-op collector whose count() is guarded.
  • Distinct from the existing per-call openviking_rerank_* metrics (which use account_id/provider/model_name labels) — no name/registration collision.

Tests / docs

Wired end-to-end with coverage at each hop: retriever counters (emit / silent-when-disabled / silent-when-untriggered / sum-across-docs), counter→summary mapping, and the Prometheus bridge render. Metrics-doc rows added (docs/en|zh/concepts/12-metrics.md). Unit suite green; ruff clean.

Review

Four inline review-bot comments triaged and independently verified. Valid ones fixed (metrics docs corrected to "emitted only on truncation; series absent, not 0" + PromQL note; docstring corrected to name both counters); cap or 0 and a micro-optimization "duplicate scan" declined with rationale (the field is never None; the extra pass over ~20 short abstracts is negligible and the readable split is preferable). No production logic changed.

Stacked on the max_chars_per_doc PR (#2999) — it depends on the char cap. Please review/merge #2999 first; until then this PR's diff includes the v0 commits. Part of #2880.

@chethanuk
chethanuk force-pushed the feat/2880-rerank-truncation-counter branch from 15d4f82 to 603a53c Compare July 4, 2026 02:56
@chethanuk

Copy link
Copy Markdown
Contributor Author

Please review cc: @qin-ctx @ZaynJarvis @zhoujh01 @yufeng201 @chenjw - Please let me know if you require any changes, or if the project is currently accepting contributions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant