Skip to content

[BUG] Client does not detect result-shape mismatch when RrfRank compositions degenerate #500

@tazarov

Description

@tazarov

Summary

When arithmetic methods on *RrfRank are composed in mathematically degenerate ways (e.g. rrf.Log(), rrf.Max(Val(0))), the Go client accepts the resulting query, sends it, receives a well-formed but semantically degenerate response, and surfaces it to the caller with no signal that anything went wrong.

The server is NOT buggy — this was the original framing of this issue, but research against the official docs and the chroma-core/chroma Rust source corrected the picture:

  • Official docs (trychroma.com/cloud/search-api/overview): "Results are ordered by score (ascending - lower is better)".
  • The Rust server's rrf() at rust/types/src/execution/operator.rs explicitly negates its sum (Ok(-sum)) to match the lower-is-better convention.
  • Arithmetic composition on top of an auto-negated (≤ 0) value then hits deterministic IEEE 754 behavior:
    • f32::ln(negative) = NaN → NaN rows are dropped, leaving an empty inner Scores slice visible to the client.
    • f32::max(negative, 0) = 0 → every score collapses to exactly 0, insertion-order IDs.
    • f32::abs(negative) = -negative → ordering reverses.

This is math, not a bug. The bug is on the client side: the Go client has no defense-in-depth check that catches result-shape degeneration before handing the response back to the caller.

Scope (narrowed)

This issue now covers one focused, orthogonal client-side defense:

Detect result-shape mismatch on Search responses. If the response has a populated inner IDs[0] but an empty or otherwise cardinality-mismatched inner Scores[0], wrap the response with a descriptive error so the caller sees the degeneration instead of silently receiving a broken result.

This is defense-in-depth against ANY future degenerate path (not just Log / Max(0)). It complements but does not duplicate #501's build-time rejection of known-degenerate compositions.

Repro

See TestCloudClientSearchRRFArithmetic in pkg/api/v2/client_cloud_test.go — the Log case currently pins:

require.NotEmpty(t, sr.IDs, "Log: outer IDs must not be empty")
require.Equal(t, []DocumentID{"1", "2", "3", "4", "5"}, sr.IDs[0], ...)
require.Empty(t, sr.Scores[0], "Log: inner Scores must be empty (degenerate)")

Once this fix lands, those assertions flip: Search should return an error on the Log row (or a structured warning rank), and the test should assert the error path instead of pinning the degenerate response.

Relationship to #501

Both are valuable independently:

Phase Placement

Phase 29 (roadmap). Should be implemented alongside #501 so both guards land together.

Discovery

Observed during Phase 21.1 cloud test execution; framing corrected during PR review of #496 after cross-referencing chroma-core/chroma Rust source.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions