feat: Warn when a similarity query does not use the vector index

[islamaliev]

Relevant issue(s)

Resolves #5173

Description

A similarity query uses the vector index only in one shape: a single similarity field, ordered by it descending, with a limit and no filter. Miss any of those and the query still returns the right answer, it just reads the whole collection on the way. Nothing in the response said so, so a query that looks fine in development gets slower as the collection grows.

It now reports a warning in extensions naming which part of the shape ruled the index out, so the author knows what to change. A field with no vector index does not warn, since there is nothing to fall back from.

This also fixes a separate bug it uncovered, worth reviewing on its own. A filtered similarity query was reaching the index when it should not have: the guard against it read the filter from a field that is already cleared by that point, so it never fired. The index returned the nearest documents, the filter then rejected some or all of them, and nothing refilled the gap, so the query could return fewer documents than match, sometimes none, while matching documents sat there unreturned. It now reads the filter from where it actually lives. This is on develop today.

Draft until #5189 merges, which this builds on.

Tasks

• I made sure the code is well commented, particularly hard-to-understand areas.
• I made sure the repository-held documentation is changed accordingly.
• I made sure the pull request title adheres to the conventional commit style (the subset used in the project can be found in tools/configs/chglog/config.yml).
• I made sure to discuss its limitations such as threats to validity, vulnerability to mistake and misuse, robustness to invalidation of assumptions, resource requirements, ...

How has this been tested?

Integration tests for each shape that falls back, plus the shape the index serves as the control, so the tests fail if every query starts warning. Test support was added so a request asserts its warnings, with expecting none as the default, which caught existing tests that had started warning.

The fallback tests run on the HTTP, CLI and C clients. That is what proves the warning survives serialization, which the extensions work could not test on its own.

The data loss fix has a regression test that fails without it: a filter matching one document the graph would not have returned.

Full integration suite green.

Specify the platform(s) on which this was tested:

• macOS

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.41%. Comparing base (de03fa3) to head (3421c56).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #5197      +/-   ##
===========================================
+ Coverage    76.36%   76.41%   +0.04%     
===========================================
  Files          648      648              
  Lines        51753    51753              
===========================================
+ Hits         39521    39542      +21     
+ Misses        8896     8881      -15     
+ Partials      3336     3330       -6     

Flag	Coverage Δ
all-tests	76.41% <0.00%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 9 files with indirect coverage changes

---

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update de03fa3...3421c56. Read the comment docs.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[islamaliev]

is already resolved as part of #5189