Revert pruning PRs (#331)

# Revert #282 and #327 to restore green CI for VecSim tests (via Claude)

## Problem

Private CI `build-share-lib.yml` workflow's `test-shared-libs` jobs
(VecSim tests) — reduced, gcc-11, and gcc-14 on rockylinux8 — started
failing between **Apr 29** (last green: `0ce34e1`) and **May 6** (first
red: `8050bec`).

### Failing tests

- `SVSTest.quant_modes`
- `SVSTest.save_load`

from `VectorSimilarity/tests/unit/test_svs.cpp:2975`, with assertions
like:

```
Expected equality of these values:
  id         Which is: 55
  (idx + 45) Which is: 54
```

## Root cause

The `test-shared-libs` (reduced) job clones `RedisAI/VectorSimilarity`
and runs its test suite against the SVS shared library. The failure
window coincided with SVS commit **`629a79c`** — *"Enhance heuristic
pruning to handle duplicate clusters (#282)"* — which added a
post-pruning step to both `IterativePruneStrategy` and
`ProgressivePruneStrategy` in `include/svs/index/vamana/prune.h`:

> If `all_duplicates && anchor_set && !result.empty()`, overwrite
`result.back()` with the closest candidate from the pool whose distance
≠ `anchor_dist`.

On the VectorSimilarity test fixture (`v[i] = (i,i,i,i)`, dim=4, n=100),
every node has **symmetric pairs of equidistant neighbors** (e.g. nodes
`i-1` and `i+1` both at `d²=4` from node `i`). These look like
"duplicate clusters" to the heuristic, but geometrically they are
symmetric pairs *in opposite directions*, not redundant edges.

The post-prune step:

1. Overwrote a genuine nearest-neighbor edge with a strictly farther
candidate.
2. Biased replacement toward one direction (the first non-tied pool
entry), which systematically removed edges on one side of every node.

**Result:** the constructed Vamana graph is no longer traversable to
certain neighbors for certain queries. For `query = (50,50,50,50)`, the
exact top-10 by ID (`[45..54]`) can't all be reached; search returns
`55` in place of `54`.

## Attempted fixes

### Attempt 1 — `9cee0f1` (#327, "Fix duplicate-cluster trigger to
require >=2 kept candidates")

Tightened the guard from `!result.empty()` to `result.size() >= 2`.
Correctly prevented the branch from firing when only one neighbor was
retained, but did **not** address the core bug: the heuristic still
overwrites a genuine nearest edge with a farther candidate when 2+
retained neighbors tie in distance.

**Result:** the `test-shared-libs` failures persisted.

### Attempt 2 — append-only variant

Branch:
[`dev/eglaser-sharedci-fix`](https://github.com/intel/ScalableVectorSearch/tree/dev/eglaser-sharedci-fix)

Changed both strategies in `include/svs/index/vamana/prune.h`:

- `result.back() = ...` → `result.push_back(...)` (append instead of
overwrite).
- Added `result.size() < max_result_size` guard (only fire when there's
room).

**Rationale:** if every kept neighbor ties at one distance, each one is
a legitimate nearest edge and must not be replaced. Append a diversity
edge only when capacity allows.

**Result:** the failure pattern changed but the test still failed:

| | Top-10 by ID | Missing |
|---|---|---|
| Before attempt 2 | `[45..53, 55]` | vector **54** |
| After attempt 2 | `[46..55]` | vector **45** (entire set shifted +1) |

The same graph-asymmetry problem reappeared on the opposite side because
the appended "diversity" candidate is still systematically biased
(always the first pool entry past the tie band, which stably falls on
one side of the symmetric ring). The heuristic gains an asymmetric edge
toward lower IDs instead of losing an asymmetric edge toward higher IDs.

## Conclusion

Both #327 and the append-only variant confirm that the diversity
heuristic's **detection criterion is fundamentally wrong**: it uses
distance equality to infer directional redundancy, but same-distance
neighbors can be (and in symmetric data, typically are) in *opposite
directions*.

No local tweak to the guard or the insertion method fixes this — the
criterion itself needs redesign to detect **directional collapse**
rather than **distance collapse**. That's a nontrivial design task and
out of scope for an immediate unblock.

## This PR

Reverts:

- **#282** (`629a79c`) — *Enhance heuristic pruning to handle duplicate
clusters*
- **#327** (`9cee0f1`) — *Fix duplicate-cluster trigger to require >=2
kept candidates* (only exists to patch #282)

This restores the pre-April pruning behavior, which restores green CI
for VecSim tests.

## Follow-up

Issue **#80** (the original motivation for #282) should be reopened with
a note that the fix needs to detect **directional** rather than
**distance** redundancy.

Author: ethanglaser

include/svs/index/vamana/prune.h

@@ -130,9 +130,6 @@ void heuristic_prune_neighbors(
 
     auto pruned = std::vector<PruneState>(poolsize, PruneState::Available);
     float current_alpha = 1.0f;
-    float anchor_dist = 0.0f;
-    bool anchor_set = false;
-    bool all_duplicates = true;
     while (result.size() < max_result_size && !cmp(alpha, current_alpha)) {
         size_t start = 0;
         while (result.size() < max_result_size && start < poolsize) {
@@ -148,16 +145,6 @@ void heuristic_prune_neighbors(
             const auto& query = accessor(dataset, id);
             distance::maybe_fix_argument(distance_function, query);
             result.push_back(detail::construct_as(lib::Type<I>(), pool[start]));
-
-            if (all_duplicates) {
-                if (!anchor_set) {
-                    anchor_dist = pool[start].distance();
-                    anchor_set = true;
-                } else if (pool[start].distance() != anchor_dist) {
-                    all_duplicates = false;
-                }
-            }
-
             for (size_t t = start + 1; t < poolsize; ++t) {
                 if (excluded(pruned[t])) {
                     continue;
@@ -184,44 +171,6 @@ void heuristic_prune_neighbors(
         }
         current_alpha *= alpha;
     }
-
-    // Add a diversity edge if a duplicate cluster is detected.
-    // A "cluster" requires at least 2 kept candidates sharing the same
-    // distance; a single retained neighbor is not a cluster and must not
-    // be replaced (doing so would discard the only true nearest-neighbor
-    // edge for that node).
-    if (all_duplicates && anchor_set && result.size() >= 2) {
-        auto result_id = [](const I& r) -> size_t {
-            if constexpr (std::integral<I>) {
-                return static_cast<size_t>(r);
-            } else {
-                return static_cast<size_t>(r.id());
-            }
-        };
-        for (size_t t = 0; t < poolsize; ++t) {
-            const auto& candidate = pool[t];
-            auto cid = candidate.id();
-            if (cid == current_node_id || candidate.distance() == anchor_dist) {
-                continue;
-            }
-            bool in_result = false;
-            for (const auto& r : result) {
-                if (result_id(r) == static_cast<size_t>(cid)) {
-                    in_result = true;
-                    break;
-                }
-            }
-            assert(
-                !in_result &&
-                "Candidate with non-anchor distance should not already be in result"
-            );
-            if (in_result) {
-                continue;
-            }
-            result.back() = detail::construct_as(lib::Type<I>(), candidate);
-            break;
-        }
-    }
 }
 
 template <
@@ -254,9 +203,6 @@ void heuristic_prune_neighbors(
     std::vector<float> pruned(poolsize, type_traits::tombstone_v<float, decltype(cmp)>);
 
     float current_alpha = 1.0f;
-    float anchor_dist = 0.0f;
-    bool anchor_set = false;
-    bool all_duplicates = true;
     while (result.size() < max_result_size && !cmp(alpha, current_alpha)) {
         size_t start = 0;
         while (result.size() < max_result_size && start < poolsize) {
@@ -272,16 +218,6 @@ void heuristic_prune_neighbors(
             const auto& query = accessor(dataset, id);
             distance::maybe_fix_argument(distance_function, query);
             result.push_back(detail::construct_as(lib::Type<I>(), pool[start]));
-
-            if (all_duplicates) {
-                if (!anchor_set) {
-                    anchor_dist = pool[start].distance();
-                    anchor_set = true;
-                } else if (pool[start].distance() != anchor_dist) {
-                    all_duplicates = false;
-                }
-            }
-
             for (size_t t = start + 1; t < poolsize; ++t) {
                 if (cmp(current_alpha, pruned[t])) {
                     continue;
@@ -300,44 +236,6 @@ void heuristic_prune_neighbors(
         }
         current_alpha *= alpha;
     }
-
-    // Add a diversity edge if a duplicate cluster is detected.
-    // A "cluster" requires at least 2 kept candidates sharing the same
-    // distance; a single retained neighbor is not a cluster and must not
-    // be replaced (doing so would discard the only true nearest-neighbor
-    // edge for that node).
-    if (all_duplicates && anchor_set && result.size() >= 2) {
-        auto result_id = [](const I& r) -> size_t {
-            if constexpr (std::integral<I>) {
-                return static_cast<size_t>(r);
-            } else {
-                return static_cast<size_t>(r.id());
-            }
-        };
-        for (size_t t = 0; t < poolsize; ++t) {
-            const auto& candidate = pool[t];
-            auto cid = candidate.id();
-            if (cid == current_node_id || candidate.distance() == anchor_dist) {
-                continue;
-            }
-            bool in_result = false;
-            for (const auto& r : result) {
-                if (result_id(r) == static_cast<size_t>(cid)) {
-                    in_result = true;
-                    break;
-                }
-            }
-            assert(
-                !in_result &&
-                "Candidate with non-anchor distance should not already be in result"
-            );
-            if (in_result) {
-                continue;
-            }
-            result.back() = detail::construct_as(lib::Type<I>(), candidate);
-            break;
-        }
-    }
 }
 
 ///

tests/svs/index/vamana/prune.cpp

@@ -17,10 +17,6 @@
 // header under test
 #include "svs/index/vamana/prune.h"
 
-// core
-#include "svs/core/data/simple.h"
-#include "svs/core/distance/euclidean.h"
-
 // catch2
 #include "catch2/catch_test_macros.hpp"
 
@@ -50,65 +46,4 @@ CATCH_TEST_CASE("Pruning", "[index][vamana]") {
             CATCH_REQUIRE(v::excluded(v::PruneState::Pruned) == true);
         }
     }
-
-    CATCH_SECTION("Duplicate Cluster Trap") {
-        auto data = svs::data::SimpleData<float>(6, 4);
-        auto d0 = std::vector<float>{1.0f, 1.0f, 1.0f, 1.0f};
-        auto d4 = std::vector<float>{2.0f, 1.0f, 1.0f, 1.0f};
-        auto d5 = std::vector<float>{1.5f, 1.0f, 1.0f, 1.0f};
-
-        for (size_t i = 0; i < 4; ++i) {
-            data.set_datum(i, d0);
-        }
-        data.set_datum(4, d4);
-        data.set_datum(5, d5);
-
-        auto dist = svs::distance::DistanceL2();
-        auto accessor = svs::data::GetDatumAccessor{};
-
-        std::vector<svs::Neighbor<size_t>> pool = {
-            {size_t{0}, 0.0f},
-            {size_t{1}, 0.0f},
-            {size_t{2}, 0.0f},
-            {size_t{3}, 0.0f},
-            {size_t{4}, 1.0f}};
-
-        CATCH_SECTION("Iterative Strategy Fix") {
-            std::vector<svs::Neighbor<size_t>> result;
-            v::heuristic_prune_neighbors(
-                v::IterativePruneStrategy{},
-                2,
-                1.3f,
-                data,
-                accessor,
-                dist,
-                size_t{5},
-                std::span<const svs::Neighbor<size_t>>(pool),
-                result
-            );
-
-            CATCH_REQUIRE(result.size() == 2);
-            CATCH_REQUIRE(result[0].id() == 0);
-            CATCH_REQUIRE(result[1].id() == 4);
-        }
-
-        CATCH_SECTION("Progressive Strategy Fix") {
-            std::vector<svs::Neighbor<size_t>> result;
-            v::heuristic_prune_neighbors(
-                v::ProgressivePruneStrategy{},
-                2,
-                1.3f,
-                data,
-                accessor,
-                dist,
-                size_t{5},
-                std::span<const svs::Neighbor<size_t>>(pool),
-                result
-            );
-
-            CATCH_REQUIRE(result.size() == 2);
-            CATCH_REQUIRE(result[0].id() == 0);
-            CATCH_REQUIRE(result[1].id() == 4);
-        }
-    }
 }