added clear on the w.items slice (#1016)

* added clear on the w.items slice

---------

Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
Co-authored-by: Ashwin Nambiar <Ashwin.Nambiar@ibm.com>
Co-authored-by: Sean Chittenden <sean.chittenden@crowdstrike.com>

Author: 3 people

CHANGELOG.md

@@ -244,6 +244,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ### Fixed
 
+- Fix `opensearchutil.BulkIndexer` retaining a worker's peak batch memory after a traffic burst subsides. `(*worker).flush` released a completed batch with `w.items = w.items[:0]`, which keeps the slice's backing array -- and every `BulkIndexerItem` it holds, including each item's `Body` (an `io.ReadSeeker` over the caller's document bytes) and its `OnSuccess`/`OnFailure` closures -- reachable until a later batch of equal or greater size overwrites the slots. A worker that peaked at N items during a backlog replay stayed pinned at ~N items' worth of document bodies and closures indefinitely, even after traffic dropped. `flush` now `clear`s the item slice before truncating, dropping those references so the GC can reclaim them ([#912](https://github.com/opensearch-project/opensearch-go/issues/912))
 - Fix an unbounded connection/heap leak in node discovery when the cluster has a dedicated cluster manager (`cluster_manager` role with no work roles). The node was filtered out of the `allConns` inventory while the router received the unfiltered added/removed diffs, so `findConnectionByURL` never matched it: a new `*Connection` was created every discovery cycle and the stale one was never evicted, accumulating without bound in the round-robin fallback pool whose `checkDead` health checks repopulated a per-connection `poolRegistry` `sync.Map` each cycle (leak rate scaled with discovery frequency). `allConns` is now the full connection inventory so discovery reuses and evicts symmetrically, and dedicated cluster managers are excluded at request-routing selection instead: `RoundRobinPolicy` skips them in its `DiscoveryUpdate` add path and `multiServerPool.Next()` skips non-seed dedicated cluster managers during selection (including the no-router fallback). A user-supplied seed is exempt, so discovery still bootstraps against a dedicated cluster manager seed. See also the `IncludeDedicatedClusterManagers` removal under Removed ([#1004](https://github.com/opensearch-project/opensearch-go/pull/1004))
 - Fix `cmd/osgen` silently dropping typed structs on Go type-name collisions, and add a completeness guard so future collisions fail generation instead of degrading output. Two distinct spec schemas that derived the same Go name were reduced to one by the type registry, dropping the other to raw `json.RawMessage` (or mis-typing a field): the search `profile` container collided with the per-search `SearchProfile` item, and multiple response bodies in the `flow_framework.common` and `security_analytics.findings` groups all derived `<Group>Resp`. Colliding refs are now disambiguated via documented override tables (`typeNameCollisions` / `respTypeNameCollisions`), and a panic-guard requires any new colliding ref to be enumerated. Regenerating restores typed responses for the affected operations (e.g. flow_framework `search`/`search_state`, security_analytics `search_finding_correlations`, and the search response's `profile` field), and a response schema also referenced structurally (a search hit's `_source`) is now emitted as a standalone type instead of dangling ([#989](https://github.com/opensearch-project/opensearch-go/pull/989))
 - Fix plugin dispatch methods discarding the transport response, leaving `Inspect().Response` nil on every typed plugin response. The generated plugin dispatch template dropped the `*opensearch.Response` returned by `request()` (`if _, err := request(...)`); it now assigns it (`resp.response, err = request(...)`), matching the core client. Also serialize the discovery-path warmup recalculation and ready-list partitioning in `createOrUpdateMultiNodePoolWithLock` under the pool write lock -- `recalculateWarmupParamsWithLock`/`getWarmupParamsWithLock` (renamed to reflect the requirement) and the `mu.activeCount` write touched `mu`-guarded fields without holding `pool.mu`, racing `resurrectWithLock` (follow-up to [#981](https://github.com/opensearch-project/opensearch-go/pull/981)) ([#989](https://github.com/opensearch-project/opensearch-go/pull/989))

opensearchutil/bulk_indexer.go

@@ -569,6 +569,7 @@ func (w *worker) flush(ctx context.Context) error {
 	)
 
 	defer func() {
+		clear(w.items)
 		w.items = w.items[:0]
 		w.buf.Reset()
 	}()