[PROPOSAL] Drop OpenSearch < 2.18 from the v5 client

[sean-]

Summary

The v5 client (v5preview/opensearchapi) supports only OpenSearch releases that are currently receiving patches at the time of each opensearch-go release. Users on older releases stay on the v4 client.

As of today (per https://opensearch.org/releases/) that means:

• v4 (current GA): supports OpenSearch 1.3.20+, 2.x, 3.x. No change.
• >=v5: supports 2.19.x and 3.x. Re-evaluated at every tagged release of opensearch-go.

Each opensearch-go release pins its supported set to the OpenSearch project's then-current "still patched" lines. When OpenSearch drops a line from patches, the next opensearch-go release drops it from CI, the spec floor, and the docs.

Why a moving window instead of a fixed floor

Rather than re-pin to a new fixed number that will go stale the same way, tie the supported set to whatever OpenSearch itself is patching.

Concrete benefits:

1. CI cost and signal: today's matrix spins up ~70 workers across versions that the project itself no longer ships fixes for. Most of the failures we chase on those legacy lanes are server-side bugs that will never be patched. Trimming to currently-patched lines gives faster, cleaner signal on bugs we can actually fix.
2. Spec / codegen alignment: as we add x-version-added / x-version-deprecated annotations to the spec and generated code, per-minor-version test coverage becomes redundant with the annotations. We test the supported set; the annotations document the rest.
3. Predictable maintenance: contributors know exactly which versions a PR has to work on (the currently-patched set), instead of inheriting an open-ended historical compatibility burden.
4. No surprises for users: anyone on a release that OpenSearch itself has stopped patching is already off the supported path; opensearch-go just stops paying CI cost to pretend otherwise. Those users have v4 as the documented fallback until they upgrade.

Test-suite signal

The data motivating this change is the test-skip volume. Numbers are from v5preview/opensearchapi/ on this branch.

Metric	Count
*_integ_gen_test.go files in v5preview	480
testutil.SkipIfVersion(...) call sites	214
Direct t.Skip(...) call sites (fixture / known-bug)	374

Distribution of SkipIfVersion(t, c, "<", "X.Y", ...) thresholds:

Floor	Sites	Status under proposal
1.1 - 2.18	174	Deleted (unconditional)
2.19	20	Kept (>= 2.19 is current)
3.0 - 3.4	37	Kept
2.18 exact	3	Deleted
Total	214

Plugin areas (ML, flow_framework, search_relevance, LTR, security, ...) hold most of the version churn. Tightening the supported set collapses most of that variance.

Proposal

1. Define the v5 supported set as the OpenSearch releases currently receiving patches within the last 12mo of each opensearch-go release (e.g. test against any OpenSearch version released in the last 12mo).
2. Update COMPATIBILITY.md and UPGRADING.md:
   • v4: OpenSearch 1.3.20 - 3.x (unchanged).
   • v5: tracks the current OpenSearch supported releases. Today: 2.19.x and 3.x.
3. Trim CI to the current supported set. The next release re-evaluates the matrix.
4. Delete SkipIfVersion sites that gate on versions outside the supported set; let those tests run unconditionally.
5. Re-evaluate the "response struct does not match" skips against the supported set; fix at the codegen / spec level instead of skipping.
6. Lean on x-version-added / x-version-deprecated annotations in the spec and generated code to document version provenance, instead of using CI matrix breadth as a substitute.
7. Update generator templates to stop emitting SkipIfVersion(... < <below-current> ...) for new endpoints.

Non-goals

• No change to the v4 client. Users on lines OpenSearch no longer patches stay on v4.

Migration story for users

• "I'm on OpenSearch 1.3 / 2.0 / 2.7 / ... (a line OpenSearch no longer patches)" -> use opensearch-go v4.
• "I'm on OpenSearch 2.19 or 3.x" -> v5 is the supported path.

[sean-]

Ref's #835

[Jakob3xD]

Where did you find that 2.18 is the current LTS? Looking at https://opensearch.org/releases/, 2.18 does not receive any patches. Only 2.19.x gets patched. Therefore, I would assume 2.19.x are the versions we should support.

I totally understand your issue and already run into it in the past. Therefore, I am okay with dropping support for older Opensearch versions <= 2.18.

I would suggest opening a PR and merging this, once we made the v4 release of the main branch.

[sean-]

Sorry, crossed wires from $JOB's support policy with the project's. You're right that 2.19.x is the line still being patched, not 2.18.

Rather than re-pin to a different fixed number that will go stale the same way, I'd like to push the question one step further: is there any reason not to always trim the supported set to whatever OpenSearch is currently patching at the time opensearch-go cuts a release?

Spinning up ~70 CI workers has been an interesting deep dive into OpenSearch's historical bugs and instability problems, but it isn't particularly useful to test every minor version once we get in the habit of tagging the spec and generated code with x-version-added / x-version-deprecated annotations per release. The annotations document the historical surface; CI tests the surface we currently support.

Concretely: trim the CI matrix to whatever is currently receiving patches at release time. Today that means 2.19.x and 3.x; next release we re-evaluate. Users on lines OpenSearch no longer patches stay on v4, which is unchanged. The proposal above has been updated.

[Jakob3xD]

is there any reason not to always trim the supported set to whatever OpenSearch is currently patching at the time opensearch-go cuts a release?

Yes, coming from Operational perspective, there are many use cases of how people use Opensearch and what extra tools they use. So there are users out there that can not update there opensearch clusters in a reasonable time due to other tools blocking them.
Therefore, we should always test against all releases of the current major and the latest release of the last major.
So every 3.x.x and every 2.19.x release.

I understand your perspective of trying to keeping things lean to also reduce the maintenance coast but as already argued above, this would not be a good solution.

[sean-]

I wear an operational hat, too, which is why I updated the above to have a 6mo lookback (i.e. test everything that has been released in the last 6mo). Would that cover your needs, or does it need to be longer? I'm trying to arrive at some form of a "policy" that we can just apply to keep things current and moving, vs subjective and eventually we accumulates extra versions.

[Jakob3xD]

In theory, Opensearch tries to release every 8 weeks. Looking back at all the 2.x releases. 2.x was used for 3 years. The thing is, that the 2.19x releases are scheduled like every 15 weeks, which is also included but what if they postpone it or skip the release? Therefore, the rule should include, that it tests against the latest release of the last major, independent of the time it was released.
Is there a specific reason why you chose 6mo and not 12mo? With only 6mo you include only the last 3-4 releases of the current major? With 12mo it would be the last 6-7 releases.

[sean-]

I just pulled 6mo out of thin air, but it's inline with semi-regular patching schedules from organizations (security issues not withstanding). If we want to use 12mo, I'm fine with that (updated the issue to reflect that change). That would include:

• 3.1.0 - 3.6.0
• 2.19.3 - 2.19.5

If we bump to 18mo, it would also include:

• >2.19.0
• >1.3.20

which... hrm. If we waited until June 9th to have this conversation, wouldn't include 1.3.20. 😆 But, point taken. 12mo? I agree that 6mo was too aggressive.

[Jakob3xD]

Agreed to 12mo.

[sean-]

Tagging issue to #835 for completion as soon as the v4 branch is cut and main can continue with v5 related work.