You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
OpenSearch returns HTTP 200 for partial successes -- bulk item failures,
search-shard failures, single-doc replica failures -- so callers must
remember a second check after `err == nil`. v4 added an opt-in boolean
(Config.ReturnQueryErrors) that converted ALL partial-failure shapes
into typed Go errors. That single switch is too coarse: callers who
want shard-level errors but tolerate bulk item failures (or vice versa)
have no way to express it.
Replace the boolean with internal/errmask.ErrorMask, a 15-bit field
where each bit corresponds to one wrapper schema in the proposed
x-error-responses OpenAPI extension (BulkItems, SearchShards,
WriteShards, BroadcastShards, NodeFailures, BulkByScrollFailures,
TaskFailures, MultiSearchItems, MultiDocItems, Snapshot{Create,Get}-
ShardFailures, SimulateDocFailures, RankEvalFailures,
IngestionShardFailures, PitNodeFailures). A set bit MASKS that
category; the zero value reports every category. Callers express
fine-grained policy in code (Config.Errors = errmask.BulkItems |
errmask.SearchShards) or via OPENSEARCH_GO_ERROR_MASK using
comma-separated +/- tokens (e.g. "+all,-bulk_items").
Lifecycle (matches OPENSEARCH_GO_ROUTER):
v4 (this commit): default `errmask.All` -- preserves pre-bitfield
behavior (no partial-failure errors). Config.ReturnQueryErrors=true
is honored as a deprecated alias for `errmask.None`.
v5: default flips to `errmask.None` (safe by default).
v6: Config.Errors / OPENSEARCH_GO_ERROR_MASK removed; behavior is
unconditionally `errmask.None`.
The hand-written v4 opensearchapi/api_*.go call sites now read
c.errors.Has(errmask.<Wrapper>) for each operation's wrapper category.
A new hand-written v5preview/opensearchapi/errors.go ports the same
typed-error surface (PartialBulkError, PartialSearchError,
ShardFailureError, plus the IsPartialFailure / ToleratePartial-
Failures / RequireSuccessRate helpers) using v5preview's BulkResponse-
Item and ShardSearchFailure types. v5preview Config.Errors and the
clientInit(rootClient, mask) signature are wired through both
hand-written api.go and the generated clients_gen.go.
Spec side: opensearch-openapi.yaml is patched with 15
`_common.errors___<Wrapper>` schemas under components.schemas and 115
operation entries get an x-error-responses annotation. This mirrors
the upstream proposal in opensearch-api-specification (see
issue-x-partial-failure-mode.md). Once that PR lands and we re-bundle
from source, the local patch goes away cleanly.
Generator side: cmd/osgen reads x-error-responses from the spec
extension into ir.Operation.ErrorWrappers; cmd/osgen/errwrap supplies
a hardcoded fallback for plugin operations the spec doesn't yet
annotate. The dispatch fragment carries a data-driven `wrappers` map
of {Template, Applies}: each wrapper has both an emission template and
an Applies predicate that walks the response struct (including
embeds via the type registry) to confirm the field path the template
references actually exists. This keeps generated code compilable when
spec annotations land before the underlying response schema models the
relevant field -- v5preview's CreateResp and msearch's union response
item are skipped today and will start emitting once those types
acquire the missing fields.
Ref: opensearch-project#816
Ref: opensearch-project/opensearch-api-specification/pull/1137
Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
- Add partial failure error types (`PartialBulkError`, `PartialSearchError`, `ShardFailureError`) that surface HTTP 200 partial failures as Go errors when `Config.ReturnQueryErrors` is enabled ([#816](https://github.com/opensearch-project/opensearch-go/issues/816))
92
+
- Add partial failure error types (`PartialBulkError`, `PartialSearchError`, `ShardFailureError`, `MultiSearchItemError`) that surface HTTP 200 partial failures as typed Go errors, controlled by a per-category `errmask.ErrorMask` bitfield on `Config.Errors` ([#816](https://github.com/opensearch-project/opensearch-go/issues/816))
93
93
-`PartialBulkError` returned from `Bulk` when `resp.Errors` is true, carries `FailedItems` and `SucceededCount`
94
94
-`PartialSearchError` returned from `Search`, `MSearch`, `MSearchTemplate`, `SearchTemplate`, `Scroll.Get` when `_shards.failed > 0`
95
95
-`ShardFailureError` returned from `Index`, `Document.Create`, `Document.Delete`, `Update` when replica shards fail
96
-
-`PartialFailureError` marker interface with `IsPartial() bool` for type-switching across all partial failure types
96
+
-`MultiSearchItemError` returned from `MSearch`/`MSearchTemplate` for per-sub-response Error envelopes
97
+
-`MsearchErrors` / `MsearchTemplateErrors` per-op containers (Go 1.20+ multi-error contract via `Unwrap() []error`) when 2+ wrapper categories fire on the same response
98
+
-`PartialFailureError` marker interface with `IsPartial() bool` for type-switching across all partial-failure types
99
+
- Per-Resp helper methods (`BulkItemFailures`, `SearchShardFailures`, `WriteShardFailures`, `MultiSearchItemFailures`) plus `PartialFailures(mask)` aggregator for focused inspection at the call site
100
+
-`opensearchapi.Errors(err) []error` package-level helper that flattens single- and multi-wrapper errors into a uniform slice for `switch` dispatch
97
101
- Helper functions: `IsPartialFailure`, `ToleratePartialFailures`, `RequireSuccessRate` for threshold-based error tolerance
-`Config.ReturnQueryErrors`defaults to `false` in v4 (opt-in), will flip to `true` in v5
100
-
-`OPENSEARCH_GO_PARTIAL_QUERY_ERRORS` environment variable overrides `Config.ReturnQueryErrors` at runtime
103
+
-`Config.Errors *errmask.ErrorMask` replaces a single boolean: each bit suppresses one wrapper category. v4 defaults to `errmask.All` (mask everything, preserves pre-bitfield behavior); v5+ defaults to `errmask.Empty` (report everything)
104
+
-`OPENSEARCH_GO_ERROR_MASK` environment variable overrides `Config.Errors` at runtime via comma-separated `+`/`-` tokens (lowercase snake_case wrapper names; unknown tokens silently dropped, debug-logged)
101
105
- Both `(resp, error)` are non-nil on partial failure -- response is fully populated
106
+
-`v5preview/opensearchapi` ports the same model with spec-driven types (regenerated from the OpenAPI `x-error-responses` extension on every `cmd/osgen` run)
102
107
- Add `OperationClassifier` for zero-allocation HTTP method+path to `OperationID` mapping ([#816](https://github.com/opensearch-project/opensearch-go/issues/816))
103
108
- Bit-packed `OperationID` (int64) encoding R/W flag, category, and minor operation
-[Response.Body becomes a method](#responsebody-becomes-a-method)
4
4
-[StringError for unknown JSON responses](#stringerror-for-unknown-json-responses)
5
5
-[Upgrading to >= 4.7.0](#upgrading-to->=-4.7.0)
@@ -29,52 +29,148 @@
29
29
30
30
## Upgrading to >= 5.0.0
31
31
32
-
### Partial Failure Errors (ReturnQueryErrors)
32
+
### Partial Failure Errors (Config.Errors)
33
33
34
-
Version 5.0.0 changes `Config.ReturnQueryErrors` to default to `true`. When enabled, API methods return typed errorsfor partial failures (HTTP 200 responses with embedded errors), so callers only need the standard `if err != nil` pattern. Both the response and the error are non-nil on partial failure -- the response is fully populated.
34
+
Version 5.0.0 introduces typed partial-failure errors and a per-category bitmask that controls which categories surface as Go errors. OpenSearch returns HTTP 200 for many operations that partially succeed (bulk item failures, shard failures on search, replica failures on writes), so callers historically had to remember a second response inspection after every `if err != nil { ... }`. The new model turns those partial failures into typed errors callers can match on with `errors.As`.
35
35
36
-
**Why**: OpenSearch returns HTTP 200 for operations that partially succeed (bulk item failures, shard failures on search, replica failures on writes). Before this change, callers had to remember a second error check after every operation, which is easy to forget and non-idiomatic Go.
36
+
#### Configuring the mask
37
37
38
-
**What changes in v5**: If you relied on `err == nil` meaning "no failures of any kind", your code is already correct -- you were ignoring partial failures. With `ReturnQueryErrors: true` (now the default), those partial failures surface as errors. Your `if err != nil` blocks will now catch them.
38
+
`Config.Errors`is a `*errmask.ErrorMask` pointer. A set bit suppresses (masks) that category; an unset bit reports it. Three named values cover the common cases:
39
39
40
-
**If you need the old behavior**, set `ReturnQueryErrors: false`:
|`nil`| Use the version's default (v4: `errmask.All`; v5+: `errmask.Empty`) |
43
+
|`&errmask.Empty`| Mask nothing -- every category is reported as a typed error |
44
+
|`&errmask.All`| Mask everything -- callers must inspect the response manually |
45
+
46
+
`errmask.None` and `errmask.Unknown` are aliases for `errmask.Empty`; all three equal 0. Composite masks (e.g. `errmask.SearchShards | errmask.MultiSearchItems`) suppress specific categories while leaving others reported.
47
+
48
+
The v4 default (`errmask.All`) preserves pre-bitfield behavior: partial failures are not surfaced as Go errors, so existing v4 code continues to work without modification. Opt in by setting `Config.Errors: &errmask.Empty` (or use `errmask.NewClient`-style helpers). The v5+ default flips to `errmask.Empty` so partial failures surface by default.
`OPENSEARCH_GO_ERROR_MASK` accepts a comma-separated list of `+`/`-` tokens applied left-to-right on top of `Config.Errors`. Tokens are the lowercase snake_case wrapper-schema names from the OpenAPI `x-error-responses` extension (`bulk_items`, `search_shards`, `write_shards`, ...).
61
+
62
+
```sh
63
+
# Mask everything except bulk-item errors (useful with v4: opt out of "mask everything" but suppress search-shard noise)
# Only mask search-shard failures; report every other category
67
+
export OPENSEARCH_GO_ERROR_MASK="search_shards"
68
+
69
+
# Reset to "mask everything" (mimics the v4 default)
70
+
export OPENSEARCH_GO_ERROR_MASK="all"
71
+
72
+
# Reset to "report everything" (the v5+ default)
73
+
export OPENSEARCH_GO_ERROR_MASK="none"
74
+
```
75
+
76
+
Unknown tokens are ignored (forward compatible: an older client tolerates new wrapper bits added by a newer release) and reported via the debug logger when `OPENSEARCH_GO_DEBUG` is enabled.
77
+
78
+
#### Handling typed errors
50
79
51
-
All partial failure errors implement the `PartialFailureError` interface and work with `errors.As`:
80
+
Each operation returns a typed sub-error per detected wrapper category, and operations declaring multiple `x-error-responses` entries can fire more than one. The dispatch handler applies a runtime-collapse rule:
81
+
82
+
- 0 sub-errors fired: returns `nil`.
83
+
- 1 sub-error fired: returns the bare sub-error (no wrapper allocated).
84
+
- 2+ sub-errors fired: returns the per-op error type wrapping the slice.
85
+
86
+
`errors.As` against a known sub-error type works in **both** the single and multi cases (the per-op type implements `Unwrap() []error`):
// resp is fully populated -- inspect individual items
60
-
log.Printf("%d/%d items failed",
61
-
len(bulkErr.FailedItems),
62
-
bulkErr.SucceededCount+len(bulkErr.FailedItems))
90
+
varbulkErr *opensearchapi.PartialBulkError
91
+
if errors.As(err, &bulkErr) {
92
+
log.Printf("%d/%d items failed",
93
+
len(bulkErr.FailedItems),
94
+
bulkErr.SucceededCount+len(bulkErr.FailedItems))
95
+
}
96
+
```
97
+
98
+
Callers wanting to enumerate every sub-error from a multi-wrapper op match on the per-op type:
99
+
100
+
```go
101
+
resp, err:= client.MSearch(ctx, req)
102
+
varmsErr *opensearchapi.MsearchErrors
103
+
if errors.As(err, &msErr) {
104
+
for_, sub:=range msErr.Unwrap() {
105
+
switche:= sub.(type) {
106
+
case *opensearchapi.PartialSearchError: // shard aggregation
107
+
case *opensearchapi.MultiSearchItemError: // per-sub-response Error
108
+
}
109
+
}
110
+
}
111
+
```
112
+
113
+
The `opensearchapi.Errors(err) []error` helper flattens the same shape uniformly across single- and multi-wrapper ops, so a single `switch` block handles both:
114
+
115
+
```go
116
+
resp, err:= client.MSearch(ctx, req)
117
+
for_, sub:=range opensearchapi.Errors(err) {
118
+
switche:= sub.(type) {
119
+
case *opensearchapi.PartialSearchError:
120
+
// shard aggregation
121
+
case *opensearchapi.MultiSearchItemError:
122
+
// per-sub-response Error envelope
63
123
default:
64
-
return err // transport or HTTP error
124
+
// transport / HTTP / decoding error
65
125
}
66
126
}
67
127
```
68
128
69
-
**Error types**:
129
+
A `nil``err` returns `nil`; a non-partial err (transport, HTTP, decode) returns a single-element slice containing `err`. Adding new wrapper categories later is purely additive: a new `case` picks it up; the `default` keeps catching everything else.
130
+
131
+
#### Per-Resp helper methods
132
+
133
+
Every operation declaring `x-error-responses` exposes per-wrapper helper methods on its typed response, plus a `PartialFailures(mask)` aggregator. Use these when you want focused inspection at the call site without going through the dispatch error:
134
+
135
+
```go
136
+
resp, _:= client.Bulk(ctx, req)
137
+
ife:= resp.BulkItemFailures(); e != nil {
138
+
log.Printf("%d items failed", len(e.FailedItems))
139
+
}
140
+
141
+
resp2, _:= client.MSearch(ctx, req)
142
+
ife:= resp2.SearchShardFailures(); e != nil { /* ... */ }
143
+
ife:= resp2.MultiSearchItemFailures(); e != nil { /* ... */ }
144
+
```
145
+
146
+
The `r.PartialFailures(mask errmask.ErrorMask) []error` aggregator reports every wrapper category not suppressed by `mask` -- useful when reusing the dispatch's mask gating outside the dispatch path.
|`*PartialSearchError`|`Search`, `Scroll.Get`, `SearchTemplate` (single-bit); also via `*MsearchErrors` and `*MsearchTemplateErrors` for shard aggregation |`FailedShards int`, `TotalShards int`, `Failures []ResponseShardsFailure`|
|`*MsearchErrors`|`MSearch` when 2+ wrappers fire |`Unwrap() []error` (multi-error contract) |
157
+
|`*MsearchTemplateErrors`|`MSearchTemplate` when 2+ wrappers fire |`Unwrap() []error`|
158
+
159
+
Per-op `*<Op>Errors` types are the Go 1.20+ multi-error containers; they implement `Unwrap() []error` so `errors.As` against any sub-error type still matches whether the response carried one sub-error or many.
160
+
161
+
#### Error types in v5preview `v5preview/opensearchapi/` (preview)
162
+
163
+
The v5preview surface ports the same model, but its error sub-types are spec-driven (regenerated from the OpenAPI spec on every `cmd/osgen` run):
164
+
165
+
| Field name | v4 (`opensearchapi`) | v5preview (`v5preview/opensearchapi`) |
The v5preview package additionally generates one per-op error type per operation declaring `x-error-responses` (e.g. `*v5preview/opensearchapi.MsearchErrors`, `*v5preview/opensearchapi.MsearchTemplateErrors`). Callers wanting v4-shaped field types should keep using the v4 `opensearchapi` package; v5preview is a preview surface that will become the default in v5.
76
172
77
-
**Helper functions** for common patterns:
173
+
#### Helper functions
78
174
79
175
```go
80
176
// Suppress all partial failures (best-effort operations)
0 commit comments