Skip to content

Commit 0df5b3b

Browse files
committed
docs: address PR opensearch-project#812 review feedback
- guides/metrics.md: replace nonexistent opensearchtransport.WithObserver() with the Observer field on opensearch.Config - guides/metrics.md: hoist safeFloat above the for loop so the polling example is parseable Go - guides/metrics.md: align aggregate tables (Request Counters, Pool State, Lifecycle, Health Checks) to Field (Go name) + JSON (tag) layout - guides/metrics.md: soften mechanism claims on IsOverloaded, RTTBucket, EstLoad to describe field meaning rather than how they get set - guides/error_handling.md, v5preview/opensearchapi/README.md: drop *PartialBulkError and *ShardFailureError from MSearch switch examples; fix the duplicate pre-existing example in the v5preview README - guides/error_handling.md: qualify "folded into err" with the v4-vs- v5preview default-mask caveat - guides/error_handling.md: nil-check ErrorCause.Reason deref in v5preview bulk example; switch the version-agnostic MSearch helper to log item.Status (portable across v4 and v5preview) - v5preview/opensearchapi/README.md: fix NewMuxRouter() to single-value assignment (returns Router, not (Router, error)) - v5preview/opensearchapi/MIGRATING.md: drop the forward-compatible `replace` section -- the cited path has no go.mod and the directive cannot resolve - UPGRADING.md: reword the partial-failure intro so the recommended type-switch pattern is the only advertised entry point (drops the in-tension `errors.As` mention) - CHANGELOG.md: add (opensearch-project#812) backlink to the metrics-guide bullet Signed-off-by: Sean Chittenden <sean.chittenden@crowdstrike.com>
1 parent 6d7a98a commit 0df5b3b

6 files changed

Lines changed: 63 additions & 85 deletions

File tree

CHANGELOG.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
1616
- Add `OPENSEARCH_GO_SHARD_COST` environment variable and `WithShardCosts()` router option with `r:base`/`r:amplify`/`r:exponent` curve keys and static cost overrides
1717
- Add `ShardCostConfig` field to `Config` struct for programmatic shard cost override passthrough
1818
- Add `OPENSEARCH_GO_ROUTER` environment variable to enable the DefaultRouter without code changes; set to `true` to opt in (off by default in v4, on by default in v5, removed in v6) ([#815](https://github.com/opensearch-project/opensearch-go/pull/815))
19-
- Add client-side metrics guide covering Metrics API, ConnectionMetric, PolicySnapshot, and RouterSnapshot
19+
- Add client-side metrics guide covering Metrics API, ConnectionMetric, PolicySnapshot, and RouterSnapshot ([#812](https://github.com/opensearch-project/opensearch-go/pull/812))
2020
- Add `InsecureSkipVerify` config option to disable TLS certificate verification without constructing a custom `http.Transport`, preserving `DefaultTransport` connection pooling, HTTP/2, and timeout defaults ([#786](https://github.com/opensearch-project/opensearch-go/issues/786))
2121
- Add `DisableResponseBuffering` config option to skip eager `io.ReadAll` buffering of response bodies in `Perform()`, reducing per-request allocations and TTFB for proxy and streaming use cases ([#786](https://github.com/opensearch-project/opensearch-go/issues/786))
2222
- Add per-attempt `RequestTimeout` to bound individual HTTP round-trips, preventing indefinite hangs on stalled connections ([#786](https://github.com/opensearch-project/opensearch-go/issues/786))
@@ -102,7 +102,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
102102
- `opensearchapi.Errors(err) []error` package-level helper that flattens single- and multi-wrapper errors into a uniform slice; recommended call-site pattern is a `for`/`switch` over the result (not `errors.As` against a specific type)
103103
- Helper functions: `IsPartialFailure`, `ToleratePartialFailures`, `RequireSuccessRate` for threshold-based error tolerance
104104
- Operation constants: `OperationIndex`, `OperationCreate`, `OperationUpdate`, `OperationDelete`
105-
- Per-Resp helper methods (`BulkItemFailures`, `SearchShardFailures`, `WriteShardFailures`, `MultiSearchItemFailures`, `PartialFailures(mask)`) exist on the response types as engine machinery for the dispatch; new code should prefer the `for`/`switch` pattern over `opensearchapi.Errors(err)` for forward compatibility
105+
- Per-Resp helper methods (`BulkItemFailures`, `SearchShardFailures`, `WriteShardFailures`, `MultiSearchItemFailures`, `PartialFailures(mask)`) exist on the response types as engine machinery for the dispatch; new code should prefer a `for`/`switch` over `opensearchapi.Errors(err)` rather than the per-Resp helpers, for forward compatibility
106106
- `Config.Errors *errmask.ErrorMask` replaces a single boolean: each bit suppresses one wrapper category. v4 defaults to `errmask.All` (mask everything, preserves pre-bitfield behavior); v5+ defaults to `errmask.Empty` (report everything)
107107
- `OPENSEARCH_GO_ERROR_MASK` environment variable overrides `Config.Errors` at runtime via comma-separated `+`/`-` tokens (lowercase snake_case wrapper names; unknown tokens silently dropped, debug-logged)
108108
- Both `(resp, error)` are non-nil on partial failure -- response is fully populated

UPGRADING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434

3535
### Partial Failure Errors (Config.Errors)
3636

37-
Version 5.0.0 introduces typed partial-failure errors and a per-category bitmask that controls which categories surface as Go errors. OpenSearch returns HTTP 200 for many operations that partially succeed (bulk item failures, shard failures on search, replica failures on writes). The new model turns those partial failures into typed errors callers can match on with `errors.As`.
37+
Version 5.0.0 introduces typed partial-failure errors and a per-category bitmask that controls which categories surface as Go errors. OpenSearch returns HTTP 200 for many operations that partially succeed (bulk item failures, shard failures on search, replica failures on writes). The new model turns those partial failures into typed errors that callers can dispatch on; idiomatic partial error handling is shown below.
3838

3939
**Default behavior change:**
4040

guides/error_handling.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -86,14 +86,18 @@ for _, sub := range opensearchapi.Errors(err) {
8686
len(e.FailedItems),
8787
e.SucceededCount+len(e.FailedItems))
8888
for _, item := range e.FailedItems {
89-
// BulkRespItem.ID and BulkRespItem.Error are pointers in v5preview.
89+
// BulkRespItem.ID, BulkRespItem.Error, and ErrorCause.Reason are pointers in v5preview.
9090
id := ""
9191
if item.ID != nil {
9292
id = *item.ID
9393
}
9494
if item.Error != nil {
95+
reason := ""
96+
if item.Error.Reason != nil {
97+
reason = *item.Error.Reason
98+
}
9599
log.Printf(" %s %s/%s: %s",
96-
item.Error.Type, item.Index, id, item.Error.Reason)
100+
item.Error.Type, item.Index, id, reason)
97101
}
98102
}
99103
default:
@@ -219,7 +223,9 @@ resp, err := client.Bulk(ctx, req)
219223
if err != nil {
220224
return err
221225
}
222-
// resp is fully populated; partial failures (if any) are folded into err.
226+
// resp is fully populated; partial failures (if any) are folded into err
227+
// when the wrapper bits are unmasked (the v5preview default, or v4 with
228+
// Config.Errors: errmask.New()).
223229
```
224230

225231
**Inspect categories with a `for`/`switch`** -- when partial error handling is appropriate. Partial error handling lets the client and its application recover from known failure modes they can tolerate (e.g. continue serving a search with a few failed shards, or retry only the bulk items the server rejected) instead of failing the whole operation. The `default` arm catches transport / HTTP / decode errors and any partial-failure category added in a future release:
@@ -228,14 +234,10 @@ if err != nil {
228234
resp, err := client.MSearch(ctx, req)
229235
for _, sub := range opensearchapi.Errors(err) {
230236
switch e := sub.(type) {
231-
case *opensearchapi.PartialBulkError:
232-
log.Printf("%d items failed", len(e.FailedItems))
233237
case *opensearchapi.PartialSearchError:
234238
log.Printf("%d/%d shards failed", e.FailedShards, e.TotalShards)
235239
case *opensearchapi.MultiSearchItemError:
236240
log.Printf("%d sub-queries failed", len(e.Items))
237-
case *opensearchapi.ShardFailureError:
238-
log.Printf("%s: %d/%d shards failed", e.Operation, e.FailedShards, e.TotalShards)
239241
default:
240242
return err
241243
}
@@ -289,7 +291,7 @@ func handleMSearchError(err error) {
289291
metrics.ShardFailures.Add(int64(e.FailedShards))
290292
case *opensearchapi.MultiSearchItemError:
291293
for _, item := range e.Items {
292-
log.Printf("sub-query %d failed: %s", item.Index, item.Error.Reason)
294+
log.Printf("sub-query %d failed (status=%d)", item.Index, item.Status)
293295
}
294296
default:
295297
log.Printf("non-partial msearch error: %v", e)

guides/metrics.md

Lines changed: 49 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -34,39 +34,39 @@ The top-level `Metrics` struct contains aggregate counters, per-connection detai
3434

3535
### Request Counters
3636

37-
| Field | Type | Description |
38-
| ----------- | ------------- | ----------------------------------------- |
39-
| `requests` | `int` | Total requests performed by the transport |
40-
| `failures` | `int` | Total request failures |
41-
| `responses` | `map[int]int` | Response count by HTTP status code |
37+
| Field | JSON | Type | Description |
38+
| ----------- | ----------- | ------------- | ----------------------------------------- |
39+
| `Requests` | `requests` | `int` | Total requests performed by the transport |
40+
| `Failures` | `failures` | `int` | Total request failures |
41+
| `Responses` | `responses` | `map[int]int` | Response count by HTTP status code |
4242

4343
### Connection Pool State
4444

45-
| Field | Type | Description |
46-
| --------------------- | ----- | ------------------------------------------- |
47-
| `live_connections` | `int` | Non-dead connections (active + standby) |
48-
| `dead_connections` | `int` | Connections in the dead list |
49-
| `standby_connections` | `int` | Connections in the standby partition |
50-
| `overloaded_servers` | `int` | Connections the client considers overloaded |
45+
| Field | JSON | Type | Description |
46+
| -------------------- | --------------------- | ----- | ------------------------------------------- |
47+
| `LiveConnections` | `live_connections` | `int` | Non-dead connections (active + standby) |
48+
| `DeadConnections` | `dead_connections` | `int` | Connections in the dead list |
49+
| `StandbyConnections` | `standby_connections` | `int` | Connections in the standby partition |
50+
| `OverloadedServers` | `overloaded_servers` | `int` | Connections the client considers overloaded |
5151

5252
### Connection Lifecycle Counters
5353

54-
| Field | Type | Description |
55-
| ---------------------- | ----- | ---------------------------------------------------- |
56-
| `connections_promoted` | `int` | Dead to ready transitions (successful resurrections) |
57-
| `connections_demoted` | `int` | Ready to dead transitions |
58-
| `zombie_connections` | `int` | Dead connections forcibly retried |
59-
| `standby_promotions` | `int` | Standby to active transitions |
60-
| `standby_demotions` | `int` | Active to standby transitions |
54+
| Field | JSON | Type | Description |
55+
| --------------------- | ---------------------- | ----- | ---------------------------------------------------- |
56+
| `ConnectionsPromoted` | `connections_promoted` | `int` | Dead to ready transitions (successful resurrections) |
57+
| `ConnectionsDemoted` | `connections_demoted` | `int` | Ready to dead transitions |
58+
| `ZombieConnections` | `zombie_connections` | `int` | Dead connections forcibly retried |
59+
| `StandbyPromotions` | `standby_promotions` | `int` | Standby to active transitions |
60+
| `StandbyDemotions` | `standby_demotions` | `int` | Active to standby transitions |
6161

6262
### Health Check Counters
6363

64-
| Field | Type | Description |
65-
| ----------------------- | ----- | -------------------------------------------------- |
66-
| `health_checks` | `int` | Baseline `GET /` health checks performed |
67-
| `cluster_health_checks` | `int` | `GET /_cluster/health?local=true` checks performed |
68-
| `health_checks_success` | `int` | Successful health check outcomes |
69-
| `health_checks_failed` | `int` | Failed health check outcomes |
64+
| Field | JSON | Type | Description |
65+
| --------------------- | ----------------------- | ----- | -------------------------------------------------- |
66+
| `HealthChecks` | `health_checks` | `int` | Baseline `GET /` health checks performed |
67+
| `ClusterHealthChecks` | `cluster_health_checks` | `int` | `GET /_cluster/health?local=true` checks performed |
68+
| `HealthChecksSuccess` | `health_checks_success` | `int` | Successful health check outcomes |
69+
| `HealthChecksFailed` | `health_checks_failed` | `int` | Failed health check outcomes |
7070

7171
---
7272

@@ -76,30 +76,30 @@ Each connection produces a `ConnectionMetric` in `Metrics.Connections`. Connecti
7676

7777
### Core Fields
7878

79-
| Field | JSON | Type | Description |
80-
| ------------------ | ------------------ | ------------ | ----------------------------------- |
81-
| `URL` | `url` | `string` | Node URL |
82-
| `Failures` | `failures` | `int` | Failure count (omitted when zero) |
83-
| `IsDead` | `dead` | `bool` | In the dead list |
84-
| `IsStandby` | `standby` | `bool` | In the standby partition |
85-
| `IsOverloaded` | `overloaded` | `bool` | Marked overloaded by stats poller |
86-
| `IsWarmingUp` | `warming_up` | `bool` | In warmup phase after promotion |
87-
| `IsHealthChecking` | `health_checking` | `bool` | Currently being health-checked |
88-
| `NeedsCatUpdate` | `needs_cat_update` | `bool` | Shard placement data is stale |
89-
| `Weight` | `weight` | `int` | Effective weight for selection |
90-
| `DeadSince` | `dead_since` | `*time.Time` | When the connection was marked dead |
91-
| `OverloadedSince` | `overloaded_since` | `*time.Time` | When overload was detected |
92-
| `State` | `state` | `ConnState` | Packed connection state word |
79+
| Field | JSON | Type | Description |
80+
| ------------------ | ------------------ | ------------ | ------------------------------------------------------ |
81+
| `URL` | `url` | `string` | Node URL |
82+
| `Failures` | `failures` | `int` | Failure count (omitted when zero) |
83+
| `IsDead` | `dead` | `bool` | In the dead list |
84+
| `IsStandby` | `standby` | `bool` | In the standby partition |
85+
| `IsOverloaded` | `overloaded` | `bool` | Whether the connection is currently flagged overloaded |
86+
| `IsWarmingUp` | `warming_up` | `bool` | In warmup phase after promotion |
87+
| `IsHealthChecking` | `health_checking` | `bool` | Currently being health-checked |
88+
| `NeedsCatUpdate` | `needs_cat_update` | `bool` | Shard placement data is stale |
89+
| `Weight` | `weight` | `int` | Effective weight for selection |
90+
| `DeadSince` | `dead_since` | `*time.Time` | When the connection was marked dead |
91+
| `OverloadedSince` | `overloaded_since` | `*time.Time` | When overload was detected |
92+
| `State` | `state` | `ConnState` | Packed connection state word |
9393

9494
### Router Fields
9595

9696
Populated when request routing is active and the connection has observed traffic.
9797

9898
| Field | JSON | Type | Description |
9999
| ----------- | ------------ | ---------- | ------------------------------------------------------------------ |
100-
| `RTTBucket` | `rtt_bucket` | `*int64` | Quantized RTT tier (lower is closer) |
101-
| `RTTMedian` | `rtt_median` | `*string` | Median RTT as a human-readable duration |
102-
| `EstLoad` | `est_load` | `*float64` | Estimated load: `inFlight / cwnd` |
100+
| `RTTBucket` | `rtt_bucket` | `*int64` | Quantized RTT tier (smaller values indicate lower observed RTT) |
101+
| `RTTMedian` | `rtt_median` | `*string` | Median observed RTT as a human-readable duration |
102+
| `EstLoad` | `est_load` | `*float64` | Estimated per-connection load |
103103
| `MCSR` | `mcsr` | `*int` | Adaptive `max_concurrent_shard_requests` value (nil when disabled) |
104104

105105
### Node Metadata
@@ -169,6 +169,13 @@ Each index with an active routing slot produces an entry in `Router.Indexes`.
169169
Poll metrics on a timer for logging or export to an external monitoring system.
170170

171171
```go
172+
func safeFloat(f *float64) float64 {
173+
if f == nil {
174+
return 0
175+
}
176+
return *f
177+
}
178+
172179
ticker := time.NewTicker(30 * time.Second)
173180
defer ticker.Stop()
174181

@@ -207,13 +214,6 @@ for range ticker.C {
207214
}
208215
}
209216
}
210-
211-
func safeFloat(f *float64) float64 {
212-
if f == nil {
213-
return 0
214-
}
215-
return *f
216-
}
217217
```
218218

219219
## JSON Export
@@ -249,6 +249,6 @@ The metrics API is pull-based: call `client.Metrics()` inside your collector's `
249249

250250
## Observer API
251251

252-
For event-driven observability (as opposed to polling), implement the `ConnectionObserver` interface and pass it via `opensearchtransport.WithObserver()`. The observer receives callbacks for connection lifecycle events (promote, demote, overload), routing decisions, health checks, and shard map invalidations. See the [routing guide](routing.md) for details on observer events.
252+
For event-driven observability (as opposed to polling), implement the `opensearchtransport.ConnectionObserver` interface and set it on the `Observer` field of `opensearch.Config`. The observer receives callbacks for connection lifecycle events (promote, demote, overload), routing decisions, health checks, and shard map invalidations. See the [routing guide](routing.md) for details on observer events.
253253

254254
The metrics API and observer API are complementary: metrics give you aggregate snapshots for dashboards, while the observer gives you per-event detail for tracing and debugging.

v5preview/opensearchapi/MIGRATING.md

Lines changed: 0 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -26,22 +26,6 @@ import "github.com/opensearch-project/opensearch-go/v4/v5preview/opensearchapi"
2626

2727
The package qualifier (`opensearchapi.X`) does not change. When v5 ships, the only edit per file is dropping `/v4/v5preview` from the import path and replacing `v4` with `v5`.
2828

29-
### Forward-compatible `replace` directive (optional)
30-
31-
To write code today against the eventual v5 import path, add a `replace` to `go.mod`:
32-
33-
```
34-
replace github.com/opensearch-project/opensearch-go/v5/opensearchapi => github.com/opensearch-project/opensearch-go/v4/v5preview/opensearchapi v4.7.0
35-
```
36-
37-
Then import as if v5 already shipped:
38-
39-
```go
40-
import "github.com/opensearch-project/opensearch-go/v5/opensearchapi"
41-
```
42-
43-
When v5 ships, drop the `replace` line; nothing else changes.
44-
4529
## Field renames you'll hit
4630

4731
### `Indices` -> `Index` on multi-index Req types

v5preview/opensearchapi/README.md

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -238,10 +238,6 @@ for _, sub := range opensearchapi.Errors(err) {
238238
log.Printf("shard agg: %d/%d shards failed", e.FailedShards, e.TotalShards)
239239
case *opensearchapi.MultiSearchItemError:
240240
log.Printf("%d sub-queries failed", len(e.Items))
241-
case *opensearchapi.PartialBulkError:
242-
log.Printf("%d items failed", len(e.FailedItems))
243-
case *opensearchapi.ShardFailureError:
244-
log.Printf("%s: %d/%d shards failed", e.Operation, e.FailedShards, e.TotalShards)
245241
default:
246242
return err // transport / HTTP / decoding error
247243
}
@@ -284,14 +280,10 @@ if err != nil {
284280
resp, err := client.MSearch(ctx, req)
285281
for _, sub := range opensearchapi.Errors(err) {
286282
switch e := sub.(type) {
287-
case *opensearchapi.PartialBulkError:
288-
log.Printf("%d items failed", len(e.FailedItems))
289283
case *opensearchapi.PartialSearchError:
290284
log.Printf("%d/%d shards failed", e.FailedShards, e.TotalShards)
291285
case *opensearchapi.MultiSearchItemError:
292286
log.Printf("%d sub-queries failed", len(e.Items))
293-
case *opensearchapi.ShardFailureError:
294-
log.Printf("%s: %d/%d shards failed", e.Operation, e.FailedShards, e.TotalShards)
295287
default:
296288
return err
297289
}
@@ -351,7 +343,7 @@ client, _ := opensearchapi.NewClient(opensearchapi.Config{
351343
})
352344

353345
// Caller-provided Router is preserved.
354-
custom, _ := opensearchtransport.NewMuxRouter()
346+
custom := opensearchtransport.NewMuxRouter()
355347
client, _ = opensearchapi.NewClient(opensearchapi.Config{
356348
Client: opensearch.Config{Addresses: addrs, Router: custom},
357349
})

0 commit comments

Comments
 (0)