Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,9 +181,10 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
### Security

### Dependencies

- Bump golangci-lint from v2.11.2 to v2.11.4
- Bump `github.com/aws/aws-sdk-go-v2/config` from 1.32.6 to 1.32.7 ([#767](https://github.com/opensearch-project/opensearch-go/pull/767))
- Bump `golang.org/x/mod` from 0.33.0 to 0.35.0 ([#805](https://github.com/opensearch-project/opensearch-go/pull/805))
- Bump `github.com/aws/aws-sdk-go-v2/config` from 1.32.6 to 1.32.7 ([#767](https://github.com/opensearch-project/opensearch-go/pull/767))

## [4.6.0]

Expand Down
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
module github.com/opensearch-project/opensearch-go/v4

go 1.24.0
go 1.25.0

require (
github.com/aws/aws-sdk-go-v2 v1.41.1
github.com/aws/aws-sdk-go-v2/config v1.32.7
github.com/aws/aws-sdk-go-v2/credentials v1.19.7
github.com/stretchr/testify v1.11.1
github.com/wI2L/jsondiff v0.7.0
golang.org/x/mod v0.33.0
golang.org/x/mod v0.35.0
golang.org/x/sync v0.19.0
)

Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
github.com/wI2L/jsondiff v0.7.0 h1:1lH1G37GhBPqCfp/lrs91rf/2j3DktX6qYAKZkLuCQQ=
github.com/wI2L/jsondiff v0.7.0/go.mod h1:KAEIojdQq66oJiHhDyQez2x+sRit0vIzC9KeK0yizxM=
golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8=
golang.org/x/mod v0.33.0/go.mod h1:swjeQEj+6r7fODbD2cqrnje9PnziFuw4bmLbBZFrQ5w=
golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM=
golang.org/x/mod v0.35.0/go.mod h1:+GwiRhIInF8wPm+4AoT6L0FA1QWAad3OMdTRx4tFYlU=
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
Expand Down
52 changes: 26 additions & 26 deletions guides/cluster_health_checking.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,10 +80,10 @@ The capacity model values (`clientsPerServer`, `healthCheckRate`) are auto-deriv

### Configuration

| Config Field | Default | Description |
| ---------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `MaxRetryClusterHealth` | `4h` | Retry interval for re-probing unavailable nodes. `0` = use default, `<0` = disable probing entirely. |
| `HealthCheckRequestModifier` | `nil` | Callback applied to every health check request (both `GET /` and `/_cluster/health`). Use this to inject custom auth headers. |
| Config Field | Default | Description |
| --- | --- | --- |
| `MaxRetryClusterHealth` | `4h` | Retry interval for re-probing unavailable nodes. `0` = use default, `<0` = disable probing entirely. |
| `HealthCheckRequestModifier` | `nil` | Callback applied to every health check request (both `GET /` and `/_cluster/health`). Use this to inject custom auth headers. |

```go
client, err := opensearchapi.NewClient(opensearchapi.Config{
Expand Down Expand Up @@ -130,11 +130,11 @@ The response is a JSON object:

The `status` field reports overall cluster health:

| Status | Meaning |
| -------- | -------------------------------------------------------------------------------------------------------------- |
| `green` | All primary and replica shards are assigned. |
| Status | Meaning |
| --- | --- |
| `green` | All primary and replica shards are assigned. |
| `yellow` | All primary shards are assigned, but some replicas are not. The cluster is functional but not fully redundant. |
| `red` | Some primary shards are unassigned. Data loss or unavailability may be occurring. |
| `red` | Some primary shards are unassigned. Data loss or unavailability may be occurring. |

A single-node development cluster will always report `yellow` because there is no second node to host replica shards. This is expected and does not indicate a problem.

Expand All @@ -150,16 +150,16 @@ The client uses poll-and-parse instead: issue the request, read the `status` fie

## HTTP Response Status Codes

| HTTP Status | Meaning |
| ----------- | ----------------------------------------------------------------------------------- |
| **200** | Success. Parse the `status` and `timed_out` fields from the response body. |
| **400** | Malformed request (invalid query parameters). |
| **401** | Authentication failure: credentials are missing or invalid. |
| **403** | Authorization failure: the user is authenticated but lacks the required permission. |
| **408** | The request timed out. Only occurs when `wait_for_*` parameters are used. |
| **429** | The node's thread pool rejected the request (server-side backpressure). |
| **500** | Unexpected server error. |
| **503** | The node is not ready to accept requests (e.g., still starting up). |
| HTTP Status | Meaning |
| --- | --- |
| **200** | Success. Parse the `status` and `timed_out` fields from the response body. |
| **400** | Malformed request (invalid query parameters). |
| **401** | Authentication failure: credentials are missing or invalid. |
| **403** | Authorization failure: the user is authenticated but lacks the required permission. |
| **408** | The request timed out. Only occurs when `wait_for_*` parameters are used. |
| **429** | The node's thread pool rejected the request (server-side backpressure). |
| **500** | Unexpected server error. |
| **503** | The node is not ready to accept requests (e.g., still starting up). |

### Distinguishing Auth Errors from Cluster Failures

Expand Down Expand Up @@ -265,21 +265,21 @@ PUT /_plugins/_security/api/rolesmapping/health_check

Before creating a custom role, check whether an existing role already provides sufficient access:

| Role | Permissions | Notes |
| ------------------------------ | --------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| `cluster_monitor` | `cluster:monitor/*` | Covers health, stats, and all monitoring endpoints. Broader than necessary for health checks alone. |
| `opensearch_dashboards_server` | Includes cluster monitoring among other privileges. | Intended for the Dashboards service account. |
| Role | Permissions | Notes |
| --- | --- | --- |
| `cluster_monitor` | `cluster:monitor/*` | Covers health, stats, and all monitoring endpoints. Broader than necessary for health checks alone. |
| `opensearch_dashboards_server` | Includes cluster monitoring among other privileges. | Intended for the Dashboards service account. |

The minimal custom `health_check` role with only `cluster:monitor/health` follows the principle of least privilege.

## Transitioning from an Unsecured Cluster

When a cluster transitions from no security (Security plugin disabled) to security enabled, the behavior changes:

| Before security | After security |
| ----------------------------------------- | ---------------------------------------------------------------------------------------------- |
| All requests succeed with no credentials. | Requests without valid credentials return **401**. |
| No permission checks. | Requests with valid credentials but missing `cluster:monitor/health` privilege return **403**. |
| Before security | After security |
| --- | --- |
| All requests succeed with no credentials. | Requests without valid credentials return **401**. |
| No permission checks. | Requests with valid credentials but missing `cluster:monitor/health` privilege return **403**. |

If the client is performing health checks and the cluster enables security, health probes will begin returning 401. This is expected. The client should surface this condition clearly rather than reporting the cluster as unhealthy: the cluster is reachable, but credentials need to be configured.

Expand Down
40 changes: 20 additions & 20 deletions guides/error_handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@ This design maximizes availability but requires careful error checking in client

### Operations That Can Have Partial Failures

| Operation | HTTP Status | Partial Failure Indicator | Impact |
| ----------------------- | ----------- | ------------------------- | ----------------------------------------- |
| **Bulk** | 200 | `errors: true` | Data loss - some documents not indexed |
| **Search** | 200 | `_shards.failed > 0` | Incomplete results - missing data |
| **Index/Update/Delete** | 201/200 | `_shards.failed > 0` | Durability risk - no replica confirmation |
| **Refresh** | 200 | `_shards.failed > 0` | Incomplete refresh |
| **Cluster operations** | 200 | `_shards.failed > 0` | Incomplete stats/operations |
| Operation | HTTP Status | Partial Failure Indicator | Impact |
| --- | --- | --- | --- |
| **Bulk** | 200 | `errors: true` | Data loss - some documents not indexed |
| **Search** | 200 | `_shards.failed > 0` | Incomplete results - missing data |
| **Index/Update/Delete** | 201/200 | `_shards.failed > 0` | Durability risk - no replica confirmation |
| **Refresh** | 200 | `_shards.failed > 0` | Incomplete refresh |
| **Cluster operations** | 200 | `_shards.failed > 0` | Incomplete stats/operations |

## Checking for Partial Failures

Expand Down Expand Up @@ -341,23 +341,23 @@ func (m *OperationMetrics) Report() {

### Bulk Operation Errors

| Error Type | Description | Retryable? | Action |
| ----------------------------------- | ---------------------------------- | ---------- | ----------------------------------------------------------------------- |
| `mapper_parsing_exception` | Invalid document format | No | Fix document |
| `version_conflict_engine_exception` | Version conflict | Maybe | Retry with updated version |
| `document_missing_exception` | Document not found (update/delete) | No | Skip or create |
| `es_rejected_execution_exception` | Queue full | Yes | Retry with backoff |
| `circuit_breaking_exception` | Circuit breaker tripped | Yes | Retry with backoff |
| `timeout_exception` | Operation timeout | Yes | Retry; see [Bulk: Timeout Configuration](bulk.md#timeout-configuration) |
| Error Type | Description | Retryable? | Action |
| --- | --- | --- | --- |
| `mapper_parsing_exception` | Invalid document format | No | Fix document |
| `version_conflict_engine_exception` | Version conflict | Maybe | Retry with updated version |
| `document_missing_exception` | Document not found (update/delete) | No | Skip or create |
| `es_rejected_execution_exception` | Queue full | Yes | Retry with backoff |
| `circuit_breaking_exception` | Circuit breaker tripped | Yes | Retry with backoff |
| `timeout_exception` | Operation timeout | Yes | Retry; see [Bulk: Timeout Configuration](bulk.md#timeout-configuration) |

### Shard Failure Reasons

| Reason Type | Description | Action |
| ---------------------------------- | --------------------- | -------------------- |
| `shard_not_available_exception` | Shard not ready | Retry or wait |
| Reason Type | Description | Action |
| --- | --- | --- |
| `shard_not_available_exception` | Shard not ready | Retry or wait |
| `primary_missing_action_exception` | Primary shard missing | Check cluster health |
| `search_phase_execution_exception` | Search phase failed | Review query |
| `illegal_argument_exception` | Invalid parameters | Fix query |
| `search_phase_execution_exception` | Search phase failed | Review query |
| `illegal_argument_exception` | Invalid parameters | Fix query |

## Complete Example: Production-Ready Bulk Indexer

Expand Down
10 changes: 5 additions & 5 deletions guides/node_discovery_and_roles.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,11 @@ OpenSearch 3.0 introduced two significant role changes:

These are distinct roles serving different purposes:

| Role | Purpose | Data Source | Write Traffic |
| -------- | ------------------------- | -------------------------------- | ------------------------- |
| `search` | Dedicated search replicas | Replicated from primary shards | None (read-only replicas) |
| `warm` | Searchable snapshots | Snapshot storage (local/remote) | None (read-only) |
| `data` | General purpose | Local shards (primary + replica) | Yes (indexing + search) |
| Role | Purpose | Data Source | Write Traffic |
| --- | --- | --- | --- |
| `search` | Dedicated search replicas | Replicated from primary shards | None (read-only replicas) |
| `warm` | Searchable snapshots | Snapshot storage (local/remote) | None (read-only) |
| `data` | General purpose | Local shards (primary + replica) | Yes (indexing + search) |

## Cluster Manager Filtering

Expand Down
16 changes: 8 additions & 8 deletions guides/response_buffering.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,15 +54,15 @@ In both cases, always call `res.Body.Close()` when done.

## When to Use

| Scenario | Recommendation |
| -------------------------------------------------- | --------------------------------- |
| Standard API calls (CRUD, search, cluster ops) | Leave buffering enabled (default) |
| Reverse proxy forwarding large responses | Disable buffering |
| Streaming bulk responses to clients | Disable buffering |
| Scroll/PIT with large result sets piped downstream | Disable buffering |
| Scenario | Recommendation |
| --- | --- |
| Standard API calls (CRUD, search, cluster ops) | Leave buffering enabled (default) |
| Reverse proxy forwarding large responses | Disable buffering |
| Streaming bulk responses to clients | Disable buffering |
| Scroll/PIT with large result sets piped downstream | Disable buffering |

## Configuration Reference

| Field | Type | Default | Location |
| -------------------------- | ------ | ------- | ------------------------------------------------- |
| Field | Type | Default | Location |
| --- | --- | --- | --- |
| `DisableResponseBuffering` | `bool` | `false` | `opensearch.Config`, `opensearchtransport.Config` |
8 changes: 4 additions & 4 deletions guides/retry_backoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,10 @@ To limit total wait time when the server is unresponsive, use a context with a d

`RequestTimeout` and context deadlines serve different purposes:

| Mechanism | Scope | Applies to |
| --------------------- | ----------- | ----------------------------------------------------------- |
| `RequestTimeout` | Per-attempt | Each individual HTTP round-trip (including each retry) |
| `context.WithTimeout` | Total | The entire operation across all attempts and backoff delays |
| Mechanism | Scope | Applies to |
| --- | --- | --- |
| `RequestTimeout` | Per-attempt | Each individual HTTP round-trip (including each retry) |
| `context.WithTimeout` | Total | The entire operation across all attempts and backoff delays |

Use both together for defense in depth: `RequestTimeout` prevents any single attempt from hanging indefinitely, while a context deadline caps the total wall-clock time.

Expand Down
Loading
Loading