Skip to content

Commit 8cfd5bf

Browse files
committed
fix: Correct Elasticsearch deployment guide pagination claim
The deployment guide stated the connector issues a single `_search` request per query returning at most 10,000 hits and advised users to accelerate for larger result sets. In reality, the connector automatically paginates using PIT + `search_after` for queries without LIMIT or with LIMIT > 10,000 — matching what the connector index.md already documents (corrected in PR #1712) and what the code implements at query_table.rs:231.
1 parent 179393a commit 8cfd5bf

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

  • website/docs/components/data-connectors/elasticsearch

website/docs/components/data-connectors/elasticsearch/deployment.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ Long-running search responses (very large `LIMIT`, deep pagination, or expensive
5757
## Capacity & Sizing
5858

5959
- **Throughput**: Bounded by the Elasticsearch cluster's request handling and (for kNN) HNSW search cost. Plan refresh intervals and concurrent query load to stay within the cluster's tested capacity.
60-
- **Result size**: The connector issues a single `_search` request per query, returning at most 10,000 hits (bounded by the Elasticsearch `index.max_result_window` setting). Queries with `LIMIT N` fetch `min(N, 10000)` rows. For result sets larger than 10,000, accelerate the dataset.
60+
- **Result size**: For queries with `LIMIT N` where N ≤ 10,000, the connector issues a single `_search` request. For larger result sets or queries without `LIMIT`, the connector automatically paginates using Point-In-Time (PIT) + `search_after`, fetching all matching documents in 10,000-hit batches (bounded by the Elasticsearch `index.max_result_window` setting per batch).
6161
- **Mapping fetches**: At dataset registration the connector fetches the index mapping once via `GET /<index>/_mapping`. Mapping changes after registration are not picked up until the runtime restarts.
6262

6363
## Search Routing

0 commit comments

Comments
 (0)