Skip to content

Commit b254f15

Browse files
committed
Update README pagination section for max_rows behavior
1 parent e5bc861 commit b254f15

1 file changed

Lines changed: 16 additions & 7 deletions

File tree

README.md

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -210,15 +210,24 @@ Path parameters are required — omitting them fails at query planning, before a
210210

211211
## Pagination
212212

213-
Queries without `LIMIT` return a single page of results (whatever the API's default page size is). Add `LIMIT` to fetch across multiple pages automatically:
213+
sqlize paginates automatically using the standard `Link` header (`rel="next"`) or common response body fields (`next`, `next_url`). This works with GitHub, GitLab, Stripe, and most REST APIs without configuration.
214214

215-
```sql
216-
SELECT number, title FROM issues
217-
WHERE owner = 'rust-lang' AND repo = 'rust'
218-
LIMIT 250;
219-
```
215+
Each table scan fetches pages lazily until one of these limits is reached:
216+
217+
- **SQL `LIMIT`** — when DataFusion can push it down (simple queries), only the needed pages are fetched
218+
- **`max_rows`** (default 1000) — caps total rows per table scan when no SQL `LIMIT` applies (e.g. JOINs, aggregations, or queries without `LIMIT`)
219+
220+
The current page always completes, so actual row count may slightly exceed the cap.
220221

221-
sqlize follows pagination using the standard `Link` header (`rel="next"`) or common response body fields (`next`, `next_url`). This works with GitHub, GitLab, Stripe, and most REST APIs without configuration.
222+
Override the default:
223+
224+
```sh
225+
# CLI flag (one-off)
226+
sqlize --spec specs/github.json --max-rows 5000
227+
228+
# Environment variable (persistent)
229+
export SQLIZE_MAX_ROWS=5000
230+
```
222231

223232
## Why SQL
224233

0 commit comments

Comments
 (0)