Commit ad1facf
Add PolarsCursor for native Polars DataFrame support
This commit introduces PolarsCursor and AsyncPolarsCursor for native Polars
DataFrame support without requiring PyArrow as a mandatory dependency.
Key features:
- PolarsCursor: Synchronous cursor that returns results as Polars DataFrames
- AsyncPolarsCursor: Asynchronous cursor for non-blocking query execution
- Support for both regular queries and UNLOAD operations
- Proper type conversion for all Athena data types including decimal with precision/scale
- CSV reading via fsspec with S3 support
- Parquet reading via Polars native object_store
Implementation details:
- DefaultPolarsTypeConverter: Converts Athena types to Polars dtypes
- DefaultPolarsUnloadTypeConverter: Minimal converter for UNLOAD operations
- AthenaPolarsResultSet: Result set with Polars DataFrame conversion
- SQLAlchemy dialect support via AthenaPolarsDialect
Additional improvements:
- Added get_dtype method to base Converter class for extensible type handling
- Moved is_unload property to AthenaResultSet base class
- Updated pandas and arrow result sets to use get_dtype method
- Added as_polars() method to Arrow cursors for interoperability
- Comprehensive documentation and tests
Closes #436
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>1 parent 81cf7d6 commit ad1facf
File tree
30 files changed
+2806
-45
lines changed- docs
- api
- pyathena
- arrow
- pandas
- polars
- spark
- sqlalchemy
- tests
- pyathena
- polars
- resources/queries
30 files changed
+2806
-45
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
| |||
44 | 45 | | |
45 | 46 | | |
46 | 47 | | |
| 48 | + | |
47 | 49 | | |
48 | 50 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
319 | 319 | | |
320 | 320 | | |
321 | 321 | | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
322 | 332 | | |
323 | 333 | | |
324 | 334 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| 35 | + | |
| 36 | + | |
35 | 37 | | |
36 | 38 | | |
37 | 39 | | |
| |||
43 | 45 | | |
44 | 46 | | |
45 | 47 | | |
46 | | - | |
| 48 | + | |
47 | 49 | | |
48 | 50 | | |
49 | 51 | | |
| |||
0 commit comments