Skip to content

Commit a64dfbd

Browse files
Merge pull request #630 from laughingman7743/feature/s3fs-cursor
Add S3FS Cursor for lightweight CSV-based result reading
2 parents 217f863 + 1e08adc commit a64dfbd

27 files changed

+2554
-5
lines changed

docs/api.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ This section provides comprehensive API documentation for all PyAthena classes a
1212
api/connection
1313
api/pandas
1414
api/arrow
15+
api/s3fs
1516
api/spark
1617
api/converters
1718
api/filesystem
@@ -35,6 +36,7 @@ Specialized Integrations
3536

3637
- :ref:`api_pandas` - pandas DataFrame integration
3738
- :ref:`api_arrow` - Apache Arrow columnar data integration
39+
- :ref:`api_s3fs` - Lightweight S3FS-based cursor (no pandas/pyarrow required)
3840
- :ref:`api_spark` - Apache Spark integration for big data processing
3941

4042
Infrastructure

docs/api/s3fs.rst

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
.. _api_s3fs:
2+
3+
S3FS Integration
4+
================
5+
6+
This section covers lightweight S3FS-based cursors, CSV readers, and data converters.
7+
8+
S3FS Cursors
9+
------------
10+
11+
.. autoclass:: pyathena.s3fs.cursor.S3FSCursor
12+
:members:
13+
:inherited-members:
14+
15+
.. autoclass:: pyathena.s3fs.async_cursor.AsyncS3FSCursor
16+
:members:
17+
:inherited-members:
18+
19+
S3FS CSV Readers
20+
----------------
21+
22+
S3FSCursor supports pluggable CSV reader implementations to control how NULL values
23+
and empty strings are handled when parsing Athena's CSV output.
24+
25+
.. autoclass:: pyathena.s3fs.reader.AthenaCSVReader
26+
:members:
27+
28+
.. autoclass:: pyathena.s3fs.reader.DefaultCSVReader
29+
:members:
30+
31+
S3FS Data Converters
32+
--------------------
33+
34+
.. autoclass:: pyathena.s3fs.converter.DefaultS3FSTypeConverter
35+
:members:
36+
37+
S3FS Result Set
38+
---------------
39+
40+
.. autoclass:: pyathena.s3fs.result_set.AthenaS3FSResultSet
41+
:members:
42+
:inherited-members:

docs/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ Documentation
2020
cursor
2121
pandas
2222
arrow
23+
s3fs
2324
spark
2425
testing
2526
api

0 commit comments

Comments
 (0)