feat(s3): add pandas_mode to to_csv to pass file open mode (e.g. a) t… - #3363
Open
bujjibabukatta wants to merge 10 commits into
Open
feat(s3): add pandas_mode to to_csv to pass file open mode (e.g. a) t…#3363bujjibabukatta wants to merge 10 commits into
bujjibabukatta wants to merge 10 commits into
Conversation
…hrough to pandas, resolving collision with dataset mode parameter
bujjibabukatta
force-pushed
the
fix/#2409
branch
from
June 11, 2026 12:23
5385803 to
b482ff4
Compare
Contributor
Author
|
Hi @kukushking could you please review and approve pull request? |
bujjibabukatta
force-pushed
the
fix/#2409
branch
from
June 11, 2026 13:17
e3cb664 to
b482ff4
Compare
kukushking
requested changes
Jun 14, 2026
kukushking
left a comment
kukushking
left a comment
Collaborator
There was a problem hiding this comment.
Code checks are failing: https://github.com/aws/aws-sdk-pandas/actions/runs/27397307636/job/80967231139?pr=3363
bujjibabukatta
force-pushed
the
fix/#2409
branch
from
June 17, 2026 16:27
7d73308 to
370d187
Compare
bujjibabukatta
force-pushed
the
fix/#2409
branch
from
June 17, 2026 17:26
370d187 to
ee0da55
Compare
bujjibabukatta
force-pushed
the
fix/#2409
branch
from
June 17, 2026 18:19
e739299 to
8cf6499
Compare
bujjibabukatta
force-pushed
the
fix/#2409
branch
from
June 17, 2026 18:25
8cf6499 to
22561e1
Compare
Contributor
Author
|
Hi @kukushking I fixed it.could you please review and approve pull request? |
Contributor
Author
|
Hi @kukushking, @goleash-4alight I have fixed code . Please review and approve pull request? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
wr.s3.to_csv() has a mode parameter for dataset write behaviour ("append", "overwrite"). pandas df.to_csv() also has a mode parameter for file open mode ("w", "a"). These collide — there was no way to pass pandas mode="a" to append to an existing file.
Solution
Add a pandas_mode parameter to to_csv that passes the file open mode through to pandas. Since S3 has no native append, the implementation reads the existing file content first and writes it back before appending the new data.
Changes
awswrangler/s3/_write_text.py — add pandas_mode param, implement S3-level append in _to_text, fix mode=mode being incorrectly passed to df.to_csv()
tests/unit/test_moto.py — add test_csv_pandas_mode_append