Skip to content

fix: upload 0-byte object when closing open_s3_object in write mode - #3422

Open
hsusul wants to merge 1 commit into
aws:mainfrom
hsusul:fix/open-s3-object-empty-write
Open

fix: upload 0-byte object when closing open_s3_object in write mode#3422
hsusul wants to merge 1 commit into
aws:mainfrom
hsusul:fix/open-s3-object-empty-write

Conversation

@hsusul

@hsusul hsusul commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes #3421

Affected API

awswrangler.s3._fs.open_s3_object / awswrangler.s3

Background & Root Cause

When an S3 object stream is opened in write mode (mode="w" or mode="wb") using open_s3_object, and closed after writing 0 bytes (e.g. creating an empty file, writing b"", or exiting the context manager without write()), _S3ObjectBase.close() previously checked:

if self.writable():
    if self._parts_count > 0:
        ... # complete multipart upload
    elif self._buffer.tell() > 0:
        ... # put_object

When self._parts_count == 0 and self._buffer.tell() == 0 (0 bytes written), both conditions evaluated to False. Consequently, put_object was skipped, leaving the S3 object non-existent if new, or retaining pre-existing content if overwriting.

Corrected Behavior

_S3ObjectBase.close() now calls put_object whenever self._parts_count == 0, uploading Body=self._buffer.getvalue() (which is b"" when 0 bytes are written). This matches standard Python file semantics (open(path, "wb")), correctly creating or truncating the S3 object to 0 bytes.

Local Reproduction & Verification

Added a unit test test_open_s3_object_empty_file_write in tests/unit/test_moto.py covering:

  1. Opening a non-existent S3 path in "wb" mode without writing, verifying does_object_exist returns True and ContentLength == 0.
  2. Opening an existing S3 path containing prior data in "w" mode and writing "", verifying the object is truncated to 0 bytes (ContentLength == 0).

Exact Validation Results

  • pytest tests/unit/test_moto.py -> 46 passed in 3.25s
  • pytest tests/unit/test_utils.py -> 18 passed in 0.02s
  • ruff format --check . -> Passed
  • ruff check . -> Passed
  • git diff --check -> Clean

AWS Integration Tests Not Run

Live AWS service tests (requiring AWS infrastructure / credentials) were not executed. Verification was completed using the repository's unit test suite with moto.

Compatibility & Resource Ownership Behavior

  • Preserves all public APIs and existing return types.
  • Idempotent stream closure logic remains intact.
  • No third-party dependencies added or mutated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: wr.s3.open_s3_object in write mode does not create or overwrite 0-byte objects

1 participant