Skip to content

MinIO custom S3 endpoint does not support our touch implementation #141

Open
@pjbull

Description

@pjbull

Our touch implementation depends on the MetadataDirective="REPLACE":

# just a touch, so "REPLACE" metadata
if src == dst:
o = self.s3.Object(src.bucket, src.key)
o.copy_from(
CopySource={"Bucket": src.bucket, "Key": src.key},
Metadata=self._get_metadata(src).get("extra", {}),
MetadataDirective="REPLACE",
)

However, this fails our test for touch when the rig is custom_s3_rig running against our MinIO test server:

before_touch = datetime.now()
sleep(1)
p.touch()
if not getattr(rig, "is_custom_s3", False):
# Our S3Path.touch implementation does not update mod time for MinIO
assert datetime.fromtimestamp(p.stat().st_mtime) > before_touch

I suspect that MinIO doesn't support this S3 functionality yet or it is a bug. We potentially could workaround by checking if the endpoint_url is set to a non-AWS URL and potentially do the fallback entire copy in that case.

This needs research to understand if this is a MinIO bug we could file to be fixed upstream or if we should workaround it.

When this is fixed, we should re-enable this test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    S3bugSomething isn't workinghelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions