Access to private s3 bucket fails when requester_pays is False

[matthewhanson]

As a bucket owner, or one with valid credentials to a private bucket, I must set requester_pays to True, even if the bucket is not requester pays.

The problem lies here.

    def _create_client(self) -> ClientCreatorContext:
        retries = {
            "max_attempts": self.max_attempts,
            "mode": self.retry_mode,
        }
        if self.requester_pays:
            config = botocore.config.Config(retries=retries)
        else:
            config = botocore.config.Config(signature_version=UNSIGNED, retries=retries)
        return self.session.create_client(
            "s3",
            region_name=self.region_name,
            endpoint_url=self.endpoint_url,
            config=config,
        )

If requester pays is False, but the bucket is private then it uses unsigned URLs.

It's an easy work around to requester_pays=True, because even though it's not a requester pays bucket the s3 service will ignore the parameter.

[gadomski]

Thanks for the report ... FYI, if I set aside cycles for this project I'm probably more likely to implement obstore (#246) as we would get a lot more auth*n for free.

Happy to take PRs to fix this directly, tho.