S3 url() resolution slow with querystring_auth=False (unless custom_domain is set)

[oliverhaas]

I noticed our S3 file.url calls were taking ~2ms each, which adds up quickly when rendering pages with many images or having 1000+ image links in an API response. After profiling, I found the bottleneck is botocore's endpoint resolution machinery.

Even with querystring_auth=False, the code path still goes through generate_presigned_url():

connection = (
    self.connection if self.querystring_auth else self.unsigned_connection
)
url = connection.meta.client.generate_presigned_url(
    "get_object", Params=params, ExpiresIn=expire, HttpMethod=http_method
)

And generate_presigned_url does expensive endpoint resolution regardless of whether it's actually signing anything (project is on AWS):

100 .url() calls profiled:

botocore/signers.py:generate_presigned_url     235ms total
botocore/regions.py:construct_endpoint         195ms
botocore/endpoint_provider.py:resolve_endpoint 161ms

The admittedly pretty solid workaround is just setting custom_domain, but that obviously was not clear to me until today.

So my question would be if this is known, and maybe there should be some kind of base-url/endpoint caching or whatever.

Or, of course, I'm missing something obvious, but I looked through the issues and the docs.

And: Thanks for the great package.