SSL validation failed for https://server.test/bucket/file.txt [Errno 2] No such file or directory when setting verify to string in S3Storage. #1551
Description
Issue
When setting STORAGES["default"]["OPTIONS"]["verify"] = str(False), such as when parsing an .env file without proper bool casting, a FileNotFoundError: [Errno 2] No such file or directory error is raised by urllib3.
Cause
The origin is boto3.session.Session.client() interpreting verify according to their docs:
- verify (boolean/string) –
Whether or not to verify SSL certificates. By default SSL certificates are verified. You can provide the following values:
- False - do not validate SSL certificates. SSL will still be used (unless use_ssl is False), but SSL certificates will not be verified.
- path/to/cert/bundle.pem - A filename of the CA cert bundle to uses. You can specify this argument if you want to use a different CA cert bundle than the one used by botocore.
Unfortunately this is not clear from django-storages docs, where it merely states:
verify or AWS_S3_VERIFY
Default: None
Whether or not to verify the connection to S3. Can be set to False to not verify certificates or a path to a CA cert bundle.
Suggestion
Internally cast verify to bool or modify docs:
verify or AWS_S3_VERIFY
Default: None
Whether or not to verify the connection to S3. Can be set to:
False- do not validate SSL certificates. SSL will still be used (unless use_ssl is False), but SSL certificates will not be verified.path/to/cert/bundle.pem- A filename of the CA cert bundle to uses. You can specify this argument if you want to use a different CA cert bundle than the one used by botocore.
Though I have not tested wether the custom path is properly supported in django-storages.
Error Traceback
Internal Server Error: /mymodel/create
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 461, in ssl_wrap_socket
context.load_verify_locations(ca_certs, ca_cert_dir, ca_cert_data)
FileNotFoundError: [Errno 2] No such file or directory
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/botocore/httpsession.py", line 465, in send
urllib_response = conn.urlopen(
^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 841, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/urllib3/util/retry.py", line 465, in increment
raise reraise(type(error), error, _stacktrace)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/urllib3/util/util.py", line 39, in reraise
raise value
File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 787, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 488, in _make_request
raise new_e
File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 464, in _make_request
self._validate_conn(conn)
File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1093, in _validate_conn
conn.connect()
File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 796, in connect
sock_and_verified = _ssl_wrap_socket_and_match_hostname(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 975, in _ssl_wrap_socket_and_match_hostname
ssl_sock = ssl_wrap_socket(
^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/urllib3/util/ssl_.py", line 463, in ssl_wrap_socket
raise SSLError(e) from e
urllib3.exceptions.SSLError: [Errno 2] No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/views/generic/base.py", line 105, in view
return self.dispatch(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/contrib/auth/mixins.py", line 73, in dispatch
return super().dispatch(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/contrib/auth/mixins.py", line 109, in dispatch
return super().dispatch(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/views/generic/base.py", line 144, in dispatch
return handler(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/views/generic/edit.py", line 182, in post
return super().post(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/views/generic/edit.py", line 151, in post
return self.form_valid(form)
^^^^^^^^^^^^^^^^^^^^^
File "/app/view/views.py", line 202, in form_valid
self.object = form.save()
^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/forms/models.py", line 554, in save
self.instance.save()
File "/app/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 892, in save
self.save_base(
File "/app/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 998, in save_base
updated = self._save_table(
^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 1161, in _save_table
results = self._do_insert(
^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/db/models/base.py", line 1202, in _do_insert
return manager._insert(
^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/db/models/query.py", line 1852, in _insert
return query.get_compiler(using=using).execute_sql(returning_fields)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1835, in execute_sql
for sql, params in self.as_sql():
^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1760, in as_sql
self.prepare_value(field, self.pre_save_val(field, obj))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/db/models/sql/compiler.py", line 1708, in pre_save_val
return field.pre_save(obj, add=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/db/models/fields/files.py", line 338, in pre_save
file.save(file.name, file.file, save=False)
File "/app/.venv/lib/python3.12/site-packages/django/db/models/fields/files.py", line 99, in save
self.name = self.storage.save(name, content, max_length=self.field.max_length)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/django/core/files/storage/base.py", line 49, in save
name = self._save(name, content)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/storages/backends/s3.py", line 565, in _save
obj.upload_fileobj(content, ExtraArgs=params, Config=self.transfer_config)
File "/app/.venv/lib/python3.12/site-packages/boto3/s3/inject.py", line 764, in object_upload_fileobj
return self.meta.client.upload_fileobj(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/context.py", line 123, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/boto3/s3/inject.py", line 675, in upload_fileobj
return future.result()
^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/s3transfer/futures.py", line 111, in result
return self._coordinator.result()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/s3transfer/futures.py", line 287, in result
raise self._exception
File "/app/.venv/lib/python3.12/site-packages/s3transfer/tasks.py", line 142, in __call__
return self._execute_main(kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/s3transfer/tasks.py", line 165, in _execute_main
return_value = self._main(**kwargs)
^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/s3transfer/upload.py", line 796, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
File "/app/.venv/lib/python3.12/site-packages/botocore/client.py", line 602, in _api_call
return self._make_api_call(operation_name, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/context.py", line 123, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/client.py", line 1060, in _make_api_call
http, parsed_response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/client.py", line 1084, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/endpoint.py", line 119, in make_request
return self._send_request(request_dict, operation_model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/endpoint.py", line 200, in _send_request
while self._needs_retry(
^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/endpoint.py", line 360, in _needs_retry
responses = self._event_emitter.emit(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/hooks.py", line 412, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/hooks.py", line 256, in emit
return self._emit(event_name, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/hooks.py", line 239, in _emit
response = handler(**kwargs)
^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 207, in __call__
if self._checker(**checker_kwargs):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 284, in __call__
should_retry = self._should_retry(
^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 320, in _should_retry
return self._checker(attempt_number, response, caught_exception)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 363, in __call__
checker_response = checker(
^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 247, in __call__
return self._check_caught_exception(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/retryhandler.py", line 416, in _check_caught_exception
raise caught_exception
File "/app/.venv/lib/python3.12/site-packages/botocore/endpoint.py", line 279, in _do_get_response
http_response = self._send(request)
^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/endpoint.py", line 383, in _send
return self.http_session.send(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/botocore/httpsession.py", line 492, in send
raise SSLError(endpoint_url=request.url, error=e)
botocore.exceptions.SSLError: SSL validation failed for https://server.test/bucket/file.txt [Errno 2] No such file or directoryImage
- podman container
ghcr.io/astral-sh/uv:python3.12-bookworm-slim