Skip to content

aiohttp does not skip response body when a HEAD request response has a body when using C extensions #10322

Open
@jonathon-love

Description

@jonathon-love

Describe the bug

aiohttp barfs on a dropbox download, that other software (i.e. curl) doesn't seem to have difficulty with.

To Reproduce

import aiohttp
import asyncio

async def download_file(url):
    async with aiohttp.ClientSession() as session:
        async with session.head(url, allow_redirects=True) as resp:
            pass

url = 'https://www.dropbox.com/scl/fi/818zs3kiois9qkv2axv9t/Tooth-Growth.omv?rlkey=f4m2lv069py0ezn8udmlxsosn&st=2owyrsrs&dl=1'

asyncio.run(download_file(url))

Expected behavior

200 OK

Logs/tracebacks

Traceback (most recent call last):
  File "/Users/XXXX/Library/Caches/pypoetry/virtualenvs/fred-L1l92eZt-py3.12/lib/python3.12/site-packages/aiohttp/client_proto.py", line 263, in data_received
    messages, upgraded, tail = self._parser.feed_data(data)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "aiohttp/_http_parser.pyx", line 558, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadHttpMessage: 400, message:
  Invalid character in chunk size:

    b'\x1f\x8b\x08'
      ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/XXXX/Library/Caches/pypoetry/virtualenvs/fred-L1l92eZt-py3.12/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 1059, in start
    message, payload = await protocol.read()  # type: ignore[union-attr]
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/XXXX/Library/Caches/pypoetry/virtualenvs/fred-L1l92eZt-py3.12/lib/python3.12/site-packages/aiohttp/streams.py", line 671, in read
    await self._waiter
aiohttp.http_exceptions.HttpProcessingError: 400, message:
  Invalid character in chunk size:

    b'\x1f\x8b\x08'
      ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/Cellar/[email protected]/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.12.3/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "<stdin>", line 3, in download_file
  File "/Users/XXXX/Library/Caches/pypoetry/virtualenvs/fred-L1l92eZt-py3.12/lib/python3.12/site-packages/aiohttp/client.py", line 1425, in __aenter__
    self._resp: _RetType = await self._coro
                           ^^^^^^^^^^^^^^^^
  File "/Users/XXXX/Library/Caches/pypoetry/virtualenvs/fred-L1l92eZt-py3.12/lib/python3.12/site-packages/aiohttp/client.py", line 730, in _request
    await resp.start(conn)
  File "/Users/XXXX/Library/Caches/pypoetry/virtualenvs/fred-L1l92eZt-py3.12/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 1061, in start
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 400, message="Invalid character in chunk size:\n\n  b'\\x1f\\x8b\\x08'\n    ^", url='https://uc7f6f4bf01d26ef6c48853d3c4c.dl.dropboxusercontent.com/cd/0/get/CiHGZDofY40BymZ9BQFqKYP2T_NQroYSU7UaqEPQtQDkAYFB_GCSpSAvxX-j1cFVIVHx0qSE5QFz-cJ5cNrnweRXgVIyXn04tg3k_asEek_G5zxbLThKClANeLOqWVCZKB5o3tm7u0TEiWx3aiwqB3-A/file?dl=1'


### Python Version

```console
$ python --version

`Python 3.12.3`

aiohttp Version

$ python -m pip show aiohttp

`Version: 3.11.11`

multidict Version

$ python -m pip show multidict

Version: 6.1.0

propcache Version

$ python -m pip show propcache

0.2.1

yarl Version

$ python -m pip show yarl

1.18.3

OS

linux, macOS

Related component

Client

Additional context

No response

Code of Conduct

  • I agree to follow the aio-libs Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions