Support free-threaded Python 3.13 build

[ngoldbaum]

Is your feature request related to a problem?

Python 3.13 will be available as an experimental free-threaded build that has the GIL disabled. It should hopefully be possible to build aiohttp on the free-threaded build with an updated Cython. I just did so (with some makefile hacking so the build uses a prerelease cython):

Python 3.13.0rc1 experimental free-threading build (main, Aug 14 2024, 13:20:46) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import aiohttp
<frozen importlib._bootstrap>:488: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'aiohttp._helpers', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.

Describe the solution you'd like

I don't know what thread safety issues exist in aiohttp, but assuming any C/C++ code is thread safe and the cython code is thread safe, you can declare the modules support free-threading by following https://py-free-threading.github.io/porting/.

Describe alternatives you've considered

Not supporting it?

Related component

Server, Client

Additional context

No response

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[Dreamsorcerer]

If you'd like to help, I think the first thing would be to add a new test run to the CI, so we have some test coverage over this new approach. But, I'd assume the Cython code is likely thread-safe as it's just used for parsers.

Probably need the current 3.13 tests passing first though, which will hopefully happen next week after uvloop publish a 3.13 wheel.

[webknjaz]

I think that getting our in-org hosted deps to run CI / make builds might be another blocker. I think that aiohttp's CI should make use of those wheels rather than compiling them.

[Dreamsorcerer]

3.13 is fully tested and passing now, if anyone wants to look at adding the free-threading testing.

[webknjaz]

@Yushu2606 what's your point? That doesn't seem related to the discussion here.

[ngoldbaum]

@Yushu2606 you probably aren't building with a nightly cython. Right now there isn't a tagged cython version or release that supports free-threading.

[patrizok]

Please add cython version or release that supports free-threading.

[mshakery]

Is this forgotten?

[Dreamsorcerer]

No, plenty of work has been done on aiohttp dependencies, so looks like it's shaping up for a release in the near future. Plenty of the ecosystem is still not ready for free-threaded, so it's not been a trivial change (e.g. needing to use an alpha release of Cython).

[electroglyph]

i'm interested, too. is there a list of free-threaded blockers that need to be addressed?

[electroglyph]

welp, i just built it for free-threaded on Windows. here's how I did it, i'm sure there will be opinions =)
there's surely a less hacky way to go with the macros...

i've only tested the http client, which works, but i get this msg, any pointers on how to remove this warning would be appreciated:

<frozen importlib._bootstrap>:488: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'aiohttp._http_writer', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.

here's what i changed:

setup.py:

reason: cython's maze of #defs and #undefs

macros = [("GRAALVM_PYTHON", "1"), ("Py_GIL_DISABLED", "1")]
extensions = [
    Extension("aiohttp._websocket.mask", ["aiohttp/_websocket/mask.c"], define_macros=macros),
    Extension(
        "aiohttp._http_parser",
        [
            "aiohttp/_http_parser.c",
            "aiohttp/_find_header.c",
            "vendor/llhttp/build/c/llhttp.c",
            "vendor/llhttp/src/native/api.c",
            "vendor/llhttp/src/native/http.c",
        ],
        define_macros=[("LLHTTP_STRICT_MODE", 0),("GRAALVM_PYTHON", "1"), ("Py_GIL_DISABLED", "1")],
        include_dirs=["vendor/llhttp/build"],
    ),
    Extension("aiohttp._http_writer", ["aiohttp/_http_writer.c"], define_macros=macros),
    Extension("aiohttp._websocket.reader_c", ["aiohttp/_websocket/reader_c.c"], define_macros=macros),
]

_http_parser.c, line 30742:
reason: aiohttp/_http_parser.c(30744): error C2039: 'ob_refcnt': is not a member of '_object'

#if !CYTHON_USE_TP_FINALIZE
    assert(Py_REFCNT(self) > 0);
    // if (likely(--self->ob_refcnt == 0)) {
    //     return;
    // }
    {
        Py_ssize_t refcnt = Py_REFCNT(self);
        _Py_NewReference(self);
        __Pyx_SET_REFCNT(self, refcnt);
    }

websocket\reader_c.c line 18485:
reason: aiohttp/_websocket/reader_c.c(18487): error C2039: 'ob_refcnt': is not a member of '_object'

#if !CYTHON_USE_TP_FINALIZE
    assert(Py_REFCNT(self) > 0);
    // if (likely(--self->ob_refcnt == 0)) {
    //     return;
    // }
    {
        Py_ssize_t refcnt = Py_REFCNT(self);
        _Py_NewReference(self);
        __Pyx_SET_REFCNT(self, refcnt);
    }

[Dreamsorcerer]

Well, the warning is because it doesn't support free-threading. Fixing the warning is done by supporting free-threading, i.e. what this issue tracks.
I think multidict and frozenlist have free-threading now, and yarl I believe is ready to merge and release. That'll probably be all the dependencies sorted, then it's just aiohttp itself that needs a similar update, so seems like it's getting close now.