Support for intercepting httpx custom transports (BaseTransport) such as httpx-curl-cffi?

[epicwhale]

I use vcrpy to record/replay httpx requests for offline unit tests. This works when httpx uses the default stack, but fails when httpx.Client/AsyncClient is configured with a custom transport (e.g https://github.com/vgavro/httpx-curl-cffi)

It looks like VCR patches httpcore, so custom transports bypass interception?

Is there an intended/official way to support recording/replay for such custom httpx.BaseTransport implementations?

[seowalex]

I'm surprised custom transports do not work given #943 was supposed to fix exactly this case (and works for Hishel, which uses a custom caching transport). Patching at the httpcore level means that any higher-level transports should continue to work with the patched httpcore.

What error are you facing?

[epicwhale]

I'm surprised custom transports do not work given #943 was supposed to fix exactly this case (and works for Hishel, which uses a custom caching transport). Patching at the httpcore level means that any higher-level transports should continue to work with the patched httpcore.

What error are you facing?

It just wasn't recording from my try outs. When I dug deeper, these were my findings I had written down -

    With default httpx transport:
        httpx.AsyncClient
            → httpx.AsyncHTTPTransport
            → httpcore.AsyncConnectionPool.handle_async_request  ← VCR patches HERE
            → network (via httpcore/HTTP libs)

    With curl_cffi transport:
        httpx.AsyncClient
            → httpx_curl_cffi.AsyncCurlTransport.handle_async_request  ← VCR MISSES this!
            → curl_cffi.requests.AsyncSession
            → network (via libcurl)

Perhaps it's because not all httpx transport layers necessarily use the httpcore level?

[Greesb]

We have a nearly similar issue, we use a httpx.ASGITransport and vcrpy seems to bypass the request done with an ASGITransport, causing them to not be recorded, or replayed if using a cassette from vcrpy < 8.0.

[seowalex]

Ah, I hadn't thought about that. Just curious though, previously when it was patched at the httpx level, (if I'm not wrong) the custom transports would still have been bypassed, but that was ok for your use cases as long as the request was recorded?

[Greesb]

In vcrpy<8.0 it is not bypassed because, it was the (Async)Client's _send_single_request method that was patched, which then calls the transport's handle_async_request, so it worked fine since the patch was at the client's level.

The new patchs are targeting the ConnectionPool, but it looks like there's no connection pool for a httpx.ASGITransport.
httpx.ASGITransport, httpx.HTTPTransport and httpx.AsyncHTTPTransport all inherits from httpx.AsyncBaseTransport, which only defines handle_async_request as a required implementation (and some other context manager methods).

[seowalex]

Ok, so it seems like there are two types of transports here:

1. Transports that manipulate the request/response in some form and otherwise proxy it through HTTPTransport.
2. Transports that replace HTTPTransport altogether with a custom transport.

It seems that my PR targeted (1), but in the process stopped (2) from working. I will need to think about the best way to do this so that both can be achieved.

[seowalex]

I looked more into this and I don't think there are any great options. To support (1), we need to patch httpcore, and to support (2), we need to patch httpx. Unfortunately, we cannot patch both at the same time because in the common use case httpx calls httpcore, which will cause 2 identical sets of requests/responses to be recorded.

I can only see a few alternatives, none of which are great:

1. Keep the current patch target, and accept that custom HTTPX transports will have to be custom patched.
   1. I don't like this as this feels like a regression, given that this was previously possible in v7.
2. Revert the patch target to httpx.
   1. This works, but no longer causes custom transports to be invoked, which was the original problem I was trying to solve.
3. Introduce a custom config to VCR.py to choose which target to patch, httpx or httpcore.
   1. This supports both (1) and (2), but introduces library-specific options, which I don't think VCR.py currently has (and I'm not sure it's a wise idea).

[seowalex]

Tagging @kevin1024 @hartwork @jairhenrique in case you have any opinions about alternative 3.

[kevin1024]

Unfortunately, we cannot patch both at the same time because in the common use case httpx calls httpcore, which will cause 2 identical sets of requests/responses to be recorded.

I don’t have the context you have, but could we use some kind of lock to ensure we only record once per interaction? I’d recording at httpx layer, the lock keeps us from recording at the httpcore layer and vice versa? In my opinion, adding extra config to set where vcr is mocking adds quite a bit of extra work for the user and our nice abstraction starts leaking :)

…

-------- Original Message --------

On Friday, 12/26/25 at 06:14 Seow Alex ***@***.***> wrote: seowalex left a comment [(kevin1024/vcrpy#968)](#968 (comment)) I looked more into this and I don't think there are any great options. To support (1), we need to patch httpcore, and to support (2), we need to patch httpx. Unfortunately, we cannot patch both at the same time because in the common use case httpx calls httpcore, which will cause 2 identical sets of requests/responses to be recorded. I can only see a few alternatives, none of which are great: - Keep the current patch target, and accept that custom HTTPX transports will have to be [custom patched](https://vcrpy.readthedocs.io/en/latest/advanced.html#custom-patches). - I don't like this as this feels like a regression, given that this was previously possible in v7. - Revert the patch target to httpx. - This works, but no longer causes custom transports to be invoked, which was the original problem I was trying to solve. - Introduce a custom config to VCR.py to choose which target to patch, httpx or httpcore. - This supports both (1) and (2), but introduces library-specific options, which I don't think VCR.py currently has (and I'm not sure it's a wise idea). — Reply to this email directly, [view it on GitHub](#968 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AAAFE23X2CVAVI46RG5HF5L4DUKBTAVCNFSM6AAAAACPAFRUPWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMOJSG4ZDANBYG4). You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[seowalex]

Yes, I do not like the potential leaking of abstractions either. Using a lock while recording is well and good, but I don't think that works for replaying (unless we introduce a lot of advanced logic, and even then I'm not sure it's possible).

Possibly we may have to revert back to patching httpx as unfortunately I think it covers the widest variety of use cases.

[epicwhale]

If it helps in any way, here's how I ended up patching/fixing it for the transport layer I raised this bug for: https://gist.github.com/epicwhale/4f5e95b6ccee3206b9f53e7352074bef (which heavily used LLMs to reverse engineer and help me work through the nuances of vcrpy, pytest-recording combination) to get my tests back on track.

[seowalex]

Yes, that will work but we probably want to avoid patching each transport since it doesn't generalise well to other transports (and there are probably many more we don't know about).