Best practice for deduplicating in-flight requests

[bartoszirzyk]

Motivation

Hello,
I wanted to ask how to approach with removing duplicated request in flight? I was wondering about creating dedicated Middleware that will catch all simultaneous calls, fire first one and on return share result with other callers. This might work fine, but I faced an issue where the first call is cancelled. This triggers cancel handler in URLSessionTransport that cancels request and returns nothing. This propagates to all callers waiting for the response.

Proposed solution

In iOS 27 there will be shield mechanism https://developer.apple.com/documentation/swift/withtaskcancellationshield(operation:)
but I need to have compatibility with older iOS versions.

I thought about shared detached Task, but ClientMiddleware protocol assumes that next closure is non escaping, which would be required in order to pass it into a new task.

Other option I would see is to be able to control cancel behaviour in URLSessionTransport. A flag in configuration that determines if canceling task means that dataTask is cancelled/ operation checks for cancellation. Right now call is fired anyway by URLSession, but later checks in e. g. bidirectionalStreamingRequest:

            try Task.checkCancellation()
            return (try HTTPResponse(response), responseBody)

makes impossible to react in higher layers. I would like to have more freedom of controlling such behaviour. If task is cancelled, it would make sense to still have a response and its body to evaluate and maybe clean up.

Alternatives considered

No response

Additional information

No response

[simonjbeaumont]

Hello, I wanted to ask how to approach with removing duplicated request in flight? I was wondering about creating dedicated Middleware that will catch all simultaneous calls, fire first one and on return share result with other callers.

IMO this is the right instinct, yeah.

I thought about shared detached Task, but ClientMiddleware protocol assumes that next closure is non escaping, which would be required in order to pass it into a new task.

I would caution against detached tasks if possible.

Right now call is fired anyway by URLSession, but later checks in e. g. bidirectionalStreamingRequest:

            try Task.checkCancellation()
            return (try HTTPResponse(response), responseBody)

makes impossible to react in higher layers.

Is it specifically that final checkCancellation or any others. FWICT at that point the function already has the result in hand and may as well return it. Unless I'm missing something, we could probably remove that final check before the return. Would it be enough? And WDYT @guoye-zhang?

[bartoszirzyk]

I think this would need to be somehow parametrised since we have 2 scenarios:

• request is fired and we cancel data task - no response
• response is captured and at the same time cancel is triggered - we have response and try Task.checkCancellation() will throw error

In order to achieve full control we should be able to say either always cancel on cancel (current implementation) or hey let me handle that and manage myself. WDYT @simonjbeaumont

In general I am opened to any approach that will work to achieve my desired behaviour.

Imho big impact and kinda breaking would be adding escaping to middleware closures that in fact would allow consumers to do mor stuff.
Tempering URLSessionTransport seems more non breaking since it might be behind a configuration flag with default value pointing to current implementation.

I am a bit more for enabling such thing on middleware level since this feels more accurate and universal, but I am very curious about your point of view

[guoye-zhang]

Implementing something equivalent to withTaskCancellationShield using unstructured tasks is fine if you scope the unstructured-ness inside a single function to make it structured. You might be able to use withoutActuallyEscaping for the closure.

Do we have to work around a resolved language limitation by changing the API?

[bartoszirzyk]

@guoye-zhang so basically I can do some work in middleware and at some point have something like this:

                let result = try await withoutActuallyEscaping(next) { escapingClosure in
                    if #available(iOS 26.0, *) {
                        try await Task.immediate { try await escapingClosure(request, body, baseURL) }.value
                    } else {
                        try await Task { try await escapingClosure(request, body, baseURL) }.value
                    }
                }

and this should be safe from withoutActuallyEscaping perspective?

[guoye-zhang]

You still need to make sure that the closure does not actually escape by awaiting for the detached task inside the scope

[bartoszirzyk]

What do you mean by that? Imagine we have middleware where we just do this.

    public func intercept(_ request: HTTPRequest,
                          body: HTTPBody?,
                          baseURL: URL,
                          operationID: String,
                          next: @Sendable (HTTPRequest, HTTPBody?, URL) async throws -> (HTTPResponse, HTTPBody?)) async throws -> (HTTPResponse, HTTPBody?) {
try await withoutActuallyEscaping(next) { escapingClosure in
                    if #available(iOS 26.0, *) {
                        try await Task.immediate { try await escapingClosure(request, body, baseURL) }.value
                    } else {
                        try await Task { try await escapingClosure(request, body, baseURL) }.value
                    }
                }
    }

all it does is passing closure further but inside the task and awaiting value inside withoutActuallyEscaping closure. How this closure can escape? Documentation says

What I understand is that as long as I do all operations inside withoutActuallyEscaping and I wait by awaiting value this should be save, right?

[guoye-zhang]

Oh I didn’t see the .value at the end, the code should work

[bartoszirzyk]

@guoye-zhang awesome! Thanks for confirmation.

Ok you can decide, if you want to close this issue or make some changes in libraries to enable such behaviour in different way