Skip to content

feat: 2-second wall-clock timeout for Cloud Rewrite #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ZhaoChaoqun merged 1 commit into from
Mar 23, 2026
Merged

feat: 2-second wall-clock timeout for Cloud Rewrite #16

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 33 additions & 2 deletions Sources/CloudRewriteService.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ final class CloudRewriteService {
private let apiKeyProvider: () -> String?
private let endpoint: URL

/// Wall-clock timeout for the entire rewrite operation.
/// If the API doesn't respond within this duration, the original text is used.
private static let rewriteTimeout: Duration = .seconds(2)

/// Models to try in priority order. First available model wins.
private static let preferredModels = [
"qwen-3-235b-a22b-instruct-2507",
Expand Down Expand Up @@ -114,7 +118,31 @@ final class CloudRewriteService {
}

do {
return try await rewrite(text: text, apiKey: apiKey)
return try await withThrowingTaskGroup(of: String.self) { group in
// Race 1: the actual API call
group.addTask { [self] in
try await self.rewrite(text: text, apiKey: apiKey)

@claude claude Bot Mar 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: if modelProbeTask hasn't resolved yet when this runs (line 159), the await modelProbeTask.value eats into the 2s budget. Since the probe is fired eagerly at init, this is unlikely in steady state — but on a cold start with a slow /v1/models endpoint, the first rewrite call could time out before the actual chat completion request even begins.

Not blocking — just worth being aware of. If it becomes an issue in practice, you could move the model resolution outside the racing task group so the 2s clock only starts when the HTTP call is made.

}

// Race 2: timeout deadline
group.addTask {
try await Task.sleep(for: Self.rewriteTimeout)
throw RewriteTimeoutError()
}

// First completed result wins; cancel the loser
guard let result = try await group.next() else {
return text
}
group.cancelAll()
return result
}
} catch is RewriteTimeoutError {
cloudRewriteLogger.warning("Cloud rewrite timed out (\(Self.rewriteTimeout, privacy: .public)), using original text (\(text.count, privacy: .public) chars)")
return text
} catch is CancellationError {
cloudRewriteLogger.debug("Cloud rewrite cancelled")
return text
} catch let error as URLError {
cloudRewriteLogger.warning("Cloud rewrite network error: \(error.code.rawValue, privacy: .public). Fallback to original text")
return text
Expand All @@ -132,7 +160,7 @@ final class CloudRewriteService {

var request = URLRequest(url: endpoint)
request.httpMethod = "POST"
request.timeoutInterval = 15
request.timeoutInterval = 3 // Safety net; task group timeout (2s) is the primary enforcer
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
request.setValue("Bearer \(apiKey)", forHTTPHeaderField: "Authorization")

Expand Down Expand Up @@ -265,3 +293,6 @@ private struct ModelListResponse: Decodable {
let id: String
}
}

/// Thrown by the timeout racer to signal that the rewrite deadline has been exceeded.
private struct RewriteTimeoutError: Error {}
Loading