feat: 2-second wall-clock timeout for Cloud Rewrite#16
Conversation
Use withThrowingTaskGroup to race the API call against a 2s deadline. If the Cerebras API doesn't respond within 2 seconds, the original transcription text is used immediately — no more 12-second waits. Implementation: - TaskGroup race: API call vs Task.sleep(2s), first result wins - Loser is cancelled via group.cancelAll() (frees URLSession resources) - URLRequest.timeoutInterval lowered from 15s to 3s as safety net - Timeout events logged for monitoring - rewriteTimeout defined as named Duration constant Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review Summary
Clean implementation of a 2-second wall-clock timeout for the Cloud Rewrite API using withThrowingTaskGroup racing pattern.
What looks good:
- The race pattern correctly uses structured concurrency — if the timeout fires,
RewriteTimeoutErrorpropagates and the API task is cancelled via structured cancellation. URLSession.data(for:)respects cooperative cancellation, so the in-flight HTTP request is properly cleaned up.- The
URLRequest.timeoutInterval = 3safety net is a sensible backstop. - Model probe is launched eagerly at init time, so it rarely eats into the 2s budget.
No blocking issues found. One minor observation posted inline.
| return try await withThrowingTaskGroup(of: String.self) { group in | ||
| // Race 1: the actual API call | ||
| group.addTask { [self] in | ||
| try await self.rewrite(text: text, apiKey: apiKey) |
There was a problem hiding this comment.
Minor: if modelProbeTask hasn't resolved yet when this runs (line 159), the await modelProbeTask.value eats into the 2s budget. Since the probe is fired eagerly at init, this is unlikely in steady state — but on a cold start with a slow /v1/models endpoint, the first rewrite call could time out before the actual chat completion request even begins.
Not blocking — just worth being aware of. If it becomes an issue in practice, you could move the model resolution outside the racing task group so the 2s clock only starts when the HTTP call is made.
1 participant
Summary
withThrowingTaskGroup实现 API 调用 vs 2 秒 deadline 竞赛URLRequest.timeoutInterval从 15s 降至 3s 作为安全网Test plan
🤖 Generated with Claude Code