-
Notifications
You must be signed in to change notification settings - Fork 7
Open
Description
Description
Duplicate line in WorkerServiceClient::new() where with_max_elapsed_time gets called twice, second call just overwrites the first. This sets max_elapsed_time to None and retries never stop:
let backoff = ExponentialBackoffBuilder::new()
.with_initial_interval(Duration::from_millis(100))
.with_max_elapsed_time(Some(Duration::from_secs(10)))
.with_max_elapsed_time(None) // overwrites previous value
.build();Breaks
Breaks the failover logic in wait_tasks():
async fn wait_tasks(&self, proof_id: String, ids: &[String]) -> anyhow::Result<()> {
// Try to create a subscriber first, otherwise use the failover mechanism.
match self.create_subscriber(proof_id.clone()).await {
Ok((sub_tx, mut sub_rx)) => { /* use subscriber */ }
Err(err) => {
tracing::warn!("Failed to create subscriber, using failover mechanism");
self.wait_tasks_failover(proof_id, ids).await
}
}
}Inside create_subscriber(), the buggy backoff is used for open_sub:
let response = backoff::future::retry(backoff.clone(), || async {
connection
.clone()
.open_sub(request.clone())
.await
.map_err(status_to_backoff_error)
})
.await?; // blocks here indefinitely with max_elapsed_time: NoneWhen open_sub hits transient errors, it should timeout and fall back to wait_tasks_failover(). But with infinite retries, the error branch never runs and the polling fallback never kicks in.
Fix
Just remove the duplicate line:
let backoff = ExponentialBackoffBuilder::new()
.with_initial_interval(Duration::from_millis(100))
.with_max_elapsed_time(Some(Duration::from_secs(10)))
.build();Metadata
Metadata
Assignees
Labels
No labels