Skip to content

Commit 2d08d33

Browse files
KennethKnudsen97Kenneth Sylvest Knudsen
andauthored
fix(shadows): bound delta ack so a lost ack can't strand an applied delta (#139)
apply_delta_and_ack persists the delta to local storage before publishing the reported ack. If the link drops right after publish, the Update Accepted/Rejected response is lost and AWS won't re-send it, so the ack wait parked forever and wait_delta never returned — the caller never learned the delta applied. Bound the ack (DELTA_ACK_TIMEOUT=30s); on timeout keep the applied state and let the reported side reconcile on the next sync. Co-authored-by: Kenneth Sylvest Knudsen <ksk@factbird.com>
1 parent f58f0af commit 2d08d33

1 file changed

Lines changed: 32 additions & 1 deletion

File tree

‎src/shadows/shadow/cloud.rs‎

Lines changed: 32 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,17 @@ use crate::shadows::{
2222

2323
use super::Shadow;
2424

25+
/// Upper bound on how long `apply_delta_and_ack` waits for the cloud to accept
26+
/// the reported ack after a delta has already been applied to local storage.
27+
///
28+
/// The delta is persisted before the ack is published, so a link drop right
29+
/// after publish (the `UpdateAccepted`/`Rejected` response is lost and AWS does
30+
/// not re-send it) would otherwise park the ack wait forever and strand the
31+
/// applied delta — the caller never learns the state changed. On timeout we
32+
/// keep the applied state and let the reported side reconcile on a later
33+
/// sync/reconnect.
34+
const DELTA_ACK_TIMEOUT: embassy_time::Duration = embassy_time::Duration::from_secs(30);
35+
2536
/// Drop-guard that clears `Shadow::subscription` unless explicitly disarmed.
2637
///
2738
/// `handle_delta`'s lazy-subscribe path installs the delta-topic subscription
@@ -117,7 +128,27 @@ where
117128
.map_err(|_| Error::DaoWrite)?;
118129
let reported = state.into_partial_reported(delta);
119130
let cleanup = state.desired_cleanup(delta);
120-
self.update_shadow(cleanup, Some(reported)).await?;
131+
// The delta is already committed to local storage above. Bound the cloud
132+
// ack so a link drop right after the update is published can't park the
133+
// wait indefinitely and strand the applied delta (see DELTA_ACK_TIMEOUT).
134+
// On timeout keep the applied state; the reported side reconciles on the
135+
// next sync (the delta stays pending cloud-side until reported).
136+
match embassy_time::with_timeout(
137+
DELTA_ACK_TIMEOUT,
138+
self.update_shadow(cleanup, Some(reported)),
139+
)
140+
.await
141+
{
142+
Ok(Ok(_)) => {}
143+
Ok(Err(e)) => return Err(e),
144+
Err(_timeout) => {
145+
warn!(
146+
"[{:?}] delta ack timed out after {}s; state applied locally, reported ack deferred",
147+
S::NAME.unwrap_or(CLASSIC_SHADOW),
148+
DELTA_ACK_TIMEOUT.as_secs(),
149+
);
150+
}
151+
}
121152
Ok(state)
122153
}
123154

0 commit comments

Comments
 (0)