Cloudflare target: scheduled agent fails every run — alarm-driven continuation returns 403 "Request not allowed"

[danielbroadhurst]

Scheduled agent fails every run: dispatch(sub_...) failed: 403 {"error":{"type":"forbidden","message":"Request not allowed"}} during alarm-driven continuation

Environment

• @flue/runtime 2.0.3, @flue/vite 2.0.3, @flue/cli 2.0.3, @flue/sdk 2.0.3
• Cloudflare target, wrangler 4.113.0, compatibility_date: "2026-06-01", nodejs_compat
• @cloudflare/sandbox 0.11.0 (container-backed sandbox attached via cloudflareSandbox(getSandbox(...)))
• agents (Cloudflare Agents SDK) 0.20.1, resolved purely from @flue/vite's own dependency pin (^0.20.1) — not overridden in our project, and already the latest published version

Symptom

An agent (SentryTriage) dispatched every 5 minutes from a Worker scheduled() cron handler fails on essentially every run:

{
  "level": "error",
  "message": "[flue:submission-processing] {\"agentName\":\"SentryTriage\",\"instanceId\":\"sentry-triage-scheduled\",\"submissionId\":\"sub_01KZTT3BWX1T0WBSRDT93DDQ5F\",\"operation\":\"process\",\"outcome\":\"failed\"}     at Session.throwIfError (index.js:56398:23)\n    at Session.resumeConversationToCompletion (index.js:56598:10)\n    at async index.js:56683:5\n    at async Session.withCallOverrides (index.js:55704:11)\n    at async index.js:55881:70\n    at async Session.runExclusive (index.js:55932:11)",
  "$workers": { "eventType": "alarm", "entrypoint": "FlueSentryTriageAgent", "executionModel": "durableObject" }
}

The underlying FlueError, captured via wrangler tail --format json:

FlueError: dispatch(sub_01KZTT3BWX1T0WBSRDT93DDQ5F) failed: 403 {"error":{"type":"forbidden","message":"Request not allowed"}}
    at Session.throwIfError (index.js:56398:23)
    at Session.resumeConversationToCompletion (index.js:56598:10)
    at async index.js:56683:5
    at async Session.withCallOverrides (index.js:55704:11)
    at async index.js:55881:70
    at async Session.runExclusive (index.js:55932:11)

Full diagnosticsChannelEvents for one failing run (parsed from raw tail output):

schedule:execute  agent=FlueSentryTriageAgent name=sentry-triage-scheduled payload={callback: __flueWakeAgentSubmissions}
schedule:create   agent=FlueSentryTriageAgent name=sentry-triage-scheduled payload={callback: __flueWakeAgentSubmissions}
fiber:run:started agent=FlueSentryTriageAgent name=sentry-triage-scheduled fiberName=flue:submission-attempt
fiber:run:failed  agent=FlueSentryTriageAgent name=sentry-triage-scheduled elapsedMs=781–1595 (varies by run)
  error: dispatch(sub_...) failed: 403 {"error":{"type":"forbidden","message":"Request not allowed"}}

Note there is no model-call (chat) or tool-call trace event anywhere between fiber:run:started and fiber:run:failed — the failure happens before any model turn, inside resumeConversationToCompletion's own setup/continuation, in well under 1.6s.

Separately, the admission call that queues the submission — our own application code calling dispatch(SentryTriage, {...}) from the Worker's scheduled() handler (not from inside a Durable Object alarm) — always succeeds:

{ "event": { "request": { "url": "https://flue.invalid/__flue/internal/dispatch", "method": "POST" }, "response": { "status": 200 } }, "eventType": "fetch", "entrypoint": "FlueSentryTriageAgent" }

So: dispatch/admission from a normal fetch-context succeeds; the subsequent processing, kicked off from the Durable Object's own alarm()-driven __flueWakeAgentSubmissions (per the source comment below), fails with a 403 whose body looks like a generic Cloudflare-edge rejection rather than an application-level error from any provider we call (Anthropic, Sentry, Slack, GitHub — none of which use this JSON shape, and none of which are reached before the failure per the trace above).

What we ruled out via bisection

Built a minimal second agent (ScheduleDebug: useModel + usePersistentState + one trivial tool, no external API calls) and dispatched it on its own cron to isolate variables one at a time:

1. Sandbox-less, 1-minute cadence — 3/3 clean runs, while SentryTriage failed on its one concurrent attempt in the same capture window. Rules out a blanket bug affecting every scheduled Flue/Cloudflare agent.
2. Same cloudflareSandbox/getSandbox container as SentryTriage, still 1-minute cadence — still 3/3 clean (2.5–3.4s per run, so the sandbox/container RPC calls — three exists calls per run against the Sandbox DO, each logged "outcome": "ok" — are not the failure point). Rules out the sandbox/container attachment itself as the sole cause.
3. Currently testing: offsetting ScheduleDebug to a ~5-minute cadence (matching SentryTriage's) to see whether cadence/an idle gap between runs correlates with the failure (results pending at time of filing).

Also ruled out: a stale/mismatched agents SDK version (already at the latest published 0.20.1, resolved only from @flue/vite's own pin, no override in this project) and any application-level provider error (the failure precedes any model or tool call in the trace).

Working theory

@flue/runtime's Cloudflare target source contains this comment (dist/cloudflare/internal.mjs, on the generated Durable Object class):

/**
* Durable schedule target that owns submission supervision: armed at
* zero delay by admission/abort/recovery/fiber-settle boundaries and
* at 30s as the heartbeat while unsettled work exists. Dispatched
* from the Durable Object's alarm invocation as one bounded,
* storage-only pass that reconciles, enforces deadlines, and starts
* attempt fibers detached — the fibers outlive the invocation on the
* SDK's runFiber keepAlive/recovery machinery.
*/
__flueWakeAgentSubmissions() {
    return runtime.drainSubmissions(this);
}

So every submission's actual processing is kicked off from inside the DO's own alarm() handler, and that processing appears to make its own internal continuation call (surfaced generically in the error as dispatch(submissionId)) — most likely another getAgentByName(...).fetch(...)-style self-call through the agents SDK, matching fetchAgent() in dist/cloudflare/internal.mjs ((await getAgentByName(binding, instanceId)).fetch(request)).

This matches a reported Cloudflare platform behavior: outbound fetch() calls made from within a Durable Object's alarm() handler have been reported to return HTTP 403 "Direct IP access not allowed" (Cloudflare Error 1003), even when the identical call succeeds from a normal fetch-handler context: https://community.cloudflare.com/t/workers-durable-objects-outbound-fetch-returning-error-1003-across-multiple-us-data/913410

Open questions for Flue

1. Does the Cloudflare-target continuation path (resumeConversationToCompletion → whatever dispatch(submissionId) calls) make a self-fetch from inside the DO's alarm() handler?
2. If so, is that hitting the Cloudflare edge restriction described above, and is there a way to avoid the self-fetch entirely in that path (e.g. an in-process call instead of a fetch, or a documented pattern for DO-to-self calls that stays off the public/edge routing)?
3. Why would this correlate with agent complexity/duration — SentryTriage (5 tools, one mounted skill, claude-sonnet-4-6) fails on effectively every run, while a much lighter agent on a faster cadence has not reproduced it in our testing so far? Is there a timing window during render/setup that affects whether the internal continuation call happens to hit this path?

Happy to share full raw wrangler tail --format json captures on request.

[danielbroadhurst]

Update: tested this directly rather than only via the stand-in agent — dropped SentryTriage's own cron from every 5 minutes to every 1 minute (same cadence the sandbox-equipped ScheduleDebug agent succeeded at 6/6 times) and re-deployed.

Result: still failed, 2/2 runs at the new cadence, with the identical error:

FlueError: dispatch(sub_01KZTWKP0RXZPS66DTBXHMWMB2) failed: 403 {"error":{"type":"forbidden","message":"Request not allowed"}}
  elapsedMs: 885
FlueError: dispatch(sub_01KZTWNGKSB5EX2GMBQ1CZ4KPR) failed: 403 {"error":{"type":"forbidden","message":"Request not allowed"}}
  elapsedMs: 988

Both attributed to agent: FlueSentryTriageAgent, name: sentry-triage-scheduled, same fiber:run:started → fiber:run:failed shape as before.

This rules out cadence/cold-start-after-idle as the cause — reverted the cron back to 5 minutes. The differentiator is something else specific to SentryTriage's own setup versus the minimal ScheduleDebug agent (which shares the same Cloudflare Sandbox/container and the same 1-minute cadence but has never failed): most likely candidates left un-bisected are the mounted skill (useSkill), the larger tool count (5 vs 1), the useResponseFinish-based usage-telemetry hook, or the model (claude-sonnet-4-6 vs claude-haiku-4-5). Will report back if we isolate further.

[danielbroadhurst]

Root cause found — and it's not what either of us suspected. It's specific to the Durable Object instance, not the agent's code, cadence, or the sandbox.

Bisected SentryTriage itself down to bare bones (model + sandbox only, no skill/tools/telemetry/persisted state — structurally identical to the ScheduleDebug stand-in at that point) and it still failed every run on the long-lived sentry-triage-scheduled instance id. That instance had been failing on every cron tick for over an hour of debugging by that point, accumulating a large backlog of failed/interrupted submissions in its own storage.

Ran the exact same bare-bones code under a brand-new instance id instead (sentry-triage-fresh-test-1) and it succeeded 3/3. Restored the full original agent (all 5 tools, mounted skill, claude-sonnet-4-6, persisted state) on a new permanent instance id and it now runs clean end to end — real repo clone, real investigation, no 403.

So: this looks like a specific Durable Object instance getting "stuck" after enough failed/interrupted submissions pile up — quite possibly a legitimate Cloudflare-side anti-abuse throttle on repeated failed self-fetches from one DO (matching the Error 1003 pattern linked above), rather than a general Flue bug. The workaround is straightforward (abandon the poisoned instance id, use a fresh one), but it'd be good to know:

1. Is there a known threshold (failed-attempt count, time window) that trips this?
2. Does it ever self-heal, or is a poisoned instance id stuck permanently?
3. Is there a way to detect "this instance is in a bad state" from application code, so we don't have to notice via silent failures next time?

Leaving this open since the underlying trigger (whatever pushes a DO into this state) is still unclear, even though our specific deployment is unblocked.