Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions DOCS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1747,7 +1747,7 @@ can never be spelled two ways:
| `verification_failed` | The work ran but the verification refused it — a red suite, or a reviewer that did not approve. |
| `attempt_cap` | The fix loop spent its `stop.attempt_cap` repair rounds with the suite still red. |
| `no_progress` | The SAME checks failing over a tree the repair did not change, round after round — stopped early on purpose. |
| `budget_exhausted` | `stop.budget_minutes` was reached. |
| `budget_exhausted` | A budget was reached: `stop.budget_minutes` (wall-clock), `stop.token_budget` (run lifetime), `stop.phase_token_budget` (one phase), or a phase timeout after real spend. |
| `breadth_exceeded` | The run had already changed more files than `stop.breadth_ceiling`. |
| `blocked_by_gate` | A gate or the permission allowlist refused (`GateFailure`, `PermissionBreach`). |
| `engine_exhausted` | Every engine in the fallback chain died (`EngineFailure`). |
Expand All @@ -1774,16 +1774,24 @@ real failure. An outcome the caller knows precisely is passed in (`attempt_cap`,
`no_progress`); otherwise it is derived (`goal_met` when accepted,
`verification_failed` when not) or classified from the error.

**Stop conditions** — four limits that cost zero tokens to evaluate, so a run
**Stop conditions** — limits that cost zero tokens to evaluate, so a run
that keeps re-trying the same failing thing stops instead of spending the
student's plan. They live under `stop:` in `imp/fia.config.yaml`:

| Key | Default | What it counts |
|---|---|---|
| `attempt_cap` | `3` | Repair rounds `fda_plan_build_test` and `fda_bug` may spend on a red suite (minimum 1 — a value below that is raised). |
| `attempt_cap` | `3` | Repair rounds the tested FDAs (`fda_plan_build_test`, `fda_bug`, `fda_build_test` and the `/goal` default `fda_sdlc`) may spend on a red suite (minimum 1 — a value below that is raised). |
| `no_progress_window` | `2` | Consecutive identical rounds after which the run is declared stuck. `0` turns the detector off. |
| `budget_minutes` | `0` (**off**) | Wall-clock ceiling for one run. |
| `breadth_ceiling` | `0` (**off**) | Maximum files one run may touch. Turning it on costs one tree fingerprint per phase, and a `Kind: foundation` run legitimately touches many. |
| `token_budget` | `30000000` (**on**) | Token ceiling for the RUN LIFETIME — every resume of the same `fda_id` counts against it (the baseline is read from `sessions.total_tokens`, failing open). Warned once at 50% and 80% (`budget_warning` log events), stopped at 100% as `budget_exhausted`. Checked between phases AND between sends, and enforced mid-send by the adapters' token cut. `0` = off. |
| `phase_token_budget` | `8000000` (**on**) | Token ceiling for ONE phase — all its sends, corrections and relay legs. The engine child is cut mid-send at the remaining room (SIGTERM, then SIGKILL). Never arms the relay: re-running a budget-killed phase on another engine would re-spend everything. `0` = off. |
| `phase_timeout_minutes` | `50` (**on**) | Wall-clock ceiling for ONE agent send (`code` phases have their own timeouts). A kill with almost no spend (< 500k tokens) is a hung CLI and retries like a crash — same engine once, then the relay chain; a kill after real spend stops the run as `budget_exhausted` instead of re-paying the phase. `0` = off. |

A budget-stopped run never switches engines or retries on its own: it pauses
with a calm panel pointing at the `stop:` knob — raising the limit (or `0`)
and resuming is a human decision. Cursor reports no token usage, so the token
ceilings cannot see cursor phases (the timeout still applies).

**Browser QA** (`/qa`) video retention lives under optional `qa:` in the same file:

Expand Down
13 changes: 13 additions & 0 deletions fia-templates/fia.config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,19 @@ observability:
# files repeat this many rounds — the run is stuck and
# more rounds only spend your plan. 0 = off.
# budget_minutes: 0 # wall-clock ceiling for one run. 0 = off (the default).
# token_budget: 30000000 # token ceiling for the RUN LIFETIME (every resume of
# the same run counts). Warned at 50% and 80%, stopped
# at 100% with outcome budget_exhausted. 0 = off.
# phase_token_budget: 8000000
# # token ceiling for ONE phase (all its sends and relay
# legs) — the engine is cut mid-send at the limit, so a
# single runaway phase cannot spend the whole plan.
# 0 = off.
# phase_timeout_minutes: 50
# # wall-clock ceiling for ONE agent send. A hung CLI
# with almost no output is killed and retried like a
# crash; a timeout after real spend stops the run
# cleanly instead of re-paying the phase. 0 = off.
# breadth_ceiling: 0 # max files one run may touch. 0 = off (the default);
# turning it on costs one tree fingerprint per phase,
# and a `Kind: foundation` run legitimately touches many.
Expand Down
5 changes: 5 additions & 0 deletions fia-templates/modules/agent-claude.mjs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { spawn } from 'node:child_process';
import { mkdirSync, appendFileSync } from 'node:fs';
import { dirname } from 'node:path';
import { armLimits } from './agent-limits.mjs';

const CLAUDE_BIN = process.env.CLAUDE_PATH || 'claude';
const EFFORT_LEVELS = ['low', 'medium', 'high', 'xhigh', 'max', 'ultracode'];
Expand Down Expand Up @@ -52,6 +53,7 @@ export async function runClaude(request, { onEvent, onSpawn, onExit } = {}) {
},
);
onSpawn?.(child.pid);
const limiter = armLimits(child, request.limits);

let text = '';
let tokens = 0;
Expand Down Expand Up @@ -98,6 +100,7 @@ export async function runClaude(request, { onEvent, onSpawn, onExit } = {}) {
cacheRead += usage.cacheRead;
cacheWrite += usage.cacheWrite;
lastContext = usage.input + usage.cacheRead + usage.cacheWrite;
limiter.noteTokens(tokens);
}
} catch {
text += line;
Expand All @@ -122,6 +125,7 @@ export async function runClaude(request, { onEvent, onSpawn, onExit } = {}) {
cache_write_tokens: cacheWrite,
context_tokens: lastContext,
context_window: 0,
terminated: limiter.finish(),
});
});

Expand All @@ -137,6 +141,7 @@ export async function runClaude(request, { onEvent, onSpawn, onExit } = {}) {
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
terminated: limiter.finish(),
});
});
});
Expand Down
5 changes: 5 additions & 0 deletions fia-templates/modules/agent-cursor.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import { spawn } from 'node:child_process';
import { mkdirSync, appendFileSync } from 'node:fs';
import { dirname } from 'node:path';
import { armLimits } from './agent-limits.mjs';

const CURSOR_BIN = process.env.CURSOR_AGENT_PATH || 'cursor-agent';
const FALLBACK_BIN = 'agent';
Expand Down Expand Up @@ -48,6 +49,8 @@ function spawnOnce(bin, request, { onEvent, onSpawn, onExit }) {
stdio: ['ignore', 'pipe', 'pipe'],
});
onSpawn?.(child.pid);
// No token usage in Cursor's stream — only the wall-clock timeout applies.
const limiter = armLimits(child, request.limits);

let text = '';
let assistantText = '';
Expand Down Expand Up @@ -98,6 +101,7 @@ function spawnOnce(bin, request, { onEvent, onSpawn, onExit }) {
cache_write_tokens: 0,
context_tokens: 0,
context_window: 0,
terminated: limiter.finish(),
});
});

Expand All @@ -117,6 +121,7 @@ function spawnOnce(bin, request, { onEvent, onSpawn, onExit }) {
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
terminated: limiter.finish(),
});
});
});
Expand Down
50 changes: 50 additions & 0 deletions fia-templates/modules/agent-limits.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
/**
* Adapter-side enforcement of per-send ceilings: a wall-clock timeout and a
* token cut. Engines expose no native turn cap (`claude` and `pi` have no
* --max-turns), so the harness enforces limits on the child process itself —
* this is the ONLY place a runaway phase can be stopped while it is running.
* SIGTERM first (the CLIs flush and exit), SIGKILL 10s later if it lingers.
* The reason is reported as `terminated` on the adapter result; the agent
* layer (agents.mjs doSend) classifies it — spend recorded so far is never
* lost, because accounting happens before classification.
*/
const HARD_KILL_DELAY_MS = 10000;

export function armLimits(child, limits = {}) {
let terminated = '';
let softTimer = null;
let hardTimer = null;
const kill = (why) => {
if (terminated) return;
terminated = why;
try {
child.kill('SIGTERM');
} catch {
/* already gone */
}
hardTimer = setTimeout(() => {
try {
child.kill('SIGKILL');
} catch {
/* already gone */
}
}, HARD_KILL_DELAY_MS);
hardTimer.unref?.();
};
if (limits.timeoutMs > 0) {
softTimer = setTimeout(() => kill('timeout'), limits.timeoutMs);
softTimer.unref?.();
}
return {
/** Called by the adapter as usage accumulates; cuts at the token ceiling. */
noteTokens(total) {
if (limits.maxTokens > 0 && total >= limits.maxTokens) kill('token_budget');
},
/** Called on child close: clears the timers, returns why it was cut ('' = it wasn't). */
finish() {
if (softTimer) clearTimeout(softTimer);
if (hardTimer) clearTimeout(hardTimer);
return terminated;
},
};
}
5 changes: 5 additions & 0 deletions fia-templates/modules/agent-pi.mjs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { spawn } from 'node:child_process';
import { mkdirSync, appendFileSync } from 'node:fs';
import { dirname } from 'node:path';
import { armLimits } from './agent-limits.mjs';

const PI_BIN = process.env.PI_PATH || 'pi';

Expand Down Expand Up @@ -58,6 +59,7 @@ export async function runPi(request, { onEvent, onSpawn, onExit } = {}) {
stdio: ['ignore', 'pipe', 'pipe'],
});
onSpawn?.(child.pid);
const limiter = armLimits(child, request.limits);

let text = '';
let tokens = 0;
Expand Down Expand Up @@ -98,6 +100,7 @@ export async function runPi(request, { onEvent, onSpawn, onExit } = {}) {
cacheWrite += parts.cacheWrite;
if (usage.contextSize) contextTokens = usage.contextSize;
if (usage.contextWindow) contextWindow = usage.contextWindow;
limiter.noteTokens(tokens);
}
if (event.type === 'agent_end' && event.text) text = event.text;
} catch {
Expand Down Expand Up @@ -134,6 +137,7 @@ export async function runPi(request, { onEvent, onSpawn, onExit } = {}) {
cache_write_tokens: cacheWrite,
context_tokens: contextTokens || tokens,
context_window: contextWindow,
terminated: limiter.finish(),
});
});

Expand All @@ -149,6 +153,7 @@ export async function runPi(request, { onEvent, onSpawn, onExit } = {}) {
output_tokens: 0,
cache_read_tokens: 0,
cache_write_tokens: 0,
terminated: limiter.finish(),
});
});
});
Expand Down
55 changes: 54 additions & 1 deletion fia-templates/modules/agents.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import * as permissions from './permissions.mjs';
import * as continuation from './continuation.mjs';
import { checkEngines, engineIssue } from './engines.mjs';
import { runChangedPaths } from './git-helper.mjs';
import { StopCondition } from './stop.mjs';
import { OUTCOMES } from './outcome.mjs';
import { makeStreamRecorder } from './stream-events.mjs';
import { extractJson, getOutputSchema } from './envelopes.mjs';
import { gateReport } from './gates.mjs';
Expand Down Expand Up @@ -184,6 +186,7 @@ async function send(run, phase, agent, promptText, systemText, sessionMeta) {
effort: agent.effort,
thinking: agent.thinking,
sessionId: sessionMeta.sessionId,
limits: sessionMeta.limits,
rawOutputPath,
cwd: run.repoRoot,
env: run.env,
Expand All @@ -203,6 +206,7 @@ async function send(run, phase, agent, promptText, systemText, sessionMeta) {
systemPrompt: systemText,
model: agent.model,
sessionId: sessionMeta.sessionId,
limits: sessionMeta.limits,
rawOutputPath,
cwd: run.repoRoot,
env: run.env,
Expand All @@ -222,6 +226,7 @@ async function send(run, phase, agent, promptText, systemText, sessionMeta) {
model: agent.model,
thinking: agent.thinking || 'medium',
sessionFile,
limits: sessionMeta.limits,
rawOutputPath,
tools: agent.tools,
extensions: agent.harness_engineering || [],
Expand Down Expand Up @@ -658,6 +663,27 @@ export async function execute(run, phase, call) {
}
}

/**
* A timeout that already burned this many tokens was doing real work — it
* stops the run instead of being retried as a crash (re-spending it all).
*/
const TIMEOUT_STOP_TOKENS = 500000;

/**
* Adapter-enforced ceilings for ONE send: the phase wall-clock timeout plus
* whatever token room is left under the per-phase and per-run budgets — the
* adapter cuts the child mid-send at the tighter one (agent-limits.mjs).
*/
function phaseLimits(run, spentSoFar) {
const limits = {};
if (run.stop?.phase_timeout_minutes > 0) limits.timeoutMs = run.stop.phase_timeout_minutes * 60000;
const remaining = [];
if (run.stop?.phase_token_budget > 0) remaining.push(Math.max(1, run.stop.phase_token_budget - spentSoFar));
if (run.stop?.token_budget > 0) remaining.push(Math.max(1, run.stop.token_budget - run.lifetimeTokens()));
if (remaining.length) limits.maxTokens = Math.min(...remaining);
return limits;
}

/**
* One full attempt of the phase on the agent's CURRENT engine: session
* resolution, sends, gate loop, permission enforcement, envelope persistence
Expand Down Expand Up @@ -691,7 +717,10 @@ async function attemptPhase(run, phase, call, agent, agentDir, systemText, userT
let enforced = false;

const doSend = async (promptText) => {
const result = await send(run, phase, agent, promptText, systemText, { sessionId });
const result = await send(run, phase, agent, promptText, systemText, {
sessionId,
limits: phaseLimits(run, spentTokens),
});
// Account for the call BEFORE classifying its exit. Engines can return
// real usage together with a non-zero code (limit, crash after generation,
// invalid resumed session). Dropping that spend makes both the session
Expand All @@ -707,6 +736,30 @@ async function attemptPhase(run, phase, call, agent, agentDir, systemText, userT
spentOutput += result.output_tokens || 0;
spentCacheRead += result.cache_read_tokens || 0;
spentCacheWrite += result.cache_write_tokens || 0;
// Token ceilings, AFTER accounting: a StopCondition (never an
// EngineFailure) so it can never arm the relay — re-running a
// budget-killed phase on another engine would re-spend everything.
run.checkTokenBudget({ phaseTokens: spentTokens });
if (result.terminated === 'timeout') {
const minutes = Math.round((run.stop.phase_timeout_minutes || 0));
if (spentTokens >= TIMEOUT_STOP_TOKENS) {
// Real work was underway — re-running it from zero (same engine or a
// fallback) is the failure mode these limits exist to prevent.
const stop = new StopCondition(
OUTCOMES.BUDGET_EXHAUSTED,
`${agent.name} ran past the ${minutes}-minute phase timeout after ${spentTokens} tokens — ` +
'stopping instead of re-spending the phase; raise stop.phase_timeout_minutes if the work is genuinely this long',
);
run.settle(stop.outcome, stop.message);
throw stop;
}
// Little spend = a hung CLI, not long work: classify as a crash so the
// normal path applies (one same-engine retry, then the relay chain).
throw new EngineFailure(
`${agent.name} (${agent.coding_agent}) produced almost nothing in ${minutes} minute(s) and was killed as hung`,
{ kind: 'crash', coding_agent: agent.coding_agent, model: agent.model },
);
}
if (result.returncode === 127) {
throw new EngineFailure(`${agent.name} (${agent.coding_agent}): ${result.text}`, {
kind: 'missing',
Expand Down
2 changes: 1 addition & 1 deletion fia-templates/modules/outcome.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ const LABELS = Object.freeze({
[OUTCOMES.VERIFICATION_FAILED]: 'verification failed',
[OUTCOMES.ATTEMPT_CAP]: 'attempt cap reached',
[OUTCOMES.NO_PROGRESS]: 'no progress',
[OUTCOMES.BUDGET_EXHAUSTED]: 'time budget exhausted',
[OUTCOMES.BUDGET_EXHAUSTED]: 'budget exhausted (time or tokens)',
[OUTCOMES.BREADTH_EXCEEDED]: 'change breadth exceeded',
[OUTCOMES.BLOCKED_BY_GATE]: 'blocked by a gate',
[OUTCOMES.ENGINE_EXHAUSTED]: 'engines exhausted',
Expand Down
Loading
Loading