Skip to content

Model tiering: Fable/Opus judgment-execution split (quality-boost + economy directions, default untouched) #517

Description

@Imbad0202

Motivation

Lance Martin (Anthropic), "Cost effective harnesses with Fable" (X article, 2026-07-10): many tasks have intelligence asymmetry across their tokens; harnesses can exploit it via three patterns (frontier model as orchestrator / advisor-at-checkpoints / verifier). Measured results: advisor-checkpoint config reached ~90% of Fable-solo quality at ~34% of token cost (Parameter Golf); delegation only pays when workers absorb enough tokens to offset the fixed per-handoff coordination cost (BrowseComp: +60% cost for nothing on the small subset, 96% score at 46% cost on the full set).

ARS maps onto this cleanly: light slash commands are already sonnet-pinned (correct per the small-task finding), but all pipeline agents are model: inherit — a Fable session burns Fable tokens on mechanical formatting/compilation work, and an Opus session gets no frontier judgment at the gates.

Design decision (frozen 2026-07-11)

One tiering mechanism, two explicit directions, default untouched:

  1. Default: model: inherit everywhere — session model runs everything. Quality ceiling preserved; no opt-out surprise.
  2. Quality-boost direction (Opus session): judgment-type agents upgrade to the frontier tier at Stage 2.5/4.5 gates and final review. Quality strictly rises vs Opus-solo; extra cost concentrated where the article shows it matters (mid-task re-ranking, not upfront planning).
  3. Economy direction (Fable session): execution-type agents downgrade ONE tier max — floor is Opus, never Sonnet (frozen: academic prose quality tolerance is untested; the article's 90%/34% numbers came from ML tuning, not scholarly writing). Documented explicitly as quality-for-cost with the article's numbers cited.

Agent classification (frozen; 39 agents)

Judgment-type (25) — stays on session model / upgrade candidates:

  • deep-research (10): socratic_mentor, research_question, research_architect, synthesis, devils_advocate, editor_in_chief, ethics_review, risk_of_bias, meta_analysis, source_verification
  • academic-paper (6): socratic_mentor, argument_builder, structure_architect, peer_reviewer, revision_coach, literature_strategist
  • academic-paper-reviewer (6): eic, methodology_reviewer, domain_reviewer, perspective_reviewer, devils_advocate_reviewer, editorial_synthesizer (mechanical by v3.6.2 design, but emits the final decision letter — kept judgment-type conservatively until data says otherwise)
  • academic-pipeline + shared (3+1): pipeline_orchestrator, claim_ref_alignment_audit, integrity_verification, compliance_agent (has tier-based block authority)

Execution-type (12) — economy-direction downgrade candidates (floor: Opus):

  • deep-research (4): bibliography (citation existence already handled by the deterministic gate; note the sha256 F2 lock on this file), timeline_extraction, report_compiler, monitoring
  • academic-paper (6): intake, draft_writer (highest-token, highest-savings, highest-risk downgrade point — one tier max, flagged in docs as the suite's most quality-sensitive downgrade), abstract_bilingual, citation_compliance, visualization, formatter (STAMP-ONLY by design)
  • academic-paper-reviewer (1): field_analyst
  • academic-pipeline (2): collaboration_depth (advisory-only, never blocks), state_tracker

Implementation notes

  • Opt-in switch (naming TBD, e.g. ARS_MODEL_TIERING=economy|quality-boost); absence = byte-equivalent current behavior (same philosophy as terminal_policies).
  • Prompt caching: route repeated same-stage calls (e.g. reviewer re-review loops) to the SAME worker so its cache accumulates — a fresh worker per call re-pays the context write and can erase the savings (article guidance item 4).
  • Lint: classification drift guard (an agent added without a tier assignment fails CI).
  • v3.7.0 lesson applies: tiers must be relative to the session model (never hard-pinned model ids), or a floor becomes a downgrade ceiling on the next model generation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions