Skip to content

feat(misc): add multi-agent shared-context prompt caching notebook#707

Open
hipvlady wants to merge 1 commit into
anthropics:mainfrom
hipvlady:hipvlady/multi-agent-prompt-caching
Open

feat(misc): add multi-agent shared-context prompt caching notebook#707
hipvlady wants to merge 1 commit into
anthropics:mainfrom
hipvlady:hipvlady/multi-agent-prompt-caching

Conversation

@hipvlady

Copy link
Copy Markdown

Summary

Adds a notebook under misc/ on shared-context prompt caching across a multi-agent pipeline (planner → worker → reviewer), focused on the case the existing misc/prompt_caching.ipynb doesn't cover: what happens to caching economics when the shared prefix mutates between steps.

It:

  • runs the 3-step pipeline with and without cache_control and reads cost off usage;
  • models the published cache multipliers (write 1.25×/5-min TTL, 2×/1-hour; read 0.1×) as transparent arithmetic;
  • sweeps the prefix mutation rate and plots the crossover where re-writing the cache stops paying — closed-form arithmetic over the published multipliers, not a benchmark (~0.67 for a 3-step, 5-minute-cache pipeline);
  • shows the breakpoint-placement fix (static content before the breakpoint, evolving state after), then closes by distinguishing the cost question from the coherence question once the shared state also has to be current.

Implements the proposal in #596, reworked around the mutation-cost crossover that came up in the discussion there.

Notes

  • Uses claude-haiku-4-5 for the one live call. The shared context is sized above Haiku's 4096-token cache minimum so caching actually engages (and the notebook calls out verifying cache_read_input_tokens > 0).
  • The cost-model and crossover cells are deterministic — they run with no API key and no spend, and their outputs (including the figure) are committed. The single live-API demonstration cell is guarded on ANTHROPIC_API_KEY and left unexecuted, with an adjacent "what you'll see" markdown cell, per the external-contributor testing note in CONTRIBUTING.md.
  • Adds a registry.yaml entry and an authors.yaml entry (kept sorted).

Test plan

  • ruff check and ruff format --check clean
  • scripts/validate_notebooks.py passes (no empty cells, no error outputs)
  • scripts/validate_authors_sorted.py passes
  • registry entry validates against .github/registry_schema.json
  • Maintainers: execute the guarded live cell with an API key to populate real usage numbers

Caching a shared context block across a planner/worker/reviewer pipeline,
and the prefix-mutation crossover where re-writing the cache stops paying.
Cost-model and crossover cells run with no API key; the live demo is guarded.
Adds registry.yaml and authors.yaml entries. Implements anthropics#596.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant