Skip to content

Commit 6df7e11

Browse files
NikitasT2003claude
andcommitted
test: end-to-end agent tests with a scripted LLM
Previously every test either mocked at the `invoke()` level (the outer entry point) or at `build_agent` (the factory). Nothing exercised the actual deepagents graph — so wiring regressions could slip through CI and only surface on a real LLM call (expensive to debug). This adds 11 tests that run the real deepagents graph — filesystem middleware, subagent middleware, sandbox, skill loader — with the chat model swapped for a scripted `FakeMessagesListChatModel` subclass that replays pre-canned `AIMessage`s (including tool_calls). New helper: `ScriptedLLM` (subclass of `FakeMessagesListChatModel`). - Adds the `bind_tools` / `with_structured_output` no-ops that deepagents middleware requires; the base class raises NotImplementedError on bind_tools. - Tools still get routed by the graph when our AIMessage carries `tool_calls` — they just aren't inspected by the fake model. Test groups: TestWritePathIsRealDisk write_file lands under root_dir read_file → write_file sequence TestCurateUpdatesManifest wiki-curator flow: read raw, write wiki page, edit_file the manifest to flip pending → curated. Locks in that in-place manifest updates must use edit_file (write_file refuses to overwrite existing files). TestStyleLearnerE2E ls → read sample → write STYLE.md TestDrafterE2E read STYLE.md + wiki → write chapter with [src:…] citation TestAgentGracefulOnToolError ToolMessage error is observable; agent recovers on next turn TestDeterministicOutput scripted final message is what the caller sees back TestSandboxViaFilesystemBackend parent traversal (`..`) blocked; Windows absolute paths blocked TestRecursionLimitStopsRunaway an LLM that never emits a final message is bounded by recursion_limit; graph raises GraphRecursionError TestBuildAgentWiringUsesScriptedLLM `build_agent` accepts scripted model when `make_model` is patched; 1 main + 3 subagents = 4 model calls; each subagent gets its `role=` tag. Production change alongside: pinned `FilesystemBackend(virtual_mode=True)` in `agent.build_agent()` because deepagents will flip this default in 0.5.0 and the deprecation warning was noisy. Virtual mode maps `/research/wiki/x.md` (as the LLM sees it) onto `<root_dir>/research/wiki/x.md` on disk, and bans `..` + bare absolute paths — which our sandbox tests now exercise. Stats: 11 new tests, 301 total passing, 1 skipped, ruff clean. Run time for the e2e module: ~2s (no network, no real LLM). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 0dd7c45 commit 6df7e11

2 files changed

Lines changed: 431 additions & 1 deletion

File tree

src/thesis_agent/agent.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,14 @@ def build_agent(
123123

124124
if FilesystemBackend is not None:
125125
try:
126-
kwargs["backend"] = FilesystemBackend(root_dir=str(p.root))
126+
# `virtual_mode=True` makes the agent see a clean virtual
127+
# filesystem rooted at `/` that maps onto `root_dir` on disk.
128+
# Pinned explicitly because deepagents will change the default
129+
# in 0.5.0 (per its DeprecationWarning).
130+
kwargs["backend"] = FilesystemBackend(
131+
root_dir=str(p.root),
132+
virtual_mode=True,
133+
)
127134
except Exception:
128135
pass # library will use its default
129136

0 commit comments

Comments
 (0)