Skip to content

Commit 6b3a4a1

Browse files
nsgdsclaude
andcommitted
test(agent): cover image domain in tool-RAG classifier
Verifies image-generation/edit phrasings resolve to the new `images` domain (not low_signal), that the domain seeds generate_image/edit_image, that it has a matching _DOMAIN_RULES pack, and that ordinary prompts are not misclassified as image requests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent e31af09 commit 6b3a4a1

1 file changed

Lines changed: 67 additions & 0 deletions

File tree

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
"""Regression: the agent tool-RAG domain classifier had no image/media domain,
2+
so image-generation requests matched no domain, were flagged low_signal, and had
3+
tool retrieval SKIPPED entirely — the model only received ALWAYS_AVAILABLE tools
4+
(manage_memory, ask_user, update_plan) and never `generate_image`/`edit_image`,
5+
so it could not generate images (and tended to loop on manage_memory).
6+
7+
Root cause: `_classify_agent_request` in src/agent_loop.py sets
8+
`low_signal = not continuation and not domains`; with no `images` domain, prompts
9+
like "generate two images of X" matched nothing -> low_signal -> retrieval skipped.
10+
11+
The classifier is deterministic string matching (no embeddings / no DB), so it
12+
can be exercised directly.
13+
"""
14+
15+
from src.agent_loop import (
16+
_classify_agent_request,
17+
_DOMAIN_TOOL_MAP,
18+
_DOMAIN_RULES,
19+
_domain_rules_for_tools,
20+
)
21+
22+
23+
def _classify(text):
24+
return _classify_agent_request([{"role": "user", "content": text}], text)
25+
26+
27+
def test_image_generation_requests_get_image_domain():
28+
"""Image-generation phrasings must match the `images` domain and NOT be
29+
treated as low-signal (which would skip tool retrieval)."""
30+
prompts = [
31+
"generate two images of this character: one action pose, one relaxed",
32+
"draw me a picture of a cat",
33+
"make an illustration of a spaceship",
34+
"create an image of a sunset over mountains",
35+
"design a logo for my coffee shop",
36+
]
37+
for p in prompts:
38+
intent = _classify(p)
39+
assert "images" in intent["domains"], f"expected images domain for: {p!r}"
40+
assert intent["low_signal"] is False, f"must not be low_signal: {p!r}"
41+
42+
43+
def test_image_edit_requests_get_image_domain():
44+
"""Edit/upscale/background phrasings also resolve to the image domain."""
45+
for p in ("upscale image 5", "remove the background from this photo", "inpaint the selected area"):
46+
intent = _classify(p)
47+
assert "images" in intent["domains"], f"expected images domain for: {p!r}"
48+
49+
50+
def test_image_domain_seeds_generate_and_edit_image():
51+
"""The domain must seed the actual image tools so they are offered even when
52+
semantic retrieval misses."""
53+
assert _DOMAIN_TOOL_MAP["images"] == {"generate_image", "edit_image"}
54+
55+
56+
def test_image_domain_has_a_rule_pack():
57+
"""Every domain in _DOMAIN_TOOL_MAP needs a matching _DOMAIN_RULES entry,
58+
otherwise _domain_rules_for_tools raises KeyError when the tools are selected."""
59+
assert "images" in _DOMAIN_RULES
60+
rules = _domain_rules_for_tools({"generate_image"})
61+
assert any("Image rules" in r for r in rules)
62+
63+
64+
def test_non_image_requests_do_not_match_image_domain():
65+
"""Guard against over-triggering: ordinary prompts must not be flagged image."""
66+
assert "images" not in _classify("what is the capital of France")["domains"]
67+
assert "images" not in _classify("reply to the latest email in my inbox")["domains"]

0 commit comments

Comments
 (0)