Commit f69345c
Ovtcharov
feat(agents): split chat agent into task-specific agents with prompt profiles
Before: monolithic ChatAgent with 13K-token system prompt caused 95s TTFT
for a simple "Hi!" on Gemma-4-E4B. Eval scenarios timed out at 610s.
After: 5 focused agents (chat, doc, file, data, web) + lite variants,
each with a lean prompt profile. TTFT drops from 95s to 0.12s (chat)
and 3-10s (doc). Eval pass rate: 89% judged (34/38), avg score 9.4/10.
Agent architecture:
- chat: conversation only, ~2K tokens, no tools
- doc: RAG + file search, ~5K tokens, hallucination prevention
- file: filesystem ops + discovery, ~4K tokens
- data: CSV/Excel analysis with scratchpad, ~3K tokens
- web: browser tools, ~2K tokens
- Each has a -lite variant using ~4B model for low-memory hardware
Eval framework updates:
- Per-scenario agent_type field in YAML (overrides --agent-type CLI)
- Latency validation: warns when TTFT > 30s
- Preserve eval sessions for review (no delete_session)
- Increased startup timeout 120s → 240s for Windows
- Fixed shutil.which("claude") for Windows .cmd resolution1 parent c1ed956 commit f69345c
57 files changed
Lines changed: 632 additions & 262 deletions
File tree
- eval/scenarios
- adversarial
- captured
- context_retention
- error_recovery
- personality
- rag_quality
- real_world
- tool_selection
- vision
- web_system
- src/gaia
- agents
- chat
- eval
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
| 4 | + | |
4 | 5 | | |
5 | 6 | | |
6 | 7 | | |
| |||
0 commit comments