topic: agents type: research status: research-complete last-validated: 2026-05-21 original-query: Analyze agent harness engineering framework (LangChain DeepAgents) and extract patterns for ZAO OS agent roadmap (reconstructed) tier: reference
Status: Research complete Date: March 27, 2026 Goal: Analyze Viv Trivedy's (LangChain) agent harness engineering framework and extract patterns applicable to ZAO OS's AI agent roadmap Source tweet: x.com/vtrivedy10/status/2037203679997018362
| Decision | Recommendation |
|---|---|
| Core concept | Agent = Model + Harness. The harness (system prompt, tools, filesystems, sandboxes, orchestration) matters as much as the model itself |
| For ZAO OS ElizaOS agent | Apply harness engineering principles — don't just pick a model, design the system around it (tools, memory, context strategy) |
| Virtual filesystems | USE virtual filesystem pattern for any ZAO agent that needs persistent state across sessions (onboarding agent, moderation agent, fractal tracking) |
| Ralph Loop for long tasks | For any ZAO agent doing multi-step work (e.g., weekly fractal compilation, cross-platform publishing), implement a Ralph Loop: intercept completion, relaunch in clean context with filesystem state |
| Context rot prevention | ZAO OS agents will need compaction (summarize old context) + tool output offloading (save verbose results to filesystem, keep only summaries in context) |
| DeepAgents library | EVALUATE langchain-ai/deepagents as the runtime for ZAO's planned AI agents — it provides planning, filesystem, subagents, and context management out of the box |
| Current ZAO OS state | No harness/agent infrastructure exists yet in src/. This is greenfield. Memory doc references ElizaOS agent as post-client priority |
Viv Trivedy (@vtrivedy10, LangChain) has been publishing a series on harness engineering — the discipline of building structured systems around LLMs to make them reliable work engines. The core insight:
"If you're not the model, you're the harness."
The harness includes everything that isn't the model weights:
- System prompts and tool descriptions
- Skills, MCPs, and bundled infrastructure
- Filesystem and sandbox environments
- Orchestration logic for subagent coordination
- Middleware hooks for deterministic execution
Key finding (May 2026): LangChain's coding agent improved from 52.8% to 66.5% on Terminal Bench 2.0 (rank Top 30 → Top 5) through harness engineering alone, without model changes. On Terminal Bench 2.0, Anthropic's Opus 4.6 performs significantly better in optimized third-party harnesses than in native Claude Code. Infrastructure design rivals model selection in importance.
Filesystems are the foundational agent pattern. LangChain uses them for:
- Data storage — persistent workspace across sessions
- Memory — agent knowledge that survives context window resets
- Collaboration scratchpad — shared state between subagents
- Git integration — versioning, branching, rollback for agent work
Any storage can "look like a filesystem" by mapping read, write, ls to a backend (S3, Notion, Box, Supabase, custom). Agents stay in their training distribution (great at filesystem ops) while plugging into real-world storage.
DeepAgents tools: read_file, write_file, edit_file, ls, glob, grep
ZAO OS relevance: A fractal-tracking agent could use a virtual filesystem backed by Supabase to persist weekly results, member scores, and submission history — all accessible via familiar file operations.
Rather than pre-configuring every possible tool, harnesses grant agents general-purpose code execution. This enables autonomous problem-solving without human intervention for each action type.
Isolated execution environments protect against malicious code while enabling:
- Dependency installation
- Environment scaling
- Tool pre-configuration
- Self-verification loops through testing and log inspection
Memory files (like AGENTS.md, CLAUDE.md) inject contextual knowledge at startup. Combined with web search and MCP tools for accessing information beyond training data.
ZAO OS relevance: ZAO already uses CLAUDE.md + community.config.ts for context injection. This same pattern extends to any AI agent — inject community config, member data, governance rules at startup.
As context windows fill, agent performance degrades ("context rot"). Solutions:
| Strategy | How It Works | ZAO OS Application |
|---|---|---|
| Compaction | Summarize old context before window fills | Long governance discussions, fractal history |
| Tool output offloading | Save verbose results to filesystem, keep summaries in context | API responses, member data queries |
| Skills/progressive disclosure | Load capabilities on-demand, not all at startup | Agent loads music tools only when in music context |
The most novel pattern. For tasks spanning multiple context windows:
- Agent works toward a goal using filesystem for state
- When the context window fills or agent tries to exit prematurely
- A harness-level hook intercepts the exit
- Reinjects the original prompt in a clean context window
- Agent picks up from filesystem state and continues
This enables multi-hour, multi-window autonomous work. The filesystem is the bridge between context windows.
ZAO OS relevance: A weekly fractal compilation agent could use the Ralph Loop to:
- Process 100+ weeks of historical data across multiple context windows
- Compile cross-platform publishing reports
- Generate governance summaries from multi-day proposal discussions
Models post-trained alongside specific harnesses develop specialized competence within those systems. But this can create brittle overfitting. The research shows harness optimization alone (without model retraining) can deliver significant performance gains.
langchain-ai/deepagents is LangChain's open-source agent harness library.
| Feature | Description |
|---|---|
| Planning | write_todos tool for task decomposition and progress tracking |
| Filesystem | read_file, write_file, edit_file, ls, glob, grep |
| Shell execution | execute command with sandboxing |
| Subagents | task tool for delegating work with isolated contexts |
| Context management | Auto-summarization for long conversations |
| Smart defaults | Built-in prompts teach models effective tool usage |
| Any model | Works with any LLM that supports tool calling |
from langchain.chat_models import init_chat_model
from deepagents import create_deep_agent
agent = create_deep_agent(
model=init_chat_model("anthropic:claude-sonnet-4-6"),
tools=[my_custom_tool],
system_prompt="You are a ZAO community assistant.",
)Storage can be routed to different backends:
- Local filesystem
- S3
- Notion
- Box
- SQLite
- Supabase (relevant for ZAO OS)
LangChain also launched LangSmith Fleet with primitives for agent-heavy futures:
- Agent Identity — identity + security models for agents doing work on behalf of humans
- Observability, tracing, cost tracking
- Prompt management and versioning
Three harness variants tested on coding problems:
- Basic prompt — baseline
- Self-verification — "mentally run examples, rewrite if failing"
- Step-by-step reasoning — explicit corner-case analysis before coding (4/5 problems solved)
Adding ModelCallLimitMiddleware (max 2 invocations per call) reduced costs while maintaining results. Harness constraints can improve both cost and reliability.
| Date | Topic | Key Insight |
|---|---|---|
| Mar 10 | The Anatomy of an Agent Harness | Agent = Model + Harness; harness engineering > prompt engineering |
| Mar 10 | Decoupling Storage from Compute | Give each subagent dedicated compute, share storage |
| Mar 15 | DeepAgents library | Internal agents at LangChain built on it |
| Mar 19 | LangSmith Fleet launch | Agent Identity for agent-heavy workflows |
| Mar 25 | Virtual filesystems | Filesystem pattern for data storage, memory, collaboration |
| Mar 26 | X Article (this tweet) | Comprehensive harness engineering article (1.2K likes, 182 RTs) |
- No AI agent infrastructure in
src/ - ElizaOS agent planned (memory doc:
project_elizaos_agent.md) for onboarding, support, context help CLAUDE.md+community.config.tsalready implement the "memory file" harness pattern- Supabase available as a virtual filesystem backend
ZAO Agent Harness
├── Model: Claude (via Anthropic API or DeepAgents)
├── Harness:
│ ├── System prompt: community.config.ts + member context
│ ├── Virtual filesystem: Supabase-backed (proposals, fractals, members)
│ ├── Tools: governance queries, music curation, cross-post publishing
│ ├── Memory: AGENTS.md with ZAO rules + rolling conversation summaries
│ ├── Context management: compaction + tool output offloading
│ └── Ralph Loop: for multi-step tasks (fractal compilation, report generation)
└── Subagents:
├── Onboarding agent (member intake, wallet verification)
├── Moderation agent (Perspective API + community rules)
├── Fractal agent (weekly compilation, score tracking)
└── Publishing agent (cross-platform content distribution)
Don't just "add an AI agent." Design the harness:
- What tools does it need? (Supabase queries, Neynar API, publishing APIs)
- What state persists across sessions? (member data, governance state, fractal history → virtual filesystem)
- How do you prevent context rot? (compaction for long governance threads)
- What happens when a task spans multiple windows? (Ralph Loop for weekly reports)
- How do subagents coordinate? (shared Supabase-backed filesystem, isolated compute)
- Improving Deep Agents with Harness Engineering — LangChain Blog [FULL]
- LangChain's Harness Engineering: Top 30 to Top 5 Terminal Bench 2.0 [FULL]
- Harness Engineering: Making AI Agents Reliable — DEV Community [FULL]
- The Agent Harness: Why 70% of Performance Lives Outside the Model [FULL]
- LangChain Deep Agents: Harness & Context Engineering [FULL]
- Building an Agent Harness with LangChain Deep Agents [FULL]
- deepagents GitHub repo
- State of Agent Engineering — LangChain
- Viv Trivedy (@vtrivedy10) on X
Updated 2026-05-21: LangChain's breakthrough results confirmed - 52.8% to 66.5% improvement on Terminal Bench 2.0 (gpt-5.2-codex) through harness engineering alone. Three key improvements: system prompts with self-verification loops, enhanced tools + context injection, middleware hooks for doom-loop detection (PreCompletionChecklistMiddleware). Validates that agent infrastructure design rivals model selection in importance.