Source
Key Finding
The Harness Control System paper formalizes the harness as a 729-configuration RL action space (six discrete levers: prompt_style, tool_policy, memory_policy, planning_policy, verification_policy, max_steps). The headline result: a DSPy-optimized static baseline matches or beats online bandit/REINFORCE controllers at practical episode budgets (25-60 episodes), because dense policies over large action spaces need Omega(M) samples. Even at 300 episodes (5-12x more), online controllers plateau below the static baseline.
Three Engineering Pitfalls to Adopt as Pipeline Hardening
-
Cold-start trap: uninitialized bandit argmax ties deterministically toward action 0 (which happened to be tool_policy=never -- a degenerate config). Fix: optimistic initialization and randomized tie-breaking. For Hermes: when initializing any new pipeline component, ensure the default configuration is not degenerate.
-
Crash-isolation gap: a single malformed episode took down 39 other episodes in the same job. Fix: per-episode try/except, scoring parse failures as normal failed episodes. For Hermes: wrap each pipeline stage in error handling that scores failures as normal failed cycles, not exceptions that propagate.
-
Measurement pitfall: DSPy LM disk cache returns empty usage dict on cache hits, silently under-counting cost. Fix: character-length fallback estimate. For Hermes: when using cached results (e.g., web search cache), account for the cost correctly.
Relevance to Hermes Evolution
- The Binding Constraint Thesis validates the pipeline focus on harness optimization over model optimization
- The six-lever decomposition maps to pipeline harness components (prompt_style to research/analysis skill instructions, tool_policy to available tools per stage, memory_policy to prior reports retrieved, planning_policy to analysis depth, verification_policy to test/lint/LLM-judge level, max_steps to iterations per cycle)
- The static-beats-online finding is a caution against premature RL-ification -- the pipeline current approach (human-authored skill + LLM-generated findings + LLM-judged triage) may be near-optimal at its current episode budget
- The multi-objective reward (success + verifier score + compliance - unsupported-claim penalty - cost - latency) is a more complete model for pipeline triage criteria than the current three-signal model (quality, novelty, actionability)
Impact: High | Effort: Medium
Source
Key Finding
The Harness Control System paper formalizes the harness as a 729-configuration RL action space (six discrete levers: prompt_style, tool_policy, memory_policy, planning_policy, verification_policy, max_steps). The headline result: a DSPy-optimized static baseline matches or beats online bandit/REINFORCE controllers at practical episode budgets (25-60 episodes), because dense policies over large action spaces need Omega(M) samples. Even at 300 episodes (5-12x more), online controllers plateau below the static baseline.
Three Engineering Pitfalls to Adopt as Pipeline Hardening
Cold-start trap: uninitialized bandit argmax ties deterministically toward action 0 (which happened to be tool_policy=never -- a degenerate config). Fix: optimistic initialization and randomized tie-breaking. For Hermes: when initializing any new pipeline component, ensure the default configuration is not degenerate.
Crash-isolation gap: a single malformed episode took down 39 other episodes in the same job. Fix: per-episode try/except, scoring parse failures as normal failed episodes. For Hermes: wrap each pipeline stage in error handling that scores failures as normal failed cycles, not exceptions that propagate.
Measurement pitfall: DSPy LM disk cache returns empty usage dict on cache hits, silently under-counting cost. Fix: character-length fallback estimate. For Hermes: when using cached results (e.g., web search cache), account for the cost correctly.
Relevance to Hermes Evolution
Impact: High | Effort: Medium