[IMPROVEMENT] WML: Node-Mechanism Attribution for structured skill debugging — identify failed workflow node, implicated mechanisms, smallest valid edit target

[Lexus2016]

Source

• Paper: https://arxiv.org/abs/2607.20999 (Zibin Lin, Shengli Zhang, Jul 23, 2026)

Problem

Hermes evolution pipeline does not attribute skill failures to specific workflow nodes and mechanisms. When a pipeline skill (e.g., the research skill) produces poor output, the pipeline does not diagnose which step of the skill failed (the search? the filter? the dedup? the report writing?) or which mechanism caused the failure.

Proposed Feature

Implement Node-Mechanism Attribution for structured skill debugging:

1. Failed workflow node identification -- which step in the skill workflow produced the failure
2. Implicated mechanisms -- which specific mechanisms within that node caused the failure
3. Smallest valid edit target -- the minimal change that would fix the failure

Repair Routing

• Single-mechanism defects routed to L3 resources (direct mechanism repair via third-party knowledge)
• Relational defects across mechanisms routed to L2 composition protocols (the defect is in how mechanisms interact, not in any single mechanism)

Workflow-Guided Skill Optimization (WGSO) Loop

1. Select provenance- and scope-aware third-party knowledge (reuse knowledge from other skills that addresses the same mechanism)
2. Apply bounded patches (the edit is scoped to the identified node/mechanism, not the entire skill)
3. Evaluate candidates
4. Store verified outcomes in optimizer-side memory

Results from the Paper

• SpreadsheetBench: 90.33 Hard Accuracy (DeepSeek), 74.67 (Qwen3.6-Flash)
• Transfer to WikiTableQuestions without additional optimization: 84.00 / 83.00 Denotation Accuracy
• Compiled execution sharply reduces tokens and calls relative to direct SkillAgent

Relevance to Hermes Evolution

• Attribute skill failures to workflow nodes and mechanisms -- when the research skill produces poor findings, diagnose: was it the search step, the filter step, the dedup step, or the report-writing step?
• Route repairs by defect type -- a single-mechanism defect (e.g., wrong search query) is repaired by direct knowledge; a relational defect (e.g., dedup does not account for what filter already removed) is repaired by changing the composition protocol
• Reuse third-party knowledge with provenance and scope tracking

Impact: Medium | Effort: Medium