Harness System Redesign Plan

Based on symphony-oc Architecture

Date: 2026-07-12

1. Executive Summary

This redesign adopts symphony-oc's proven orchestration patterns while maintaining Go as the implementation language. The new architecture introduces:

Multi-round review loop for quality assurance
Sophisticated retry logic with exponential backoff
Stall detection and orphan cleanup
Structured review output (JSON-based)
Improved state management with atomic persistence
New LLM model assignments:
- Planner: GLM-5.2 (strong reasoning)
- Executor: GLM-4.7 (cost-effective coding)
- Reviewer: DeepSeek-v4-flash (fast review)

2. Architecture Comparison

Current (harness-system)

Watcher → Worker Pool → Planner → Executor → Reviewer → Merger → Notifier
State Machine: Pending → Planning → Executing → Reviewing → Merging → Done
Storage: SQLite
Agent Interface: Direct Claude SDK calls

New (symphony-oc inspired)

Orchestrator → Issue Source → Worktree Pool → Agent Loop → Review Loop → Reconciler
State Machine: queued → running → reviewing → retrying → succeeded/failed
Storage: JSON with atomic writes
Agent Framework: Structured prompts with permission models

3. Key Design Changes

3.1 Orchestrator Pattern (New)

Reference: symphony_oc/orchestrator.py

The orchestrator becomes the central coordinator with these responsibilities:

type Orchestrator struct {
    cfg          *Config
    issueSources []IssueSource
    statePath    string
    lockPath     string
}

// Main loop handles:
// 1. Issue fetching from multiple sources
// 2. Dispatch decisions based on concurrency limits
// 3. Stall detection and timeout handling
// 4. Agent completion routing
// 5. Retry queue processing
// 6. Atomic state persistence

Key improvements:

File locking for safe concurrent state access
Stall detection with configurable timeout
Orphan cleanup on restart
Exponential backoff retry logic

3.2 Multi-Round Review Loop

Reference: symphony_oc/orchestrator.py:182-240

Decision table for review routing:

Verdict	Iteration	Action
PASS	< min_iterations	Re-review (consistency check)
PASS	≥ min_iterations	Reconcile (CI + PR)
FAIL	< max_iterations	Dispatch fixer
FAIL	≥ max_iterations	Mark failed

type ReviewConfig struct {
    MinIterations int  // PASS 也必须满 3 轮（一致性检查）
    MaxIterations int  // FAIL 超 5 轮放弃
}

type ReviewRecord struct {
    Iteration              int       `json:"iteration"`
    Verdict               string    `json:"verdict"`  // PASS | FAIL
    Timestamp             time.Time `json:"timestamp"`
    FilesAffected         []string  `json:"files_affected"`
    Summary               string    `json:"summary"`
    Feedback              []Issue   `json:"feedback"`
    ReviewerPID           int       `json:"reviewer_pid"`
    ReviewerStartedAt     time.Time `json:"reviewer_started_at"`
    ReviewerFinishedAt    time.Time `json:"reviewer_finished_at"`
}

3.3 Structured Review Output

Reference: symphony_oc/agents/symphony-reviewer.md:44-60

JSON schema for review output:

type ReviewResult struct {
    Verdict        string    `json:"verdict"`  // "PASS" | "FAIL"
    Iteration      int       `json:"iteration"`
    Timestamp      time.Time `json:"timestamp"`
    FilesAffected  []string  `json:"files_affected"`
    Summary        string    `json:"summary"`
    Feedback       []FeedbackItem `json:"feedback"`
}

type FeedbackItem struct {
    File      string `json:"file"`
    Line      int    `json:"line"`
    Severity  string `json:"severity"`  // critical | major | minor | style
    Issue     string `json:"issue"`
    Suggestion string `json:"suggestion"`
}

3.4 Agent Permission Model

Reference: symphony_oc/agents/symphony-worker.md and symphony-reviewer.md

Executor Agent (Write-capable)

bash:
  "go build *": allow
  "go vet *": allow
  "pytest *": allow
  "go test *": allow
  "make *": allow
  "gofmt *": allow
  "golangci-lint *": allow
  "git status": allow
  "git diff *": allow
  "git add *": allow
  "git commit *": allow
  "rm *": deny
  "git push *": deny
  "git reset --hard *": deny
  "git rebase *": deny
  "git checkout *": deny

Reviewer Agent (Read-only)

bash:
  "git status": allow
  "git diff": allow
  "git log": allow
  "git show": allow
  "*": deny
edit:
  ".san/review/*": allow
  "*": deny

4. New LLM Model Assignments

Model Selection Strategy

Agent	Model	Reasoning
Planner	GLM-5.2	Strong reasoning for complex planning
Executor	GLM-4.7	Cost-effective for high-volume code generation
Reviewer	DeepSeek-v4-flash	Fast review with good quality

Implementation

type AgentConfig struct {
    Planner AgentSpec `yaml:"planner"`
    Executor AgentSpec `yaml:"executor"`
    Reviewer AgentSpec `yaml:"reviewer"`
}

type AgentSpec struct {
    Backend string `yaml:"backend"`  // "glm" | "deepseek" | "claude"
    Model   string `yaml:"model"`
    APIKey  string `yaml:"api_key"`  // From env var
    BaseURL string `yaml:"base_url"`  // Optional override
}

// Example config:
agents:
  planner:
    backend: glm
    model: "glm-5.2"
    api_key_env: "GLM_API_KEY"
  executor:
    backend: glm
    model: "glm-4.7"
    api_key_env: "GLM_API_KEY"
  reviewer:
    backend: deepseek
    model: "deepseek-v4-flash"
    api_key_env: "DEEPSEEK_API_KEY"

5. State Management Redesign

Current State (SQLite-based)

// Current implementation uses SQLite with these tables:
// - tasks (id, issue_id, status, ...)
// - seen_issues (issue_id, ...)

New State (JSON-based with atomic writes)

Reference: symphony_oc/state.py:104-127

type RunState struct {
    IssueID       string          `json:"issue_id"`
    Title         string          `json:"title"`
    Branch        string          `json:"branch"`
    Worktree      string          `json:"worktree"`
    ContentHash   string          `json:"content_hash"`
    Status        string          `json:"status"`  // queued | running | reviewing | retrying | succeeded | failed
    Attempt       int             `json:"attempt"`
    PID           int             `json:"pid,omitempty"`
    Error         string          `json:"error,omitempty"`
    PRURL         string          `json:"pr_url,omitempty"`
    NextRetryAt   time.Time       `json:"next_retry_at,omitempty"`
    StartedAt     time.Time       `json:"started_at"`
    FinishedAt    time.Time       `json:"finished_at,omitempty"`
    ReviewCount   int             `json:"review_count"`
    ReviewPassed  bool            `json:"review_passed"`
    ReviewFeedback string         `json:"review_feedback,omitempty"`
    ReviewHistory []ReviewRecord `json:"review_history"`
}

// Atomic save with temp file + rename
func SaveRunsAtomic(path string, runs []RunState) error {
    // Write to temp file
    tmpPath := path + ".tmp"
    if err := writeJSON(tmpPath, runs); err != nil {
        return err
    }
    // Atomic rename
    return os.Rename(tmpPath, path)
}

6. Retry Logic Enhancement

Reference: symphony_oc/orchestrator.py:19-30, 56-80

Exponential Backoff

func RetryDelay(attempt int, backoffMs int, maxBackoffMs int) int {
    delay := backoffMs * int(math.Pow(2, float64(attempt-1)))
    if delay > maxBackoffMs {
        return maxBackoffMs
    }
    return delay
}

// Retry schedule:
// Attempt 1: 10s backoff
// Attempt 2: 20s backoff
// Attempt 3: 40s backoff
// Max: 60s cap

Stall Detection

func CheckStalls(runs []RunState, stallTimeoutMs int) {
    now := time.Now()
    for _, run := range runs {
        if run.Status != "running" {
            continue
        }
        elapsed := now.Sub(run.StartedAt).Milliseconds()
        if elapsed > int64(stallTimeoutMs) {
            InterruptProcess(run.PID)
            ScheduleRetry(&run, "stalled", RetryDelay(run.Attempt, 10000, 60000))
        }
    }
}

7. New Project Structure

github.com/wangke19/harness-system
├── cmd/harness/
│   └── main.go                 # Entry point
├── config/
│   ├── config.yaml            # Main configuration
│   └── config.go              # Config parsing
├── internal/
│   ├── orchestrator/          # NEW: Central coordinator
│   │   ├── orchestrator.go   # Main loop
│   │   ├── dispatcher.go     # Issue dispatch logic
│   │   ├── stall_detector.go # Stall detection
│   │   ├── retry_handler.go  # Retry logic
│   │   └── review_router.go  # Review decision table
│   ├── agent/
│   │   ├── agent.go          # Agent interface
│   │   ├── glm.go            # GLM SDK implementation
│   │   ├── deepseek.go       # DeepSeek SDK implementation
│   │   └── claude.go         # Claude SDK (optional)
│   ├── executor/
│   │   ├── executor.go      # Worktree creation + agent spawn
│   │   ├── prompt.go         # Prompt generation
│   │   └── permissions.go    # Permission validation
│   ├── reviewer/
│   │   ├── reviewer.go       # Review dispatch
│   │   ├── parser.go         # JSON result parsing
│   │   └── fixer.go          # Fix iteration handler
│   ├── reconciler/
│   │   └── reconciler.go    # CI + PR creation
│   ├── issue_source/
│   │   ├── source.go         # Interface
│   │   ├── local.go          # Local directory
│   │   └── github.go         # GitHub API
│   ├── state/
│   │   ├── state.go          # Run/Issue models
│   │   ├── persistence.go    # Atomic JSON I/O
│   │   └── retry.go          # Retry scheduling
│   └── utils/
│       ├── bash.go           # Shell execution
│       └── git.go            # Git operations
└── agents/                   # Agent prompts
    ├── planner.md
    ├── executor.md
    └── reviewer.md

8. Implementation Phases

Phase 1: Foundation (Week 1)

Set up new project structure
Implement state management (JSON-based)
Create agent interfaces for GLM and DeepSeek
Implement orchestrator skeleton

Phase 2: Core Flow (Week 2)

Implement issue source (GitHub + local)
Build dispatcher with worktree creation
Create agent prompts (planner, executor, reviewer)
Implement basic execution flow

Phase 3: Review Loop (Week 3)

Implement structured review output
Build review router with decision table
Add fix iteration logic
Implement reconciler (CI + PR)

Phase 4: Reliability (Week 4)

Add stall detection
Implement retry logic with backoff
Add orphan cleanup
Comprehensive error handling

Phase 5: Polish (Week 5)

Add monitoring and logging
Create configuration validation
Write tests
Documentation

9. Configuration Example

# config/config.yaml
server:
  poll_interval_ms: 30000
  max_concurrent: 3
  stall_timeout_ms: 1800000
  db_path: ./state/runs.jsonc

agents:
  planner:
    backend: glm
    model: "glm-5.2"
    api_key_env: "GLM_API_KEY"
    base_url: "https://open.bigmodel.cn/api/paas/v4/"
  executor:
    backend: glm
    model: "glm-4.7"
    api_key_env: "GLM_API_KEY"
    base_url: "https://open.bigmodel.cn/api/paas/v4/"
    max_retries: 3
    command_timeout_ms: 600000
    extra_args: ["--pure"]
  reviewer:
    backend: deepseek
    model: "deepseek-v4-flash"
    api_key_env: "DEEPSEEK_API_KEY"
    base_url: "https://api.deepseek.com/v1"
    min_iterations: 3
    max_iterations: 5

git:
  remote: "upstream"
  base_branch: "upstream/main"
  worktree_root: "./worktrees"
  two_commit_pr: true
  two_commit_exclude:
    - "*.lock"
    - "go.sum"
    - "vendor/"
    - "*.generated.*"

ci:
  command: "go test ./..."
  timeout_ms: 120000

tracker:
  local_dir: "./issues"
  github:
    repo: "owner/repo"
    labels: ["harness-ready"]
    active_states: ["open"]

10. Migration Strategy

Data Migration (SQLite → JSON)

// Migration utility
func MigrateSQLiteToJSON(sqlitePath, jsonPath string) error {
    // 1. Load from SQLite
    tasks := LoadFromSQLite(sqlitePath)
    
    // 2. Convert to RunState format
    runs := make([]RunState, len(tasks))
    for i, task := range tasks {
        runs[i] = ConvertTaskToRun(task)
    }
    
    // 3. Save atomically
    return SaveRunsAtomic(jsonPath, runs)
}

Backward Compatibility

Keep SQLite reader for transition period
Support both config formats during migration
Provide migration CLI tool

11. Testing Strategy

Unit Tests

State persistence (atomic writes)
Retry logic (backoff calculation)
Review routing (decision table)
Permission validation

Integration Tests

Full orchestration flow
Multi-round review loop
Stall detection and recovery
GitHub integration

E2E Tests

Local issue source
Mock agent responses
CI simulation

12. Success Metrics

Metric	Current	Target
Review accuracy	Single pass	3-pass consistency
Retry efficiency	Fixed 3x	Exponential backoff
Stall recovery	Manual	Auto-detect + retry
State corruption risk	Medium	Low (atomic writes)
Agent cost	Opus for all	Model-specific optimization

13. Open Questions

GLM SDK: Which Go SDK to use for GLM API access?
DeepSeek SDK: Need to evaluate available Go SDKs
Migration timing: Big bang vs. gradual migration?
Backward compatibility: How long to support old config format?

14. Next Steps

✅ Review and approve this redesign plan
⏳ Set up GLM and DeepSeek API access
⏳ Create new project structure
⏳ Implement agent interfaces
⏳ Begin Phase 1 implementation

References:

symphony-oc project: ~/src/github.com/wangke19/symphony-oc
GLM API docs: https://open.bigmodel.cn/dev/api
DeepSeek API docs: https://platform.deepseek.com/api-docs/

FilesExpand file tree

REDESIGN_PLAN.md

Latest commit

History