Skip to content

Add behavioral tests for supervisor recovery tools (supplement source-based tests) #301

Description

@HenryLach

Context

TP-076/077/078 added supervisor recovery tools (orch_retry_task, orch_skip_task, orch_force_merge) with source-based tests that verify code patterns exist. An external code review identified that these tests don't validate end-to-end behavior — they check for string presence in source, not actual state mutations from running the tool handlers.

This is why several bugs (#1-5 in the review) passed the test suite.

What's needed

Add behavioral tests that:

  1. Call the actual tool handler functions (or close equivalents) with mock batch state
  2. Assert the resulting state mutations (counter changes, status transitions, blocked set updates)
  3. For orch_force_merge: verify that resume after force-merge actually triggers a git merge (integration test level)

Files affected

  • extensions/tests/supervisor-recovery-tools.test.ts — add behavioral tests alongside existing source-based tests
  • extensions/tests/supervisor-force-merge.test.ts — add behavioral merge test
  • extensions/tests/supervisor-alerts.test.ts — add behavioral alert delivery test

Priority

Medium — the source-based tests provide coverage for code existence, but behavioral tests are needed to catch logic bugs like stale references and missing recomputations.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions