Skip to content

st2 eval needs targeted supervision and runner-owned fault injection #81

Description

@schickling-assistant

Problem

st2 eval supervision is an all-team boolean. A bare supervise causes every absent team seat to be reconciled from spec on each wait tick.

That is too broad for a recovery eval which intentionally interrupts one worker:

  • a deterministic controller that exits is also eligible for relaunch, risking duplicate orchestration effects;
  • the eval cannot declare that only the interrupted worker should recover; and
  • cells fall back to invoking pty restart -y --force from inside another managed PTY.

The PTY nesting guard is intentional. Requiring --force here is a symptom that eval fault injection is using the wrong surface, not a PTY bug.

Current source makes the boundary explicit: Eval::supervise is a bool, and the wait tick reconciles the complete specs set.

Required behavior

Provide a runner-owned way for an eval to select which seats are supervised and to inject an intentional interruption into one selected seat.

The exact KDL is open, but the contract should ensure:

  • only declared selected seats are eligible for recovery;
  • a control/controller seat is not relaunched merely because another worker is under test;
  • recovery is respawn-from-spec with the complete effective environment;
  • the eval can prove a changed process incarnation for the selected seat; and
  • no nested PTY attach or --force guardrail bypass is required.

This is an eval-harness capability, not a change to production catalog reconciliation.

Related: #29, #30, #77, #82.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:evalst2 eval harness and fixtures · Set: manualorigin:agentFiled or primarily produced by an AI agent · Set: manualtype:featureNew user-visible or system capability · Set: manual

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions