Skip to content

eval: cover managed agent color environment - #63

Merged
myobie merged 8 commits into
compoundingtech:mainfrom
schickling-assistant:schickling-assistant/2026-07-31-pty-fractal-composition-evals
Aug 1, 2026
Merged

eval: cover managed agent color environment#63
myobie merged 8 commits into
compoundingtech:mainfrom
schickling-assistant:schickling-assistant/2026-07-31-pty-fractal-composition-evals

Conversation

@schickling-assistant

@schickling-assistant schickling-assistant commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Why

A managed coding-agent PTY should remain color-capable when both the st2 reconciler and a later operator restart run under ambient NO_COLOR=1. An explicit Agent Spec NO_COLOR value must remain authoritative.

What

This adds a model-free composition cell covering the managed-agent environment contract through real st2 up reconciliation, real transient user scopes, PTY metadata, and standalone pty restart. The living specification defines the composed runtime boundary and traces it to R04-R05.

The cell proves ambient removal, explicit-value precedence, real scope isolation, live adoption, st2-owned replacement, standalone restart durability, and complete cleanup.

Rationale

The observation happens inside each launched task, after the complete st2 isolation/spawn path. The explicit-value leg is a negative control against unconditionally deleting NO_COLOR.

The restart leg invokes NO_COLOR=1 pty restart -y --force against the default managed session. This distinguishes a temporary launch-time scrub from the durable contract: st2 must persist an unset tombstone and PTY must apply removals after inherited values and before explicit assignments. Ordinary non-agent PTYs without an authored unset policy retain ambient inheritance.

Dependencies

  • evals main fa2d6336e12085057bdec29671df5fae870dae5f, including merged packaged machine-stream composition from eval: cover packaged pty attach stream #64
  • merged PTY d5fabc3917407aeb937a012bd97679c303e18033
  • merged st2 #132 identity 8ca0f3653aae6ca6ebcb880f82e9296016df6e08

Verification

At exact PR head 9a38e04f07ddb46de867f3fde1a36fed1cf6c398 with the merged dependency identities above:

  • composed runtime: SCORE: 11 PASS / 0 FAIL / 11 gating judges (two run steps plus nine held-out judges)
  • merged-source st2 artifact SHA-256: e193141523bb35a397f90170f6fc182e663d205eef2d95af27a94dcae806cc48
  • PTY artifact SHA-256: 1c9716d435ca56ad9b4f67056d76fa6856cdc08e6bbda1fd4be6f59952e9fde3
  • accepted old-st2 control SHA-256: d61d12b2b1189a391c196ca28f8f4ba69072d14fcbad2571fc29db1f250f4eed
  • 225 shell files parse
  • preflight reachability, model policy and mutations, run history, retired surfaces, and event-first hooks pass
  • st2 semantic loading: 59 maintained folder evals and 12 nested fleet catalogs
  • VRS scope/variation mutations, closed-set mutations, catalog generation, and diff checks pass
  • executable dependency controls discriminate ownership: accepted old st2 0fed14b with current PTY exposes ambient NO_COLOR at initial launch; merged st2 with a labeled PTY argument mutation that strips --unset-env passes initial launch but reintroduces ambient NO_COLOR on restart
  • cleanup proves an empty PTY registry, termination of every observed PID/start-time identity, and no active captured st2 transient scope or surviving exact live-observed scope cgroup path before disabling its failure trap

Incident #1350 constrained this refresh to the targeted composition and static/VRS surface; it did not start a package build, model, or full corpus run. The exact merged-source st2 binary was relinked with cargo build --bin st2 using the existing hot target cache.

Protected VRS approval request

docs/vrs/spec.md maps both composed runtime cells to the existing R04/R05 boundary. docs/vrs/requirements.md remains unchanged pending Nathan's explicit approval or revision of these clarifications:

  • R04: for a composed dependency contract, mutation evidence includes an older or deliberately broken dependency combination.
  • R05: composed runtime cells pin and report the exact artifact identity of every participating component.

Per Johannes, this visible approval request does not hold an otherwise implementation-ready PR. No vision change is proposed.

Posted on behalf of @schickling
field value
agent_name co1-sage
agent_session_id ed878dac-3735-4276-b3e0-ea1b1cd65291
agent_tool Codex CLI
agent_tool_version 0.145.0
agent_runtime Codex CLI 0.145.0
agent_model unknown
runtime_profile /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
skills_manifest /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
worktree evals/schickling-assistant/2026-07-31-schickling-assistant-2026-07-31-pty-fractal-composition-evals
machine dev3
tooling_profile dotfiles@0fb7e03

@schickling-assistant
schickling-assistant force-pushed the schickling-assistant/2026-07-31-pty-fractal-composition-evals branch from 2db74e9 to a71e19d Compare July 31, 2026 12:16
@schickling-assistant

Copy link
Copy Markdown
Contributor Author

Nathan, Johannes explicitly asked that the durable VRS—including requirements—cover these composition evals. docs/vrs/spec.md says requirements changes need your explicit approval.

May I clarify the existing requirements as follows?

  • R04: for a composed dependency contract, mutation evidence includes an older or deliberately broken dependency combination.
  • R05: composed runtime cells pin and report the exact artifact identity of every participating component.

The spec would then map managed-agent-color-env and pty-attach-machine-stream to R04/R05, including their old-dependency negative controls. No vision change.

Posted on behalf of @schickling
field value
agent_name co1-sage
agent_session_id ed878dac-3735-4276-b3e0-ea1b1cd65291
agent_tool Codex CLI
agent_tool_version 0.145.0
agent_runtime Codex CLI 0.145.0
agent_model unknown
runtime_profile /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
skills_manifest /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
worktree evals/schickling-assistant/2026-07-31-pty-fractal-composition-evals
machine dev3
tooling_profile dotfiles@0fb7e03

@schickling-assistant
schickling-assistant force-pushed the schickling-assistant/2026-07-31-pty-fractal-composition-evals branch from a71e19d to 0fafa94 Compare July 31, 2026 12:49
agent-session-id: ed878dac-3735-4276-b3e0-ea1b1cd65291
agent-tool: Codex CLI
agent-tool-version: 0.145.0
agent-model: unknown
agent-runtime-profile: /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
agent-skills-manifest: /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@0fb7e03
agent-session-id: ed878dac-3735-4276-b3e0-ea1b1cd65291
agent-tool: Codex CLI
agent-tool-version: 0.145.0
agent-model: unknown
agent-runtime-profile: /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
agent-skills-manifest: /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@0fb7e03
agent-session-id: ed878dac-3735-4276-b3e0-ea1b1cd65291
agent-tool: Codex CLI
agent-tool-version: 0.145.0
agent-model: unknown
agent-runtime-profile: /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
agent-skills-manifest: /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@0fb7e03
agent-session-id: ed878dac-3735-4276-b3e0-ea1b1cd65291
agent-tool: Codex CLI
agent-tool-version: 0.145.0
agent-model: unknown
agent-runtime-profile: /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
agent-skills-manifest: /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@0fb7e03
agent-session-id: ed878dac-3735-4276-b3e0-ea1b1cd65291
agent-tool: Codex CLI
agent-tool-version: 0.145.0
agent-model: unknown
agent-runtime-profile: /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
agent-skills-manifest: /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@0fb7e03
agent-session-id: ed878dac-3735-4276-b3e0-ea1b1cd65291
agent-tool: Codex CLI
agent-tool-version: 0.145.0
agent-model: unknown
agent-runtime-profile: /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
agent-skills-manifest: /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@0fb7e03
@schickling-assistant
schickling-assistant force-pushed the schickling-assistant/2026-07-31-pty-fractal-composition-evals branch from 0fafa94 to 9a8bd2e Compare July 31, 2026 19:45
agent-session-id: ed878dac-3735-4276-b3e0-ea1b1cd65291
agent-tool: Codex CLI
agent-tool-version: 0.145.0
agent-model: unknown
agent-runtime-profile: /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
agent-skills-manifest: /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@0fb7e03
agent-session-id: ed878dac-3735-4276-b3e0-ea1b1cd65291
agent-tool: Codex CLI
agent-tool-version: 0.145.0
agent-model: unknown
agent-runtime-profile: /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
agent-skills-manifest: /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@0fb7e03
@schickling-assistant
schickling-assistant marked this pull request as ready for review July 31, 2026 20:18
agent-session-id: ed878dac-3735-4276-b3e0-ea1b1cd65291
agent-tool: Codex CLI
agent-tool-version: 0.145.0
agent-model: unknown
agent-runtime-profile: /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
agent-skills-manifest: /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
tooling-profile: dotfiles@0fb7e03
@schickling-assistant

Copy link
Copy Markdown
Contributor Author

Final merged-dependency receipt at exact head 9a38e04f07ddb46de867f3fde1a36fed1cf6c398:

  • base: merged evals eval: cover packaged pty attach stream #64, fa2d6336e12085057bdec29671df5fae870dae5f
  • st2 merged source: 8ca0f3653aae6ca6ebcb880f82e9296016df6e08
  • st2 binary SHA-256: e193141523bb35a397f90170f6fc182e663d205eef2d95af27a94dcae806cc48
  • exact committed-head composition: 11/11
  • executable identity checks, old-st2 launch-negative control, stripped-tombstone PTY restart-negative control, PID-generation death, captured cgroup disappearance, and cleanup: green
  • targeted static/VRS gates and independent merged-identity delta review: green/clear
  • ready, CLEAN, MERGEABLE, zero threads; this repository has no hosted workflow run for the PR

R04/R05 wording remains visibly proposed and unapproved; requirements.md is unchanged. This is ready for your review without implying requirements approval.

Posted on behalf of @schickling
field value
agent_name co1-sage
agent_session_id ed878dac-3735-4276-b3e0-ea1b1cd65291
agent_tool Codex CLI
agent_tool_version 0.145.0
agent_runtime Codex CLI 0.145.0
agent_model unknown
runtime_profile /nix/store/mnx8agbdq3wiyb6vz63lhgscgazkrn98-coding-agent-runtime-profile/share/coding-agents/profile.json
skills_manifest /nix/store/5r69m9k2llmri3na81518zx0a7y0d3cn-agent-skills-corpus/share/agent-skills/manifest.json
worktree dotfiles/schickling/2026-07-20-cos-misc
machine dev3
tooling_profile dotfiles@0fb7e03

@myobie
myobie merged commit a5d9f84 into compoundingtech:main Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants