Skip to content

refactor(harness): migrate code agent to env.runner/env.sandbox (ADR 0055) #2760

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ralphbean wants to merge 2 commits into
base: main
Choose a base branch
from
+35 −53
Open

refactor(harness): migrate code agent to env.runner/env.sandbox (ADR 0055) #2760

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
496b821
refactor(harness): migrate code agent to env.runner/env.sandbox (ADR …
ralphbean Jun 29, 2026
f47cb9f
fix(test): update scaffold_test.go for env.runner migration
ralphbean Jun 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions internal/harness/scaffold_integration_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -287,9 +287,9 @@ func TestLoadRaw_GeneratedWrapperFormat(t *testing.T) {
}

// TestResolveForge_ScaffoldRunnerEnvMerge verifies that forge resolution
// produces the expected merged runner_env for each scaffold template, with
// both top-level (platform-neutral) and forge.github (platform-specific)
// keys present in the final merged state.
// produces the expected merged runner_env / env.runner for each scaffold
// template, with both top-level (platform-neutral) and forge.github
// (platform-specific) keys present in the final merged state.
func TestResolveForge_ScaffoldRunnerEnvMerge(t *testing.T) {

@fullsend-ai-review fullsend-ai-review Bot Jun 30, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] test-coverage-gap

TestResolveForge_ScaffoldRunnerEnvMerge validates only runner-side env keys after forge resolution. The diff adds eleven sandbox variables but no test asserts that h.Env.Sandbox contains the expected keys after resolution. A regression in sandbox env merging would go undetected for the code agent.

Suggested fix: Add a sandboxKeys field to the code.yaml test case and assert that h.Env.Sandbox contains the expected keys after forge resolution.

dir := t.TempDir()
harnessDir := extractScaffoldHarnessDir(t, dir)
Expand Down
45 changes: 6 additions & 39 deletions internal/scaffold/fullsend-repo/env/code-agent.env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,42 +1,9 @@
export ISSUE_NUMBER=${ISSUE_NUMBER}
export GITHUB_ISSUE_URL=${GITHUB_ISSUE_URL}

# GH_TOKEN in the sandbox is a READ-ONLY scoped app installation token
# (contents:read, issues:read, pull_requests:read). Set by
# setup-agent-env.sh from CODE_GH_TOKEN. This token CANNOT push code
# or create PRs even if the agent bypasses disallowedTools.
# The separate write-enabled PUSH_TOKEN (runner_env) never enters the sandbox.
export GH_TOKEN=${GH_TOKEN}

# Git identity — uses the GitHub App bot user's noreply email so GitHub
# links commits to the bot account (author.type === "Bot"). This makes
# the Probot DCO app auto-exempt agent commits. The GIT_BOT_EMAIL var
# is resolved at runtime by the "Resolve bot identity" workflow step.
export GIT_AUTHOR_NAME="fullsend-code"
export GIT_AUTHOR_EMAIL="${GIT_BOT_EMAIL}"
export GIT_COMMITTER_NAME="fullsend-code"
export GIT_COMMITTER_EMAIL="${GIT_BOT_EMAIL}"

# Retry budget — the agent re-runs secret scan + tests on failure.
# Pre-commit is capped at 2 runs total (not per retry) and is NOT
# re-run during retries. The post-script runs authoritative pre-commit.
export MAX_RETRIES=1

# Hard timeout for the sandbox session in seconds. The agent uses this to
# check remaining time and avoid burning the budget on retries. Must match
# timeout_minutes in harness/code.yaml. Update this if you change the
# harness timeout.
export TIMEOUT_SECONDS=2100

# Go toolchain — PATH, GOPATH, and GOMODCACHE are set in the sandbox image
# via Containerfile ENV (images/code/Containerfile line 64). Do NOT set PATH
# here: this file uses expand: true (harness host_files), so ${PATH} would be
# replaced with the GitHub Actions runner's PATH by os.ExpandEnv, clobbering
# /sandbox/workspace/bin and breaking fullsend scan + any other sandbox-local
# binaries. If OpenShell does not preserve Docker ENV, add Go to PATH via a
# non-expanded host_file or bootstrap step instead of referencing ${PATH} here.
export GOPATH="/sandbox/go"
export GOMODCACHE="/sandbox/go/pkg/mod"
# code-agent.env — shell conditional that cannot be expressed in env.sandbox.
#
# Simple passthrough vars and hardcoded config have been moved to
# harness/code.yaml under env.runner and env.sandbox (ADR 0055).
# Only the GIT_SSL_CAINFO conditional remains here because env.sandbox
# does not support shell logic.

# SSL certs — git 2.43 (Ubuntu Noble) is linked against libcurl-gnutls
# which does NOT read SSL_CERT_FILE. It requires GIT_SSL_CAINFO explicitly.
Expand Down
35 changes: 25 additions & 10 deletions internal/scaffold/fullsend-repo/harness/code.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,20 +41,35 @@ plugins:

# Environment variables available to pre/post scripts on the runner.
# These are expanded from the runner environment and NEVER enter the sandbox.
runner_env:
CODE_ALLOWED_TARGET_BRANCHES: "${CODE_ALLOWED_TARGET_BRANCHES}"
FULLSEND_OUTPUT_SCHEMA: ${FULLSEND_DIR}/schemas/code-result.schema.json
FULLSEND_OUTPUT_FILE: code-result.json
env:
runner:
CODE_ALLOWED_TARGET_BRANCHES: "${CODE_ALLOWED_TARGET_BRANCHES}"
FULLSEND_OUTPUT_SCHEMA: ${FULLSEND_DIR}/schemas/code-result.schema.json

@fullsend-ai-review fullsend-ai-review Bot Jun 29, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[low] lint-diagnostic-expected

code.yaml now declares env.sandbox alongside host_files delivering code-agent.env. Both inject into sandbox but the .env file only contains the GIT_SSL_CAINFO shell conditional that env.sandbox cannot express.

FULLSEND_OUTPUT_FILE: code-result.json
sandbox:
MAX_RETRIES: "1"
TIMEOUT_SECONDS: "2100"
GOPATH: "/sandbox/go"
GOMODCACHE: "/sandbox/go/pkg/mod"
Comment thread
qodo-code-review[bot] marked this conversation as resolved.

timeout_minutes: 35

forge:
github:
pre_script: scripts/pre-code.sh
post_script: scripts/post-code.sh
runner_env:
PUSH_TOKEN: "${PUSH_TOKEN}"
PUSH_TOKEN_SOURCE: "${PUSH_TOKEN_SOURCE}"
REPO_FULL_NAME: "${REPO_FULL_NAME}"
ISSUE_NUMBER: "${ISSUE_NUMBER}"
REPO_DIR: "${GITHUB_WORKSPACE}/target-repo"
env:
runner:
PUSH_TOKEN: "${PUSH_TOKEN}"
PUSH_TOKEN_SOURCE: "${PUSH_TOKEN_SOURCE}"
REPO_FULL_NAME: "${REPO_FULL_NAME}"
ISSUE_NUMBER: "${ISSUE_NUMBER}"
REPO_DIR: "${GITHUB_WORKSPACE}/target-repo"
sandbox:
ISSUE_NUMBER: "${ISSUE_NUMBER}"
GITHUB_ISSUE_URL: "${GITHUB_ISSUE_URL}"
GH_TOKEN: "${GH_TOKEN}"

@rh-hemartin rh-hemartin Jul 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is correct. If this gets expanded within the sandbox, there is no GH_TOKEN within the sandbox right? That is why GH_TOKEN was on a file expanded at runner level. Now I'm not sure if other similar PRs to this one are OK. Same for GIT_BOT_EMAIL. I don't think you are testing this locally and you should.

GIT_AUTHOR_NAME: "fullsend-code"
GIT_AUTHOR_EMAIL: "${GIT_BOT_EMAIL}"
GIT_COMMITTER_NAME: "fullsend-code"
GIT_COMMITTER_EMAIL: "${GIT_BOT_EMAIL}"
2 changes: 1 addition & 1 deletion internal/scaffold/scaffold_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -530,7 +530,7 @@ func TestCodeHarnessContent(t *testing.T) {
assert.Contains(t, s, "agents/code.md")
assert.Contains(t, s, "pre_script")
assert.Contains(t, s, "post_script")
assert.Contains(t, s, "runner_env")
assert.Contains(t, s, "env:")
assert.Contains(t, s, "PUSH_TOKEN")
}

Expand Down
Loading