sandbox: scaffold Vertex profile lacks **/claude.exe, so every 0.40.0-image Claude run fails with policy_denied

What happened

Every Claude run on the v0.40.0 sandbox image (Claude Code 2.1.258) that loads the Vertex profile shipped in this repo fails on its first call with API Error: Error code policy_denied, 0 tokens, model <synthetic>, ~2 s per iteration. This is what breaks functional-tests on main (#6962) and what users see on local fullsend run invocations whose --fullsend-dir carries a profiles/ directory.

The gateway log in the eval-results artifact of https://github.com/fullsend-ai/fullsend/actions/runs/33763731846 shows the real denial:

NET:OPEN [MED] DENIED /usr/lib/node_modules/@anthropic-ai/claude-code/bin/claude.exe(565) -> sts.googleapis.com:443
[policy:- engine:opa] [reason:binary '/usr/lib/node_modules/@anthropic-ai/claude-code/bin/claude.exe' not allowed in policy '_provider_vertex_ai' ...]

It is not a model-authorization or Model Armor problem (the triage hypothesis on #6962 and the retro theory in #6967 are both wrong, same misdiagnosis that fullsend-ai/agents#1117 went through). The agent never reaches STS, so Vertex never evaluates a model.

Root cause

Claude Code 2.1.2xx is a Bun-compiled binary named claude.exe even on Linux. Since fix(#6612): make the pinned Claude Code the one that runs in the sandbox #6647 (v0.38.0+) the CLAUDE_CODE_VERSION-pinned npm install is the binary that runs in the sandbox instead of the OpenShell base image's 2.1.156 /usr/local/bin/claude, and the v0.40.0 image pins 2.1.258.
internal/scaffold/fullsend-repo/profiles/fullsend-vertex-ai.yaml still allowlists only **/claude and **/node. The fleet copy in fullsend-ai/agents was fixed by fix(#1117): allow claude.exe binary in Vertex AI profile agents#1118 (ba1d46f, adds **/claude.exe; that copy also carries **/pi), but this repo's scaffold copy was never updated.
fullsend run imports the harness-resolved (base: URL) profiles first (internal/cli/run.go:1279, sandbox.ImportProfile) and then imports <fullsend-dir>/profiles/ (internal/cli/run.go:1290, sandbox.ImportProfiles). ImportProfiles deletes every profile ID found in the directory and re-imports the directory copy, so a stale directory copy silently replaces the fixed profile that was just fetched. Nothing warns: checkProviderProfileIntegrity only checks that provider types resolve to some profile, and dedupResolvedProfiles only dedups URL-resolved entries.

The functional-tests case stdout shows both steps back to back:

    Base: https://raw.githubusercontent.com/fullsend-ai/agents/10560252b0ad.../profiles/fullsend-vertex-ai.yaml (fetched)
  ...
  • Importing profile: fullsend-vertex-ai
  ✓ Profile imported: fullsend-vertex-ai (0.0s)
  • Importing provider profiles
  ✓ Provider profiles imported (0.1s)

make functional-tests runs with FULLSEND_DIR=internal/scaffold/fullsend-repo, so the unfixed scaffold copy wins every time. The fixed agents profile has not been exercised by any fleet run yet either: every status comment since the fullsend-ai/agents#1160 repin still shows opus → claude-opus-4-6 (Claude Code 2.1.156 on the old image), and this repo's own .fullsend pins fullsend-ai/agents@81b0e9bde0fa.

Proposed change

Add "**/claude.exe" and "**/pi" to binaries: in internal/scaffold/fullsend-repo/profiles/fullsend-vertex-ai.yaml, matching profiles/fullsend-vertex-ai.yaml in fullsend-ai/agents at ba1d46f. Add a unit test that fails if the scaffold Vertex profile's binaries list is missing **/claude or **/claude.exe, so the two copies cannot drift apart on this again.
In internal/cli/run.go, when a profile ID in <fullsend-dir>/profiles/ matches a harness-resolved profile that was just imported, emit a StepWarn naming the ID and both sources ("directory copy replaces harness-resolved profile"). No behaviour change beyond the warning; the override is what per-repo customization relies on, but it must be visible.

Keep the change out of .github/workflows/ (the coder app has no workflows permission).

Validation criteria

workflow_dispatch of functional-tests.yml on the PR branch runs the four triage cases on the 0.40.0 image (Agent: claude-opus-5 (v2.1.258)) and they pass with non-zero tokens. This is also the first live verification that the **/claude.exe glob is accepted by OpenShell's OPA matcher (glob.match(b.path, ["/"], p) in sandbox-policy.rego).
The new unit test passes and fails when **/claude.exe is removed from the scaffold profile.
A run whose <fullsend-dir>/profiles/ shadows a harness-resolved profile prints the warning; a run without such a directory prints nothing new.

Fixes #6962.

🤖 Finished Triage · ✅ Success · Started 4:28 PM UTC · Completed 4:32 PM UTC

Commit: 48eac1a · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $0.64

This is a thorough root cause analysis — the diagnosis is confirmed by the gateway deny log and the binary name change in Claude Code 2.1.2xx.

Related work:

Fixes functional-tests: every triage case fails with API Error policy_denied on main since 2026-09-03 #6962 (the symptom issue reporting functional-test failures)
The agents-repo copy was already fixed in fix(#1117): allow claude.exe binary in Vertex AI profile agents#1118 (ba1d46f, merged 2026-09-02), but this repo's scaffold copy was never synced
Per-repo installs: non-OpenAI sandbox profiles not imported from embedded scaffold #6929 tracks a broader issue with scaffold profile imports for non-OpenAI profiles — independent but worth cross-referencing
Review harness should fallback to previous model version on policy_denied #6967 proposed a model-version fallback for policy_denied — that issue's premise is based on a misdiagnosis (this is a binary allowlist issue, not a model authorization issue)

Proposed test case:

func TestScaffoldVertexProfile_BinaryAllowlist(t *testing.T) {
    // Load the scaffold Vertex profile
    data, err := os.ReadFile("internal/scaffold/fullsend-repo/profiles/fullsend-vertex-ai.yaml")
    require.NoError(t, err)

    var profile struct {
        Binaries []string `yaml:"binaries"`
    }
    require.NoError(t, yaml.Unmarshal(data, &profile))

    assert.Contains(t, profile.Binaries, "**/claude",
        "scaffold Vertex profile must allowlist **/claude")
    assert.Contains(t, profile.Binaries, "**/claude.exe",
        "scaffold Vertex profile must allowlist **/claude.exe (Bun-compiled binary)")
}

Labels: Critical priority: breaks all Claude runs on the 0.40.0 sandbox image and functional-tests on main; adding component/harness because the fix also touches profile import logic in internal/cli/run.go.

Next steps:

/fs-code — agent creates a PR to implement this issue
/fs-code <your instruction> — agent implements with your specific guidance

Confirmed live on OpenShell 0.0.116 with the 0.40.0 fullsend-code image (Claude Code 2.1.258): with **/claude.exe on the fullsend-vertex-ai profile, a review run against #6603 that had failed three times with this exact policy_denied went through — 47 tool calls, validation passed — so the OPA glob accepts the .exe form.

One gotcha for validating on a long-lived local gateway: ImportProfiles refreshes the workspace-scoped copy, but _provider_vertex_ai is built from the platform-scoped copy, which imports never touch, so the scaffold fix alone leaves an existing gateway denying. It took openshell provider profile export --global fullsend-vertex-ai > f.yaml, edit, openshell provider profile update --global --file f.yaml fullsend-vertex-ai (the update needs the export's resource_version). Fresh CI gateways won't see this.

/fs-code Do not add a Signed-off-by trailer to any commit (agent commits are DCO-exempt and the post-code guard rejects sign-offs). Do not touch anything under .github/workflows/.

Implement both parts of the issue as two conventional commits:

fix(scaffold): allow claude.exe and pi on the Vertex profile — add "**/claude.exe" and "**/pi" to binaries: in internal/scaffold/fullsend-repo/profiles/fullsend-vertex-ai.yaml (same list as fullsend-ai/agents profiles/fullsend-vertex-ai.yaml at ba1d46f), plus a unit test in internal/scaffold/ that reads the embedded scaffold file via the scaffold package (not a hard-coded repo-relative path) and fails if **/claude or **/claude.exe is missing.
fix(run): warn when a profiles/ directory copy shadows a harness-resolved profile — in internal/cli/run.go after the URL-resolved sandbox.ImportProfile loop and before sandbox.ImportProfiles(profilesDir), compare dirProfileIDs against result.Profiles and printer.StepWarn one line per shadowed ID naming the ID, the harness source, and the directory path. Warning only, no behaviour change, with a unit test on the pure helper that computes the shadowed set.

Run make lint and go test ./internal/scaffold/... ./internal/cli/... before pushing.

🤖 Code · ⚠️ Cancelled · Ended 4:34 PM UTC

Commit: 48eac1a · View workflow run →

fix(scaffold): allow claude.exe and pi on the Vertex profile

fix(run): warn when a profiles/ directory copy shadows a harness-reso…

fix(scaffold): allow claude.exe and pi on the Vertex profile

fix(run): warn when a profiles/ directory copy shadows a harness-reso…

test(profiles): pin the egress binary each runtime needs on its profile

fix(scaffold): allow claude.exe and pi on the Vertex profile

fix(run): warn when a profiles/ directory copy shadows a harness-reso…

test(profiles): pin the egress binary each runtime needs on its profile

Merge pull request fullsend-ai#6975 from fullsend-ai/agent/6971-scaff…

fix(#7095): skip unlisted directory profiles during import

fix(#7014): add GraphQL endpoint to scaffold GitHub RO profile

feat(#7268): resolve every builtin provider from the embedded scaffold

Navigation Menu

Repository navigation

sandbox: scaffold Vertex profile lacks **/claude.exe, so every 0.40.0-image Claude run fails with policy_denied #6971

What happened

Root cause

Proposed change

Validation criteria

2 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

sandbox: scaffold Vertex profile lacks **/claude.exe, so every 0.40.0-image Claude run fails with policy_denied #6971

Description

What happened

Root cause

Proposed change

Validation criteria

Activity

fullsend-ai-triage commented on Sep 3, 2026

fullsend-ai-triage commented on Sep 3, 2026

dhshah13 commented on Sep 3, 2026

waynesun09 commented on Sep 3, 2026

fullsend-ai-coder commented on Sep 3, 2026

2 remaining items

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions