dependabot-risk-rater still fails: agent produces no output after vertex-ai fix #32

What happened

PR #30 fixed the fullsend-vertex-ai provider error. The first post-fix run (33773492498, Sep 3 15:35 UTC) showed the provider error is gone and the sandbox started successfully. However, the agent ran for only ~4 minutes at $0.64 (very low for Opus), exited with code 0, but never wrote agent-result.json. The post-script then failed: agent-result.json not found. The run log contained a warning: env.sandbox key "FULLSEND_OUTPUT_FILE" is reserved for runner infrastructure; skipping — the harness config's FULLSEND_OUTPUT_FILE: agent-result.json in env.sandbox (line 88 of dependabot-risk-rater.yaml) was silently dropped. The agent definition at rhdh/agents/dependabot-risk-rater.md tells the agent to write to $FULLSEND_OUTPUT_DIR/${FULLSEND_OUTPUT_FILE:-agent-result.json}. This same post-script failure also occurred on Sep 2 19:22 (before the vertex-ai error started), suggesting it is a pre-existing, distinct bug.

What could go better

Two issues may be contributing:

1. FULLSEND_OUTPUT_FILE silently dropped: The runner treats FULLSEND_OUTPUT_FILE as reserved and skips the config value. If the runner does inject its own value, this is cosmetic. If it does not reach the sandbox, the agent falls back to the hardcoded default (agent-result.json) but may write to a different directory than the post-script expects.

2. Agent fails to complete its task: The $0.64 cost and ~4-minute runtime suggest the agent stalled during setup (skill loading, pr-data extraction, environment checks) and never reached the scoring/output step. The agent definition references a remote skill URL that must be fetched inside the sandbox; if skill loading fails silently, the agent may exit without producing output.

Confidence: medium. I can confirm the post-script failure and the FULLSEND_OUTPUT_FILE warning from the run logs. I cannot determine the exact root cause without the full agent transcript. The pre-existing nature of this bug (Sep 2 occurrence) increases confidence that it is independent of PR #30's changes.

Proposed change

Investigate the agent transcript from run 33773492498 to determine why the agent did not produce output. Likely areas to check:

1. Verify FULLSEND_OUTPUT_FILE reaches the sandbox: Check whether the runner's reserved-key mechanism injects the value or only blocks the config override. If the variable is not available inside the sandbox, either remove it from env.sandbox (if the runner handles it) or move it to env.runner only (line 83 already sets it there).

2. Check skill loading: The dependency-update-risk-rating skill is fetched from a remote URL inside the sandbox. Verify the skill loaded successfully. If the sandbox network policy or DNS resolution blocks it, the agent would stall.

3. Check pr-data.tar availability: The agent extracts pre-fetched PR data from /sandbox/workspace/pr-data.tar. If the host_files mount failed silently (it is marked optional: true on line 66), the agent would have no data to process.

The fix depends on root cause. If it is the env var reservation, remove FULLSEND_OUTPUT_FILE from env.sandbox (line 88) since the runner already sets it in env.runner (line 83). If it is skill loading, verify the allowed_remote_resources in the harness config covers the skill URL. If it is pr-data.tar, remove the optional: true flag so the mount failure is caught early.

Validation criteria

A /fullsend-risk-rating command on any open dependabot PR produces a risk-rating comment. The post-script should find agent-result.json and post successfully. Verify on at least one run after the fix is applied.

Generated by retro agent from #30