Skip to content

claude-code-action: background until [ -f X.completed ] loops orphan workflows past session end #594

Description

@prql-bot

Problem

A tend-mention session on PRQL/prql hung its workflow run for ~5h40m past the bot's last action, then was killed by GitHub's default 360-min timeout-minutes cap. The run shows as cancelled even though the bot's actual work completed successfully.

Run: https://github.com/PRQL/prql/actions/runs/26347739838. Session: 2026-05-24T00:44:42Z → 01:04:44Z (20 min). Job cancelled: 2026-05-24T06:44:13Z (5h40m later).

Root cause

The harness auto-backgrounded a long-running task prqlc:pull-request invocation (assigned ID bg95lbk45). The tool result, as returned to the model, reads:

Command running in background with ID: bg95lbk45. Output is being written to: /tmp/claude-1001/.../tasks/bg95lbk45.output. You will be notified when it completes. To check interim output, use Read on that file path.

The model nonetheless invented a polling pattern and issued a second Bash call with run_in_background: true:

until [ -f /tmp/claude-1001/.../tasks/bg95lbk45.completed ]; do sleep 15; done; echo "done"

No .completed marker file exists in the Claude Code background-task system — only .output. The directory listing at session end confirmed this (three .output files, zero .completed files). The until-loop polled forever.

When the model's session finished at 01:04:44Z it had not killed the polling loop. The runner's "Cleaning up orphan processes" step at 06:44:13Z terminated bun (claude-code-action), claude (harness), bash (the until-loop shell), and sleep (the inner sleep) — at which point GitHub finalized the job as cancelled.

Terminate orphan process: pid (2609) (bun)
Terminate orphan process: pid (2753) (claude)
Terminate orphan process: pid (32815) (bash)
Terminate orphan process: pid (47948) (sleep)

Why it bites every consumer

Two mechanisms converge:

  1. Stochastic model behavior. When the harness auto-backgrounds a command, the model has no advance warning. The "you will be notified" sentence does not always override a learned "poll for a marker file" pattern. Once the loop is started with run_in_background: true, the model itself never touches the background process again.
  2. No structural cleanup. Neither claude-code-action nor the tend wrapper kills lingering Claude-managed background tasks at session end. The shell child outlives the model and the harness — only the runner's orphan-process cleanup at job timeout stops it.

The bundled tend-ci-runner/skills/running-in-ci skill does cover the rollup-based PR-check polling recipe (with run_in_background: true), but it doesn't warn against polling for .completed marker files in general, and doesn't address the auto-backgrounded case where the model didn't ask for backgrounding in the first place.

Suggested fixes (any one would prevent recurrence)

  1. Harness cleanup (structural). Before declaring the session done, claude-code-action (or the tend wrapper) should send SIGTERM to any still-running Claude-managed background task. The orphan-process termination at job-timeout proves the runner already knows how — the wrapper just needs to do it earlier.
  2. Explicit guidance in running-in-ci (stochastic mitigation). Add a one-line rule under "Using your tools" or near the existing polling guidance: "Never write until [ -f X.completed ]; do sleep …; done loops. Background-task completion is signaled via task notifications, not marker files. After run_in_background: true you'll receive a <task-notification> user message — continue with other work until then."
  3. Surface auto-backgrounded commands distinctly. When the harness auto-backgrounds (the bot didn't request run_in_background: true), include a stronger reminder in the tool result, e.g. "Do not poll — wait for the task notification."

For PRQL's purposes a repo-local stopgap in running-tend.md covers (2) for this consumer. The structural fix (1) and the bundled-skill update (2) would prevent recurrence in every consumer.

Reproducer signal

The pattern surfaces whenever the harness auto-backgrounds a command (i.e., any command whose run exceeds its tool timeout) and the model attempts to wait synchronously by re-checking a filesystem marker. A grep across recent claude-session-logs JSONL for \.completed.*sleep would catch all instances.

Cost

This single occurrence wasted ~5h40m of runner minutes (one GitHub-hosted ubuntu-latest job) and converted a successful posting into a cancelled run. Direct token cost was $0 (the model wasn't doing anything for those hours), but the run-state confusion is worse than the minutes — future review-runs and CI-fix skills can't easily tell that this cancelled represented a successful workflow followed by an orphan hang.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions