Skip to content

/do implementation worktrees escape the workspace contract into the shared ~/data root — 63 orphaned dirs / 84.3 GiB, unreachable by the reap contract #3925

Description

@tomerweller

/do implementation worktrees escape the workspace contract into the shared ~/data root — 63 orphaned dirs / 84.3 GiB, unreachable by the reap contract, newest created while this tick was running

Surfaced by /monitor-tick while attributing a −0.06 GiB data-root step on the shared volume.

Same shape of contract escape as #3921, but a different family, a different creator, and three
orders of magnitude more disk
: #3921 is review-pr lens/bounce scratch (18 entries, 190 KB,
hygiene-only). This is /do's implementation worktrees — full repo checkouts plus their
cargo-target/ — landing at the bare ~/data root instead of under a session prefix.

Distinct from #3918, which owns the xshell-nonexistent-<PID> class.

Symptom

$HOME/data holds 63 bare-root do-* directories totalling 90,523,951,850 B = 84.307 GiB,
accumulating since 2026-06-15. 43 of the 63 are larger than 1 MiB — i.e. real worktrees, not stubs.

Ten largest:

11,651,192,455 B  (10.85 GiB)  do-3870-loop            2026-08-18 20:02:39
 8,405,221,739 B  ( 7.83 GiB)  do-3811-mode-b          2026-08-10 11:04:13
 2,498,159,685 B  ( 2.33 GiB)  do-3770                 2026-07-28 12:57:38
 2,461,658,587 B  ( 2.29 GiB)  do-3756                 2026-07-28 04:33:12
 2,460,764,965 B  ( 2.29 GiB)  do-3754                 2026-07-28 04:33:19
 2,270,756,254 B  ( 2.11 GiB)  do-3412                 2026-06-18 01:52:44
 2,119,114,180 B  ( 1.97 GiB)  do-3768                 2026-08-14 03:16:56
 2,013,161,673 B  ( 1.88 GiB)  do-3639                 2026-06-26 05:06:20
 1,879,893,564 B  ( 1.75 GiB)  do-3323                 2026-06-17 05:12:21
 1,879,870,605 B  ( 1.75 GiB)  do-3324-20260617-051242 2026-06-17 05:12:42

Oldest: do-3286-bootstrap (2026-06-15 22:27:58). Newest: do-3791-modeb
(2026-08-23 22:31:52) — created between two consecutive monitor ticks, so this is live, not
historical residue.

Explicitly NOT part of this report: the 108 .workspace pointer files

There are also 129 bare-root do-* files, of which 108 are do-<issue>.workspace. Those are
contract-sanctioned.claude/skills/do/SKILL.md:108 mandates exactly that path:

printf '%s\n' "$DO_WORKSPACE" > "$PW_HOME/data/do-$ISSUE.workspace"

I am not proposing any change to them, and they are excluded from every number above. This issue is
about the directories only.

Why this is off-contract

.claude/skills/do/SKILL.md states the invariant in three places — :76, :85, :444:

$WORKTREE_BASE (= ~/data/$SESSION_ID/do-$ISSUE), never inside the repo tree

and the helper that implements it cannot produce a bare root path.
scripts/lib/agent-worktree-contract.sh:258-284:

do_bootstrap() {
  local issue="$1"
  local session_id="${CLAUDE_SESSION_ID:-${SESSION_ID:-$(date +%Y%m%d-%H%M%S)}}"
  ...
  local expected_prefix="$real_home/data/$session_id/do-$issue"
  ...
  local candidate_base="${incoming_base:-$real_home/data/$session_id/do-$issue}"
  if ! WORKTREE_BASE="$(require_home_data_path "$candidate_base" "WORKTREE_BASE")"; then
  ...
  if [[ -n "$incoming_base" ]]; then
    if ! require_session_prefix "$WORKTREE_BASE" "$expected_prefix" "WORKTREE_BASE"; then

session_id has a three-level fallback terminating in $(date +%Y%m%d-%H%M%S), so it is never empty,
and any pre-seeded WORKTREE_BASE is re-validated against expected_prefix. DO_WORKTREE is then
derived as $WORKTREE_BASE/worktree. A path of the form ~/data/do-3791-modeb/do-3791/worktree
which is what exists on disk — is unreachable through this function. Something is creating the
implementation worktree without going through do_bootstrap
, taking the documented template and
dropping the $SESSION_ID segment. This is structurally the same defect as #3921's, in the sibling
do_bootstrap path rather than review_pr_bootstrap.

The same pass produced both shapes 39 seconds apart, and abandoned the correct one

This is the part that rules out "the contract just isn't wired up for this pass":

22:31:13   /home/tomer/data/20260823-223112/do-3791          8,192 B   <- contract-correct, and EMPTY
22:31:52   /home/tomer/data/do-3791-modeb/do-3791/worktree  57,556,186 B  <- bare root, full checkout

The session-scoped directory was created, left with zero files in it, and the actual worktree went
to the bare root 39 s later. So the correct path is being computed on some code path and then not used.

The reap contract provably cannot clean these up

SKILL.md:198 and :448 define the cleanup as rm -rf "$WORKTREE_BASE", and :448 names the exact
class this skill exists to prevent:

the leak this skill exists to prevent: orphaned in-bounds workspaces that never produced a merged artifact

These 63 are orphaned out-of-bounds workspaces — strictly worse, because $WORKTREE_BASE never
points at them. The pointer file proves it directly:

$ cat /home/tomer/data/do-3791.workspace
/home/tomer/data/20260823-213114/do-3791

The sanctioned pointer records the in-bounds cycle-1 path. Nothing on disk references
do-3791-modeb, so no reap step — on merge, on blocked, or on failure — can ever target it. That is
the mechanism by which 84.3 GiB accumulated over 69 days without anything noticing.

At least six distinct name shapes, i.e. more than one bypass site

 37  do-<issue>                    (plain template, session segment dropped)
 11  do-<issue>-<label>            (-modeb, -mode-b, -fixed, -loop, -bootstrap, -session, -db, -ledger)
  6  do-session-*                  (do-session-3319, do-session-1742800, do-session-20260618-193307-2118744, …)
  3  do-<issue>-<epoch>            (do-3393-1781732062, do-3419-1781751384, …)
  2  do-watchdog-*                 (do-watchdog-3773, do-watchdog-3842)
  2  do-<issue>-<date>             (do-3324-20260617-051242, do-3426-20260618)
  1  do-fallback-<timestamp>       (do-fallback-20260617-171516)
  1  do-3319-app-<epoch>

A single missing $SESSION_ID would produce only the first shape. Eight shapes — including
do-fallback-<ts> and do-session-<pid>, which look like deliberate ad-hoc fallbacks when a session
id was unavailable — point at several independent path-composition sites rather than one bad expansion.

Impact — this one is runway-relevant

Unlike #3921 (~0.0002 GiB/day, which I explicitly kept off #3749's arithmetic), this pool is material:

Secondary costs, same as #3921 but larger:

  1. The /review-pr sub-agents create huge worktrees in /tmp (11GB seen, near-OOM root FS) #2843 isolation guarantee is not held. Root-level worktrees sit outside any session prefix,
    so require_session_prefix's poisoning protection does not apply and no session expiry reaches them.
  2. Cross-run collision. ~/data/do-3791-modeb is a fixed path; two concurrent Mode-B passes on the
    same issue would share one worktree. Note that three shapes already carry epoch/date suffixes
    (do-3393-1781732062, do-3324-20260617-051242), which reads like someone already hit this.
  3. Data-root attribution noise. This issue exists because a routine −0.06 GiB step could not be
    explained without chasing one of them down.

Suggested fix

  1. Route the implementation-worktree creation through do_bootstrap (or at minimum through
    require_home_data_path + require_session_prefix) instead of composing the path inline, so the
    $SESSION_ID segment cannot be dropped. The do-fallback-* / do-session-<pid> shapes suggest
    some sites have their own session-id fallback; do_bootstrap's three-level fallback already covers
    that case and should be used rather than reimplemented.
  2. Add a startup assertion rejecting any WORKTREE_BASE/DO_WORKTREE matching ^$HOME/data/do-
    (one level). That single guard would have caught all 63. assert_no_repo_tree_scratch
    (agent-worktree-contract.sh:329) is the natural home — it is already a detection-only,
    no-deletion guard over an enumerated leak-pattern list, and this is one more pattern.
  3. Separately decide what to do with the existing 84.3 GiB. The monitor has not deleted anything
    and is not proposing an unattended sweep: these are co-tenant artifacts and several may still be
    wanted. They are, however, reclaimable by whoever owns them, and do-3870-loop (10.85 GiB) plus
    do-3811-mode-b (7.83 GiB) alone are 18.7 GiB.

Non-urgent — no validator impact, and no label per the monitor's label policy. For context: the mainnet
validator is down on the unrelated #3910 outage (38h05m, build_sha 0ac84d42, deploys held by the
blocked_active quarantine); nothing in this issue bears on that.

Filed by /monitor-tick.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions