[BUG] apm compile --clean never removes a stale CLAUDE.md after instructions move to .claude/rules/

[tillig]

Is this a bug? Yes — companion to #1138; the inverse-direction gap that #1138's dedup fix left open.

Summary

#1138 (PR #1146) made apm compile --target claude skip creating a CLAUDE.md when instructions are already deployed to .claude/rules/ (the files_written == 0 and skip_instructions guard at agents_compiler.py:785-791). That works for a clean checkout.

But it only suppresses creation. If a CLAUDE.md already exists on disk — authored in an earlier APM version, or generated before .claude/rules/ was populated — APM never removes it, on plain apm compile or apm compile --clean. The stale file lingers indefinitely, and because it predates the dedup it still contains the full instruction content that now also lives in .claude/rules/ — exactly the duplicate-context problem #1138 set out to eliminate.

Both targets share a version of this gap, but the Claude path is worse off: the AGENTS.md path at least has orphan-detection infrastructure (DistributedAgentsCompiler._find_orphaned_agents_files / _cleanup_orphaned_files), even though it doesn't currently catch the analogous redundant-file case (see the sibling AGENTS.md issue). The CLAUDE.md path has no cleanup logic at all — grep finds no rglob("CLAUDE.md"), no orphan check, no unlink anywhere in the Claude compile path.

Environment

• APM version: 0.19.0
• OS: macOS

Steps to reproduce

mkdir -p /tmp/apm-claude-stale/.apm/instructions && cd /tmp/apm-claude-stale
cat > apm.yml <<'EOF'
name: test-claude-stale
version: 1.0.0
targets: [claude]
EOF
cat > .apm/instructions/global.instructions.md <<'EOF'
---
description: A global instruction
---
This is global guidance.
EOF

# 1. Compile BEFORE .claude/rules/ exists -> CLAUDE.md is written (the "upgrade path" starting state)
apm compile --target claude
cat CLAUDE.md          # full instructions present

# 2. Simulate `apm install` populating .claude/rules/, then recompile
mkdir -p .claude/rules
echo "This is global guidance." > .claude/rules/global.md
apm compile --target claude
cat CLAUDE.md          # STILL THERE, still full instructions -> now duplicated with .claude/rules/

# 3. --clean does not help either
apm compile --target claude --clean
test -f CLAUDE.md && echo "CLAUDE.md STILL present after --clean"

Expected behavior

When instruction dedup fires (.claude/rules/ already holds the instructions), apm compile --clean should remove a pre-existing, APM-generated CLAUDE.md that would now be redundant — gated on the APM-generated marker (<!-- Generated by APM CLI -->, claude_formatter.py:26) so a hand-authored CLAUDE.md is never deleted. Plain apm compile should remain non-destructive (it already correctly avoids creating a new one).

The sibling AGENTS.md issue asks for the same --clean behavior on that path; fixing both gives consistent, predictable cleanup across targets. (Neither path does this today.)

Actual behavior

• Step 1 writes a full CLAUDE.md (expected for that starting state).
• Step 2: dedup fires and logs "Instructions already in .claude/rules/ — omitting from CLAUDE.md", but the existing CLAUDE.md is left untouched — now duplicating .claude/rules/.
• Step 3: apm compile --clean does not remove it. There is no CLAUDE.md cleanup path to invoke.

Root cause

• _compile_claude_md (agents_compiler.py:617-857) only guards new writes (line 785: if files_written == 0 and skip_instructions: → log and write nothing). It never inspects or removes an existing on-disk CLAUDE.md.
• There is no Claude analog to DistributedAgentsCompiler._find_orphaned_agents_files / _cleanup_orphaned_files (distributed_compiler.py:643, 722). The clean_orphaned / --clean flow is wired only into the AGENTS.md distributed compiler, so it never reaches the Claude target. (Note: even that AGENTS.md flow only removes files no longer in the generated set — it does not currently remove regenerated-but-redundant files; that is the subject of the sibling AGENTS.md issue.)

Suggested fix

In _compile_claude_md, when skip_instructions is true and a root CLAUDE.md already exists:

1. If config.clean_orphaned (--clean) is set and the existing file carries the APM-generated marker (claude_formatter.py:26), remove it and log the removal.
2. If the existing file lacks the marker (hand-authored), leave it and emit the same "hand-authored file will not be overwritten" warning the Copilot-root path already uses (agents_compiler.py:1131-1138).
3. Plain apm compile (no --clean): unchanged — suppress creation only, never delete. Optionally surface a hint that a stale CLAUDE.md exists and --clean would remove it.

Acceptance criteria

• Failing test: existing APM-generated CLAUDE.md + populated .claude/rules/ + apm compile --clean → CLAUDE.md removed.
• Hand-authored CLAUDE.md (no marker) is never removed; warning emitted instead.
• Plain apm compile still deletes nothing (non-destructive).
• Clean-checkout behavior from [FEATURE] Skip instructions in CLAUDE.md when already deployed to .claude/rules/ #1138 unchanged (no CLAUDE.md ever created).
• --no-dedup / --force-instructions still regenerate a full CLAUDE.md and never trigger removal.
• CHANGELOG.md ### Fixed entry referencing this issue and [FEATURE] Skip instructions in CLAUDE.md when already deployed to .claude/rules/ #1138.

Related

• [FEATURE] Skip instructions in CLAUDE.md when already deployed to .claude/rules/ #1138 — added the Claude-side dedup + skip-on-create guard (PR fix(compile): deduplicate Claude Code instructions (#1138) #1146); this issue closes the stale-pre-existing-file gap that fix left open.
• perf/dedup: parity for Copilot AGENTS.md vs .github/instructions/ (sibling of #1445) #1550 — the Copilot/AGENTS.md dedup parity work. The AGENTS.md path has orphan-removal infrastructure the Claude path lacks entirely — but it does not currently remove the analogous redundant files either; the sibling AGENTS.md issue addresses that. The two issues should be fixed consistently.