Skip to content

Usage metrics: capture unattended runs, count task invocations - #558

Merged
missingbulb merged 1 commit into
mainfrom
claude/ccr-conversation-logs-0006hx
Jul 29, 2026
Merged

Usage metrics: capture unattended runs, count task invocations#558
missingbulb merged 1 commit into
mainfrom
claude/ccr-conversation-logs-0006hx

Conversation

@missingbulb

@missingbulb missingbulb commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Closes #553. Refs #520. Extends docs/skill-usage-metrics/DESIGN.md (§3.4, §4.2, owner decisions 11–14).

Scheduled task runs were invisible from both ends, and the two halves have different causes.

Unattended sessions never reached the logs branch

Measured, not inferred: the conversation-logs branch held 47 capture files, none from an executor session, while the scheduler had dispatched hundreds of tasks. Neither existing capture event can reach one — an executor session never merges through merge-to-main (its task delivers by opening a PR, or by converging its issue), and it ends by having its container reclaimed, which is exactly the ending that fires no SessionEnd hook.

The fix is the same step, invoked deliberately instead of waited for. session-end-command.mjs is a runner, not a hook body, so the executor now runs it itself as step 5 — after its issue is converged, whichever way step 4 went — naming its dispatch issue in CLAUDINITE_SESSION_ISSUE. The capture step uses that in place of 0, so an unattended run's log files under the dispatch issue whose title names pack/task. Core still names no pack: the executor invokes a runner, the runner invokes whatever the declared packs contribute, and a repo declaring none does nothing. Double-writing with a later hook firing is safe by construction via capture's session-keyed delta.

Not assertable in advance: whether a trigger-started session can push to the logs branch at all — every existing capture came from an owner-started container. If it cannot, the step reports it in the session's final message and the outcome is exactly today's, with the task counts below unaffected.

Task invocations, from the scheduler's own log

A whole half of what a repo does on a schedule opens no session: a precondition that finds nothing to do, an agent_model: none task whose deterministic pass is the entire task, a dispatch suppressed because an earlier one is still open. Not under-counted by the capture pipeline — absent from it.

Each scheduler run now prints one machine-readable record per due task:

claudinite-task-run v1 <pack>/<task> [<slot>] <outcome>

emitted after the action loop, so it states what happened rather than what was planned (the human summary line is written before the actions run, so an agentful task whose preprocessing then requested no agent still reads there as its dispatch decision). Renderer and parser live together in engine/scheduler/run-record.mjs with a round-trip test.

Five outcomes, one counter each, keyed by pack/task:

outcome means
agent a dispatch was filed — an executor session ran it with an agent
preprocess ran with no agent: agent_model: none, or an agentful task whose preprocessing requested no agent
skipped due, but its precondition said there was nothing to do
failed preprocessing failed; the run converged it to needs-human
deferred past its precondition, but no new dispatch (this slot already filed, or an earlier one still open)

failed and deferred are not decoration: folding failed into preprocess makes a task that fails nightly look like one that works nightly, and folding deferred into agent reports executions that never happened.

usage-fold reads those records past a new runsFoldedThrough watermark and counts them at day and week grain; fleet-usage carries them at week × repo × task. Forward-only rather than recomputed — the one place the day tier departs from "pure function of its sources" — because the capture files are a local git branch re-read for free while the run logs are a paged REST resource at two calls per run, where a nightly full re-read would cost ~20× the API calls for an identical answer. The cost is stated wherever the trade is made: a counting bug fixed later applies from the fix forward and does not heal history. A per-fold cap (240 runs, ~10 days of catch-up) advances the watermark only through what it read and logs the remainder rather than truncating silently.

These rows are a census, not a sample, and both files say so — every scheduler run records every due task, unlike the skill and check counts, which see only captured sessions. The fold's precondition drops with it: it has two sources now, and the second exists in any repo that has a scheduler, so a repo that never captured still folds its task rows.

Also here

executor.md states its engine root unambiguously (the parent of the directory the file sits in), since both of its commands now address paths from it — the prior wording contradicted its own scheduler/resolve-dispatch.mjs usage.

Verification

963 tests pass; check_the_world 0 blocking (8 pre-existing file-placement advisories on engine internals, none in the new files); check_the_work clean. New tests cover the outcome deriver against every evaluation shape, the line format's round trip and its tolerance of the Actions timestamp prefix, the reader's watermark/cap/fail-soft behaviour against a fake API, the fold's task rows and their week folding, the runner's issue pass-through, and the capture step filing under a named issue.

Note on the commit: it is unsigned. This session's host leaked its socket budget (the commit-signing helper opens a connection per signed commit and never closes it; the test suite signs every scratch-repo commit), which took out signing and the git proxy together — the push landed only after the leak eased. Committer identity is correct; the signature can only be restored by re-committing where signing works.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MkAs4A9km3y5SEvVzwH2me

Two gaps, one pipeline. Scheduled task runs were invisible from both ends:
no executor session had ever reached the conversation-logs branch (47 capture
files, not one from a dispatch), and the runs that open no session at all —
precondition skips, agentless preprocessing passes, suppressed dispatches —
left no countable trace anywhere.

Capture (DESIGN §3.4): the executor runs the engine's SessionEnd runner itself
as its last step, naming its dispatch issue via CLAUDINITE_SESSION_ISSUE, so an
unattended session's log files under the task that produced it. An executor
session ends by container reclaim, which is exactly the ending no SessionEnd
hook fires on, so this could not be left to the hook. Core still names no pack:
the executor invokes a runner, the runner invokes whatever the declared packs
contribute. Double-writing stays safe by construction (capture's session-keyed
delta). executor.md also states its engine root unambiguously, since both of
its commands now address paths from it.

Task invocations (DESIGN §4.2): each scheduler run prints one machine-readable
record per due task — claudinite-task-run v1 <pack>/<task> [<slot>] <outcome>,
outcome in agent / preprocess / skipped / failed / deferred — emitted after the
action loop, so it states what happened rather than what was planned. Renderer
and parser live together in engine/scheduler/run-record.mjs with a round-trip
test. The fold reads those records from the Actions logs past a
runsFoldedThrough watermark and counts them per task at day and week grain;
fleet-usage carries them at week x repo x task. Forward-only rather than
recomputed, because the run logs are a rate-limited REST read where the capture
files are local git — the trade, and its cost (a counting fix applies forward,
it does not heal history), is stated in the code, the task doc and the design.

These rows are a census of scheduled work, beside numbers that are a sample of
captured sessions; both files say so. The fold's precondition drops with it — it
has two sources now, and the second exists in any repo that has a scheduler.

Closes #553. Refs #520.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MkAs4A9km3y5SEvVzwH2me
@missingbulb
missingbulb force-pushed the claude/ccr-conversation-logs-0006hx branch from ce85a43 to a8f182c Compare July 29, 2026 15:33
@missingbulb
missingbulb merged commit 29e843c into main Jul 29, 2026
1 check passed
@missingbulb
missingbulb deleted the claude/ccr-conversation-logs-0006hx branch July 29, 2026 22:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unattended runs are invisible: no executor session reaches the logs branch, and task invocations are counted nowhere

2 participants