Usage metrics: capture unattended runs, count task invocations - #558
Merged
Conversation
Two gaps, one pipeline. Scheduled task runs were invisible from both ends: no executor session had ever reached the conversation-logs branch (47 capture files, not one from a dispatch), and the runs that open no session at all — precondition skips, agentless preprocessing passes, suppressed dispatches — left no countable trace anywhere. Capture (DESIGN §3.4): the executor runs the engine's SessionEnd runner itself as its last step, naming its dispatch issue via CLAUDINITE_SESSION_ISSUE, so an unattended session's log files under the task that produced it. An executor session ends by container reclaim, which is exactly the ending no SessionEnd hook fires on, so this could not be left to the hook. Core still names no pack: the executor invokes a runner, the runner invokes whatever the declared packs contribute. Double-writing stays safe by construction (capture's session-keyed delta). executor.md also states its engine root unambiguously, since both of its commands now address paths from it. Task invocations (DESIGN §4.2): each scheduler run prints one machine-readable record per due task — claudinite-task-run v1 <pack>/<task> [<slot>] <outcome>, outcome in agent / preprocess / skipped / failed / deferred — emitted after the action loop, so it states what happened rather than what was planned. Renderer and parser live together in engine/scheduler/run-record.mjs with a round-trip test. The fold reads those records from the Actions logs past a runsFoldedThrough watermark and counts them per task at day and week grain; fleet-usage carries them at week x repo x task. Forward-only rather than recomputed, because the run logs are a rate-limited REST read where the capture files are local git — the trade, and its cost (a counting fix applies forward, it does not heal history), is stated in the code, the task doc and the design. These rows are a census of scheduled work, beside numbers that are a sample of captured sessions; both files say so. The fold's precondition drops with it — it has two sources now, and the second exists in any repo that has a scheduler. Closes #553. Refs #520. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MkAs4A9km3y5SEvVzwH2me
missingbulb
force-pushed
the
claude/ccr-conversation-logs-0006hx
branch
from
July 29, 2026 15:33
ce85a43 to
a8f182c
Compare
This was referenced Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #553. Refs #520. Extends
docs/skill-usage-metrics/DESIGN.md(§3.4, §4.2, owner decisions 11–14).Scheduled task runs were invisible from both ends, and the two halves have different causes.
Unattended sessions never reached the logs branch
Measured, not inferred: the
conversation-logsbranch held 47 capture files, none from an executor session, while the scheduler had dispatched hundreds of tasks. Neither existing capture event can reach one — an executor session never merges throughmerge-to-main(its task delivers by opening a PR, or by converging its issue), and it ends by having its container reclaimed, which is exactly the ending that fires noSessionEndhook.The fix is the same step, invoked deliberately instead of waited for.
session-end-command.mjsis a runner, not a hook body, so the executor now runs it itself as step 5 — after its issue is converged, whichever way step 4 went — naming its dispatch issue inCLAUDINITE_SESSION_ISSUE. The capture step uses that in place of0, so an unattended run's log files under the dispatch issue whose title namespack/task. Core still names no pack: the executor invokes a runner, the runner invokes whatever the declared packs contribute, and a repo declaring none does nothing. Double-writing with a later hook firing is safe by construction via capture's session-keyed delta.Not assertable in advance: whether a trigger-started session can push to the logs branch at all — every existing capture came from an owner-started container. If it cannot, the step reports it in the session's final message and the outcome is exactly today's, with the task counts below unaffected.
Task invocations, from the scheduler's own log
A whole half of what a repo does on a schedule opens no session: a precondition that finds nothing to do, an
agent_model: nonetask whose deterministic pass is the entire task, a dispatch suppressed because an earlier one is still open. Not under-counted by the capture pipeline — absent from it.Each scheduler run now prints one machine-readable record per due task:
emitted after the action loop, so it states what happened rather than what was planned (the human summary line is written before the actions run, so an agentful task whose preprocessing then requested no agent still reads there as its dispatch decision). Renderer and parser live together in
engine/scheduler/run-record.mjswith a round-trip test.Five outcomes, one counter each, keyed by
pack/task:agentpreprocessagent_model: none, or an agentful task whose preprocessing requested no agentskippedfaileddeferredfailedanddeferredare not decoration: foldingfailedintopreprocessmakes a task that fails nightly look like one that works nightly, and foldingdeferredintoagentreports executions that never happened.usage-foldreads those records past a newrunsFoldedThroughwatermark and counts them at day and week grain;fleet-usagecarries them at week × repo × task. Forward-only rather than recomputed — the one place the day tier departs from "pure function of its sources" — because the capture files are a local git branch re-read for free while the run logs are a paged REST resource at two calls per run, where a nightly full re-read would cost ~20× the API calls for an identical answer. The cost is stated wherever the trade is made: a counting bug fixed later applies from the fix forward and does not heal history. A per-fold cap (240 runs, ~10 days of catch-up) advances the watermark only through what it read and logs the remainder rather than truncating silently.These rows are a census, not a sample, and both files say so — every scheduler run records every due task, unlike the skill and check counts, which see only captured sessions. The fold's precondition drops with it: it has two sources now, and the second exists in any repo that has a scheduler, so a repo that never captured still folds its task rows.
Also here
executor.mdstates its engine root unambiguously (the parent of the directory the file sits in), since both of its commands now address paths from it — the prior wording contradicted its ownscheduler/resolve-dispatch.mjsusage.Verification
963 tests pass;
check_the_world0 blocking (8 pre-existingfile-placementadvisories on engine internals, none in the new files);check_the_workclean. New tests cover the outcome deriver against every evaluation shape, the line format's round trip and its tolerance of the Actions timestamp prefix, the reader's watermark/cap/fail-soft behaviour against a fake API, the fold's task rows and their week folding, the runner's issue pass-through, and the capture step filing under a named issue.Note on the commit: it is unsigned. This session's host leaked its socket budget (the commit-signing helper opens a connection per signed commit and never closes it; the test suite signs every scratch-repo commit), which took out signing and the git proxy together — the push landed only after the leak eased. Committer identity is correct; the signature can only be restored by re-committing where signing works.
🤖 Generated with Claude Code
https://claude.ai/code/session_01MkAs4A9km3y5SEvVzwH2me