Skip to content

Inline per-trace metrics + Claude Code identity on traces - #11

Merged
collincunn merged 1 commit into
mainfrom
collinc/cc-inline-metrics-and-identity
Jun 2, 2026
Merged

Inline per-trace metrics + Claude Code identity on traces#11
collincunn merged 1 commit into
mainfrom
collinc/cc-inline-metrics-and-identity

Conversation

@collincunn

Copy link
Copy Markdown
Contributor

Summary

  • Hook now posts a set of deterministic, git-grounded feedback scores on every trace close (Stop / Compact / SessionEnd). No daemon, no queue, no LLM.
  • Trace metadata + tags carry the Claude Code OAuth identity (user + org), so dashboards can distinguish users without inference.

What lands on each trace

Feedback scores (computed in the hook):

Name Source
cc.repository git remote get-url originhost/org/repo
cc.branch git branch --show-current
cc.head_sha_start / cc.head_sha_end git rev-parse HEAD at trace start / end
cc.commits_in_trace git log start..end --shortstat
cc.lines_committed sum of +/- across those commits
cc.uncommitted_lines / cc.uncommitted_files git diff HEAD --shortstat at trace close
cc.files_authored distinct file paths in Edit/Write/MultiEdit transcript entries
cc.lines_authored / cc.lines_overwritten line counts of new_string / old_string in those entries

Trace metadata (metadata.cc):

user_email, user_uuid, user_display_name, org_uuid, org_name

Read from ~/.claude.json oauthAccount. Email is also pushed onto trace.tags as user:<email> for filtering.

Why this shape

  • All values are deterministic and locally derivable — no LLM, no settle/poll delay.
  • Capturing HEAD SHA at trace start lets the close-time computation report a true commit window for the turn.
  • Identity comes from Claude Code's OAuth account (not git config user.email), so it aligns with billing/auth and gives a stable org primitive for team-level views.
  • Plain feedback scores + standard metadata keeps the surface compatible with existing Opik UI filtering, charts, and rollups.

What changed

  • src/state.go — adds Cwd, HeadSHAStart so close-time ops can anchor against trace start
  • src/api.go — adds Put; adds Metadata to Trace
  • src/identity.go~/.claude.json reader + applyToTrace helper
  • src/metrics.go — transcript walk for edit aggregates + git wrappers + feedback-score POST
  • src/main.go — captures cwd/HEAD on UserPromptSubmit/PreCompact; applies identity to every created trace; calls postTraceMetrics on trace closure

Test plan

  • Verified end-to-end against comet-all/claude-code project: 12 cc.* scores landing per trace, identity present in metadata.cc, user:<email> tag visible
  • Verified edit aggregates match real edits (single-edit turn → files_authored=1, lines_authored=2, lines_overwritten=1)
  • Compile check: make build succeeds on the four target platforms

🤖 Generated with Claude Code

Captures deterministic git-grounded metrics in the hook itself (no daemon)
and stamps every trace with the user's Claude Code OAuth identity.

Metrics posted as feedback scores on Stop/Compact/SessionEnd:
  cc.repository, cc.branch, cc.head_sha_start, cc.head_sha_end,
  cc.commits_in_trace, cc.lines_committed, cc.uncommitted_lines,
  cc.uncommitted_files, cc.files_authored, cc.lines_authored,
  cc.lines_overwritten.

Identity from ~/.claude.json oauthAccount is applied to trace.metadata.cc
(user_email, user_uuid, user_display_name, org_uuid, org_name) and to
trace.tags as `user:<email>` for one-click filtering in the Opik UI.

State carries cwd + HEAD SHA captured at trace start so the end-of-trace
git ops can compute an accurate start..end commit range.
@collincunn
collincunn merged commit a976fff into main Jun 2, 2026
1 check passed
@jverre
jverre deleted the collinc/cc-inline-metrics-and-identity branch June 12, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant