Document patterns for use in github actions

[cmungall]

ai-blame works for local agent use, but if the agent runs as part of an action, the underlying logs may be lost.

Figure out patterns for either

• running ai-blame within the action
• persisting github action agent logs (e.g using some telemetry solution, though last I checked cc integration was poor)
• mining the action logs after the fact

[github-actions]

Issue Analysis: GitHub Actions Integration for ai-blame

Summary

This issue addresses a key limitation: ai-blame currently only works with local Claude Code traces stored in ~/.claude/projects/<encoded-cwd>/. When AI agents run within GitHub Actions, their execution logs are ephemeral and not persisted to the local trace directory, making provenance extraction impossible.

Problem Context

Based on the codebase analysis:

1. Current Architecture (src/ai_blame/extractor.py:13-18):

   • ai-blame looks for traces in ~/.claude/projects/<encoded-path>/
   • Traces are JSONL files containing tool use records (Edit/Write operations)
   • Extraction works by parsing toolUseResult objects from these files

2. GitHub Actions Environment:

   • Looking at .github/workflows/claude.yml and dragon-ai.yml, agents run ephemerally
   • The dragon-ai workflow uses claude CLI with --output-format stream-json but doesn't persist logs
   • No current mechanism to save traces for later analysis

Proposed Solutions (3 Approaches)

Option 1: Run ai-blame Within the Action ⭐ RECOMMENDED

Implementation Steps:

1. Add a step after Claude execution to run ai-blame mine --apply
2. Commit the updated files with provenance metadata
3. Include in existing PR creation flow

Advantages:

• Immediate provenance capture
• No additional infrastructure needed
• Fits existing workflow

Example addition to dragon-ai.yml:

- name: Extract AI Provenance
  run: |
    uvx ai-blame mine --apply --initial-and-recent
    if [[ -n $(git status --porcelain) ]]; then
      git add .
      git commit -m "Add AI provenance metadata"
    fi

Challenge: Current ai-blame assumes traces exist in ~/.claude/projects/. Need to either:

• Configure Claude to write traces to a known location in Actions environment
• Modify ai-blame to accept alternative trace locations via --trace-dir (already supported!)

Option 2: Persist Traces Using Telemetry/Artifacts

Implementation:

1. Configure Claude Code Action to output traces to a file
2. Upload as GitHub Actions artifact
3. Optionally process in subsequent workflow or locally

Advantages:

• Decouples provenance extraction from agent execution
• Enables batch processing
• Audit trail preservation

Disadvantages:

• Requires modifications to claude-code-action
• More complex workflow
• Additional storage costs

Research Needed:

• Check if anthropics/claude-code-action@beta supports trace output configuration
• Review Claude Code SDK documentation for telemetry options

Option 3: Mine Action Logs After the Fact

Implementation:

1. Parse GitHub Actions logs containing Claude's output
2. Extract Edit/Write operations from log text
3. Reconstruct provenance from structured log output

Advantages:

• Works with existing setup
• No agent configuration changes needed

Disadvantages:

• Fragile (depends on log format stability)
• Incomplete metadata compared to native traces
• High parsing complexity

Note: The dragon-ai workflow already uses --output-format stream-json, which could be captured and parsed.

Recommended Path Forward

Short-term (Immediate):

1. Modify dragon-ai.yml to capture trace output:

   - name: Run Claude and capture traces
     run: |
       claude ... --output-format stream-json 2>&1 | tee /tmp/claude-trace.jsonl

2. Add ai-blame execution step:

   - name: Extract provenance
     run: |
       uvx ai-blame mine --trace-dir /tmp --apply --initial-and-recent

Medium-term (Documentation):

1. Document the pattern in ai-blame docs
2. Create example workflow file
3. Add troubleshooting guide for Actions environment

Long-term (Enhancement):

1. Add native support for parsing Claude's stream-json output format
2. Create GitHub Action wrapper: ai4curation/ai-blame-action
3. Support other CI/CD platforms (GitLab, CircleCI)

Implementation Considerations

1. Trace Location: The --trace-dir flag (cli.py:93-99) already supports custom locations
2. Permissions: Actions need write access to commit provenance updates
3. Incremental Updates: Consider only running on files modified in the PR
4. Performance: Trace parsing is fast, but limit to --initial-and-recent to avoid bloat

Related Work & Research Gaps

This is a relatively novel problem space. While there's extensive research on:

• Software provenance tracking (PROV-O, W3C PROV)
• CI/CD observability (OpenTelemetry)
• LLM evaluation frameworks (LangSmith, Weights & Biases)

There's limited prior art on AI agent provenance in CI/CD specifically. Key papers to review:

• PROV ontology for AI workflows
• MLOps provenance tracking
• Reproducibility in computational pipelines

(Note: Academic paper search was blocked by permissions, but this would be valuable to explore in depth)

Next Steps

1. Prototype Option 1 in dragon-ai.yml workflow
2. Test with a real PR to validate trace capture
3. Document the pattern in README
4. Consider creating a reusable action

Would you like me to implement the prototype workflow modification? Or shall I focus on documenting the pattern first?

---

Generated with Claude Sonnet 4.5 analyzing issue #6