Provenance tracking for AI-assisted curation: challenges and design considerations

[cmungall]

Problem Statement

As AI agents increasingly assist with knowledge base curation tasks (adding evidence, fixing ontology terms, validating references, etc.), there's a growing need to track provenance — which agent/model made what changes, when, and ideally why.

Without provenance, we lose:

• Auditability: Which claims were AI-generated vs human-curated?
• Reproducibility: What model version produced this content?
• Trust calibration: Users may want to weight AI-curated content differently
• Debugging: When something is wrong, tracing back to the source helps identify systematic issues

Key Design Tensions

1. Completeness vs File Bloat

Complete audit trail: Every edit, every model call, every intermediate step

• Pros: Full traceability, can reconstruct exact history
• Cons: Can massively inflate file sizes, noise obscures signal

Minimal provenance: Just first creation + last edit timestamps

• Pros: Clean files, easy to read
• Cons: Loses intermediate history, can't tell if file was edited 2 times or 200

Questions to consider:

• Should intermediate edits be recorded at all?
• If so, should there be size/count thresholds for filtering?
• Is "initial + most recent" a reasonable compromise for most use cases?

2. Storage Pattern: Inline vs Sidecar

Inline (appended to curated file):

# ... disease content ...
curation_history:
  - timestamp: "2025-12-01T08:00:00Z"
    model: claude-opus-4-5
    action: CREATED

• Pros: Self-contained, travels with the data, can't get out of sync
• Cons: Pollutes the "content" file, schema changes needed, harder to aggregate across files

Sidecar files (e.g., kb/disorders/.provenance/Asthma.jsonl):

• Pros: Keeps content files clean, can be more detailed, easier to process programmatically
• Cons: Can drift out of sync, extra files to manage, less portable

Hybrid approaches:

• Minimal inline (just a hash/signature) + detailed sidecar
• Content-addressable references to external provenance store

3. Agent Self-Logging vs Post-Hoc Extraction

Agent records its own provenance during curation:

# Agent writes this as part of the edit
curation_history:
  - timestamp: "2025-12-01T08:00:00Z"
    model: claude-opus-4-5
    description: "Added PMID:12345678 evidence for phenotype claim"

• Pros: Can include rich context (why the change was made), immediate
• Cons: Agents may forget, be inconsistent, or hallucinate provenance; adds complexity to prompts

Deterministic extraction from execution logs:

• Parse Claude Code traces, GitHub Actions logs, etc.
• Extract successful file operations after the fact
• Pros: Can't be "forgotten", deterministic, no prompt complexity
• Cons: Less context (no "why"), depends on log availability and format

4. Deployment Context Differences

Local Claude Code usage:

• Rich traces available in ~/.claude/projects/
• Full conversation context, tool calls, timestamps
• But: Local-only, not version controlled, ephemeral

GitHub Actions / CI-based agents:

• Logs available in Actions UI
• But: Harder to programmatically extract, different format
• May need explicit logging steps in workflow

Hybrid workflows (local dev + CI):

• How to maintain consistent provenance across contexts?

Open Questions

1. What's the minimal viable provenance that provides value without excessive overhead?
2. Should provenance be part of the data schema or a separate concern?
3. How do we handle provenance when multiple agents collaborate on the same file?
4. Should provenance be human-readable (YAML) or optimized for machines (JSONL)?
5. How do we handle the case where logs are unavailable (e.g., deleted, different machine)?
6. Are there existing standards we should align with? (W3C PROV, PAV ontology, C2PA for content credentials?)

Related Work

• W3C PROV-O - Provenance ontology
• PAV Ontology - Provenance, Authoring, Versioning
• C2PA - Content Credentials for media provenance
• OMIM's change history model (appended to entries)

Next Steps

• Survey existing approaches in related projects
• Define minimal provenance requirements for ai4curation use cases
• Prototype and compare inline vs sidecar approaches
• Document recommended patterns

/cc @cmungall

[github-actions]

Summary

This issue presents a comprehensive analysis of provenance tracking for AI-assisted knowledge base curation. It's a design exploration issue that identifies key technical tensions and questions rather than a specific implementation task. Status: Open for discussion and prototyping.

Context & Related Work Analysis

Standards Review

W3C PROV-O - The PROV Ontology provides a standardized framework with three core concepts:

• Entity: The curated artifacts (ontology files, knowledge base entries)
• Activity: Curation operations (adding evidence, fixing terms, validation)
• Agent: Both AI models and human curators

Key properties relevant for this use case:

• wasGeneratedBy: Links curated content to the curation activity
• wasDerivedFrom: Tracks transformations (e.g., AI-enhanced version derived from original)
• wasAttributedTo: Assigns responsibility to specific agents (model + version)
• wasAssociatedWith: Links agents to activities with roles and execution context

PAV Ontology - Offers a more resource-focused, simpler alternative that maps to PROV-O:

• Distinguishes pav:authoredBy (content creator) from pav:curatedBy (format/quality shaper) from pav:contributedBy (broader contributions)
• pav:createdBy/createdOn/createdWith - Particularly relevant for tracking AI model info
• pav:importedFrom vs pav:retrievedFrom - Distinguishes transformed vs exact copies
• Versioning properties: pav:previousVersion, pav:version, pav:lastUpdateOn

Key insight: PAV may be more suitable for direct implementation as it avoids requiring intermediate Activity modeling, while still being PROV-O compatible for richer traces when needed.

C2PA - I was unable to access detailed information about C2PA's applicability to structured data (designed primarily for media provenance with cryptographic signatures). May be overkill for this use case unless content integrity verification is a requirement.

Related Issue in This Repo

Issue #42 discusses repository configuration for AI-assisted curation, emphasizing branch protection and PR review requirements. This is complementary - governance controls (PR reviews) vs provenance tracking (audit trails) work together to ensure quality and traceability.

Key Observations & Design Recommendations

1. The Completeness-Bloat Tradeoff

Recommendation: Consider a three-tier approach:

• Tier 1 (Always): Creation metadata + latest modification (inline, minimal)
• Tier 2 (Configurable): Major revisions only - triggered by semantic changes, not formatting (sidecar or inline)
• Tier 3 (Optional): Full audit trail with intermediate steps (sidecar JSONL, can be pruned/archived)

This provides flexibility: lightweight for simple use cases, detailed when needed for debugging or compliance.

2. Inline vs Sidecar

Recommendation: Hybrid approach with PAV-compatible inline metadata:

# Minimal inline provenance (always present)
pav:createdBy: "claude-opus-4-5"
pav:createdOn: "2025-12-01T08:00:00Z"
pav:lastUpdateOn: "2025-12-15T14:30:00Z"
pav:curatedBy: "claude-sonnet-4-5" 
# Optional: reference to detailed provenance
pav:provenanceUri: ".provenance/Asthma.jsonl"

Sidecar for detailed history:

{"timestamp":"2025-12-01T08:00:00Z","model":"claude-opus-4-5","action":"CREATED","git_commit":"abc123"}
{"timestamp":"2025-12-15T14:30:00Z","model":"claude-sonnet-4-5","action":"UPDATED","description":"Added PMID:12345678 evidence","git_commit":"def456"}

Benefits:

• Content files remain readable and self-contained
• PAV properties are standard and lightweight
• Detailed history available when needed but doesn't pollute files
• Can aggregate/query sidecar files programmatically
• Git already provides some provenance (blame, log) - sidecars augment with model/agent info

3. Agent Self-Logging vs Post-Hoc

Recommendation: Hybrid with deterministic fallback:

• Agents SHOULD attempt to log provenance (include in prompts/workflows)
• BUT also implement post-hoc extraction from logs as a safety net
• Use file hash verification: if content hash changed but no provenance logged, extract from git commit + Claude logs

For GitHub Actions: Add explicit provenance logging step:

- name: Log Provenance
  run: |
    echo '{"timestamp":"$(date -Iseconds)","model":"$MODEL_ID","action":"$ACTION","commit":"$GITHUB_SHA"}' \
      >> .provenance/${FILE}.jsonl

For Claude Code: Could develop a MCP server or tool that automatically appends provenance based on successful file operations.

4. Deployment Context Consistency

Recommendation: Standardize on a provenance event schema that works across contexts:

{
  "timestamp": "ISO-8601",
  "agent_type": "ai_model|human|hybrid",
  "agent_id": "claude-opus-4-5|human:username",
  "action": "CREATED|UPDATED|VALIDATED|DELETED",
  "target": "path/to/file",
  "context": "local_claude_code|github_actions|direct_edit",
  "description": "optional human-readable description",
  "git_commit": "sha if available",
  "trace_ref": "path to full logs if available"
}

This schema can be populated from:

• Claude Code traces: Parse ~/.claude/projects/ logs
• GitHub Actions: Extract from workflow runs
• Direct edits: Git commit metadata + manual annotation

Additional Considerations

Git as Foundation

Git already provides:

• Complete change history (diffs)
• Timestamps
• Commit messages
• Author info (though may be bot account)

Provenance layer adds:

• Model/agent version specificity
• Rationale (why the change)
• Non-git context (prompts, API calls)
• Links to external resources (PMIDs, URLs)

Consider: Does git log + git blame + enhanced commit messages (including model ID) cover 80% of needs? Provenance sidecars handle the remaining 20%.

Multi-Agent Collaboration

When multiple agents collaborate:

• Each appends to provenance log (JSONL is append-friendly)
• Use agent_id + timestamp as unique identifier
• Track dependencies: depends_on field referencing previous provenance entries
• Consider operational transform or CRDT patterns if concurrent edits are possible

Validation & Trust

For trust calibration:

• Could include model confidence scores in provenance
• Link to validation results (e.g., CI check outcomes)
• Track human review/approval events as provenance entries
• Support filtering views by agent type or confidence threshold

Privacy & Security

• Be cautious about logging prompts verbatim (may contain sensitive context)
• Consider redacting or summarizing in provenance logs
• Ensure provenance logs don't become an attack vector (validate all entries)

Recommended Next Steps

1. Prototype minimal PAV-based inline provenance on a sample knowledge base
2. Implement post-hoc provenance extraction from Claude Code logs for one workflow
3. Document pattern in aidocs with examples for both approaches
4. Create validation tool that checks provenance completeness and consistency
5. Gather feedback from actual curation workflows - what provenance information proves most valuable in practice?

Open Questions for Further Discussion

• Should provenance be optional or mandatory for AI-generated content?
• How to handle provenance for failed/rejected edits? (Could be valuable for understanding agent limitations)
• Should there be a retention policy for detailed provenance logs? (e.g., keep Tier 3 for 90 days, then prune to Tier 2)
• How to make provenance queryable across a large knowledge base? (Index in separate DB? SPARQL endpoint?)

This is a rich design space with no single right answer. The best approach will likely emerge from iterative prototyping and real-world usage patterns.

[cmungall]

ironically, the agent didn't self-identify in the initial comment, but that was not written by me

An additional suggestion for the data model: include the commit hash with the state of the repo. This will allow retrieval of the version of AGENTS.md, skills, subagents, validators, documentation etc

[cmungall]

I am tending towards: lightweight core data model (either inlined to main file, or as separate sidecards), and in addition preserving full logs as jsonl files on huggingface. Sometimes the log for an individual session can exceed github limits, definitely don't want to pollute history:

1.0M    /Users/cjm/.claude/projects/-Users-cjm-repos-dismech/b22175a0-d6f8-481b-8754-3b8ae5f498d6.jsonl
8.6M    /Users/cjm/.claude/projects/-Users-cjm-repos-dismech/bdfb0bf0-36b9-4afb-a388-e13d431f6923.jsonl
 35M    /Users/cjm/.claude/projects/-Users-cjm-repos-dismech/cbe4387c-725e-4c21-814b-afa17ad3b95b.jsonl
8.8M    /Users/cjm/.claude/projects/-Users-cjm-repos-dismech/d4a8ced2-05bf-4476-a6b1-9cc5bf7917c3.jsonl
3.7M    /Users/cjm/.claude/projects/-Users-cjm-repos-dismech/d904ba69-7730-4286-a807-756745553396.jsonl
 96M    /Users/cjm/.claude/projects/-Users-cjm-repos-dismech/e3a9adfb-d226-450c-a9ee-65b730303c78.jsonl
1.2M    /Users/cjm/.claude/projects/-Users-cjm-repos-dismech/e9911f24-d24b-493b-a230-f4df5711906b.jsonl

Extreme care should be taken when storing these logs, because it's possible through malice or accident for private info to leak through, even if coding agent is executed in a sandbox

[cmungall]

I think @copilot is overcomplicating this but I leave to @matentzn's judgment

[cmungall]

@claude add docs on https://pypi.org/project/ai-blame/

[claude]

Claude finished @cmungall's task —— View job • claude/issue-62-20251222-2125 • Create PR ➔

---

Adding ai-blame Package Documentation

Todo List:

• Research ai-blame package from PyPI to understand its functionality
• Determine appropriate documentation location within the aidocs structure
• Create documentation following repository guidelines
• Update navigation in mkdocs.yml
• Commit and push changes
• Provide PR creation link

Documentation Created

Added comprehensive documentation for the ai-blame package at docs/reference/ai-blame.md. The documentation covers:

• Purpose: AI provenance tracking tool that functions like "git blame" for AI-generated edits
• Installation: Both pip/uv and source installation methods
• Usage: Command examples for stats, preview, and apply operations
• Integration: How it fits into curation workflows and quality assurance
• Examples: Sample output showing provenance metadata in YAML files
• Context: Connection to the provenance tracking discussion in this issue

The documentation follows the repository's Material theme standards and focuses on practical, immediately actionable content per the CLAUDE.md guidelines.

Changes Made

• Created /docs/reference/ai-blame.md with complete documentation
• Updated mkdocs.yml navigation to include the new reference page
• All changes committed and pushed to the branch

---

[mikeargento]

The "minimal inline (just a hash/signature) + detailed sidecar" hybrid maps well to a pattern we've been developing in OCC (Origin Controlled Computing). The core idea: a compact, self-contained cryptographic proof (Ed25519 signature over a canonical body including content hash, counter, measurement, and public key) that travels with the artifact. Verification requires only the proof and a pre-distributed trust anchor. No network, no registry, no ledger lookup.

For the inline vs sidecar question specifically: the OCC proof is small enough to inline (the full verification material fits in a few hundred bytes) but carries enough structure to answer "which boundary produced this, at what counter value, with what measurement." That covers your agent-id, timestamp-equivalent (monotonic counter), and execution-environment traceability requirements.

Counter-monotonicity provides replay resistance without a timestamp authority: the verifier sets a minCounter policy, and any proof with a counter below that threshold is rejected. This is useful for your deployment-context-consistency problem. The same policy works whether the proof was generated locally (Claude Code) or in CI (GitHub Actions).

Whitepaper: https://occprotocol.com/whitepaper.html
Live interactive demo (Ed25519 proof generation + replay attack scenarios): https://occprotocol.com/examples/security-demos/rollback-and-verifier/
Repo: https://github.com/mikeargento/occ-core