Skip to content

Vertical: fact-checking claims as an annotation type #3

Description

@ryanwi

Context

The spec already supports arbitrary type values and an open data extension bag, so fact-checking annotations are mechanically expressible today. This issue is about whether to bless claim as a recommended type with a conventional data schema, so producers and consumers in the fact-checking vertical interoperate without inventing their own conventions.

Prior art

Setty & Becker, Annotation Tool and Dataset for Fact-Checking Podcasts (TheWebConf 2025, arXiv:2502.01402, factiverse/factcheck-podcasts) — annotate-during-playback workflow for claim detection, claim spans, and stance/verdict labels on news and health podcasts. Validates the UX and gives us a reference schema to compare against.

Sketch

Add claim to the Recommended Entity Types table:

Type Description Example
claim A check-worthy factual assertion made in the audio "The E30 M3 made 192hp"

Conventional data fields for claim annotations:

{
  "startTime": 412.0,
  "endTime": 418.5,
  "type": "claim",
  "quote": "the E30 M3 made 192 horsepower",
  "data": {
    "verdict": "false",           // true | false | misleading | unverified | disputed
    "correction": "The US-spec E30 M3 made 192hp; the Euro-spec made 200hp.",
    "sources": [
      { "url": "https://...", "title": "BMW M3 E30 specifications", "publisher": "BMW Group Classic" }
    ],
    "checkedBy": "human",         // human | ai | hybrid
    "checkedAt": "2026-05-17"
  }
}

The top-level confidence and source fields already cover detection confidence and producer attribution; data.verdict covers the verification outcome separately.

Why bless it vs. leave to producers

  • The fact-checking community is small enough that a single conventional schema is realistic.
  • Verdict vocabularies vary widely (PolitiFact's six-point scale, Snopes, IFCN signatories) — picking a minimal core (true/false/misleading/unverified/disputed) and allowing producer-specific values in data.verdictDetail keeps things interoperable without flattening nuance.
  • Without a recommendation, every producer invents data.factCheck vs data.verification vs data.claim_status and consumers can't render generically.

Out of scope for v1

  • Claim deduplication / cross-episode claim IDs (could reuse canonicalId but worth a separate think).
  • Rebuttal / counter-claim relationships between annotations.
  • Multi-source aggregation (e.g., consensus verdict across multiple fact-checkers).

Decision needed

  1. Add claim to recommended types in 1.1, or wait for a producer in the wild?
  2. If yes — minimum verdict vocabulary, or leave verdict open and just standardize the field names?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions