Skip to content

cordademocracy/ResPolitica

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ResPolitica

🤗 Hugging Face Dataset

A unified dataset for political information, including from Voting Advice Applications (VAAs)


Information Mediation - Tasks & Types

ResPolitica currently covers party-position mediation ($T_{\text{party-pos}}$): given a party and a policy statement, retrieve the party's stance and rationale. This is one task type within the broader political information mediation family:

Task Type Example
Party-Position (current scope) "What is [Party]'s position on rent control?"
Navigation "Which parties support rent control?"
Comparison "How do Party A and Party B differ on migration policy?"
Issue Mapping "What are the main political conflicts around housing policy?"

Future versions will expand beyond VAA data to cover additional political information types (e.g. candidate information, debates, manifestos candidate interviews, parliamentary votes) and mediation tasks as grounding data and evaluation criteria are developed.


Data Sources

Source Coverage
🇩🇪 Wahl-O-Mat (qual-o-mat-data) DE federal + state elections, 2017–2025
🇳🇱 StemWijzer NL TK2025 only
🇪🇺 EUANDI 2024 EU Parliament 2024 (DE, ES, FR, GR, IT + EP groups)

Note: StemWijzer data for TK2017, TK2021, and TK2023 is excluded for copyright reasons.


Dataset Files

All files under datasets/processed/.

File Description Rows
respolitica_unified.parquet / .csv All observations, all sources, all fields 12,584
respolitica_unified_federal.parquet / .csv Federal elections only 4,444
respolitica_unified_states.parquet / .csv State elections only 7,030
respolitica_normalized/elections.parquet Election registry 19
respolitica_normalized/parties_canonical.parquet Global party registry 143
respolitica_normalized/parties_election.parquet Party × election appearances 344
respolitica_normalized/statements.parquet Unique statements per election 666
respolitica_normalized/observations.parquet Fact table (FKs + stance + rationale) 12,584

The respolitica_unified* files are flat denormalized exports. The respolitica_normalized/ tables are the relational decomposition of the same corpus.


Schema — respolitica_unified

Column Type Description
observation_id str Unique row identifier
source_dataset str qual-o-mat, euandi, or stemwijzer
election_id str Slug, e.g. bundestagswahl2025, nl_tk2025
election_date date ISO date
election_level str federal, state, or european
country_iso str ISO 3166-1 alpha-2
party_id str Canonical party slug
party_name_short str Short name
statement_id str Unique statement identifier
statement_text str Statement in source language
statement_text_en str English translation
stance_label str Agree, Neutral, or Disagree
stance_numeric float64 1.0 / 0.0 / -1.0
rationale_text str Party rationale in source language
rationale_text_en str English translation
quality_flag str e.g. missing_rationale, clean
has_rationale bool False when rationale is blank or placeholder

Installation

git clone <repo-url>
cd ResPolitica
uv venv && uv sync
# Dataset summary
python scripts/respolitica_data.py info

# Validate data quality
python scripts/respolitica_data.py validate

# Query by party or election
python scripts/respolitica_data.py query --election bundestagswahl2025 --party CDU
python scripts/respolitica_data.py query --country nl --format json

Generating the Full Dataset

The pipeline parses raw VAA source files and produces the processed outputs above. It also generates the Information Environments (IEs) used in the Polistemics benchmark.

# Fetch upstream source data (Qual-o-Mat, EUANDI)
python scripts/respolitica_data.py fetch --source qualomat
python scripts/respolitica_data.py fetch --source euandi

# Parse and unify all sources
python scripts/respolitica_data.py prepare

Benchmark construction (phases 1–3) generates respolitica_augmentations.jsonl and respolitica_benchmark.jsonl:

# First generate the scope CSV (required by assemble):
python scripts/pipeline.py --stages scope
python scripts/benchmark_cli.py standardize
python scripts/benchmark_cli.py standardize-phase2
python scripts/benchmark_cli.py assemble
python scripts/benchmark_cli.py validate-output

Set benchmark.standardization.model in config.yaml to a valid LiteLLM model string before running.


Credits & Licenses

Wahl-O-Mat / qual-o-mat-data

StemWijzer

  • Source: ProDemos — House for Democracy and the Rule of Law, Netherlands
  • Original tool: stemwijzer.nl

EUANDI 2024


Citation

@misc{polistemics2026,
  title   = {Polistemics: Evaluating LLMs as Information Mediators in Politics \& Elections},
  author  = {Peters, Baran},
  year    = {2026},
  note    = {Preprint}
}

License

See LICENSE.

About

A unified dataset for political information mediation, including from Voting Advice Applications (Wahl-O-Mat, StemWijzer) and EUANDI. Also behind the Polistemics Benchmark ™

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages