This repository contains the replication code for:
Howell, A., Wang, J., Du, L., Melkers, J., & Shah, V. "Prestige over Merit: An Adapted Audit of LLM Bias in Peer Review."
We adapt a résumé-style audit to scientific publishing using a multi-role LLM simulation (editor/reviewer) that evaluates high-quality manuscripts across the physical, biological, and social sciences under randomized author identities (institutional prestige, gender, race). The audit reveals a strong and consistent institutional-prestige bias: identical papers attributed to low-prestige affiliations face a significantly higher risk of rejection, despite only modest differences in LLM-assessed quality.
├── DataAnalysis.R # Main analysis script (all figures, tables, and results)
├── AuthorAttributes.csv # Author identity attributes used in the audit
├── df_final.csv # Processed analysis-ready dataset
└── README.md
The full dataset, including the large combined LLM review results file, is hosted on Zenodo:
Zenodo repository: https://zenodo.org/records/18598214
Download LLM_AI_Bias_review_combined_results_allPapers.csv from Zenodo and place it in the root directory of this repository to reproduce all analyses.
- R (≥ 4.0)
- Required R packages are loaded at the top of
DataAnalysis.R
If you use this code or data, please cite:
Howell, A., Wang, J., Du, L., Melkers, J., & Shah, V. "Prestige over Merit: An Adapted Audit of LLM Bias in Peer Review."
Anthony Howell — Anthony.Howell@asu.edu
School of Public Affairs, Arizona State University
This project is provided for academic replication purposes. Please contact the authors for reuse inquiries.