Goldfranks GROBID Reference Extraction

A Python pipeline for extracting and analyzing references from academic PDFs using GROBID.

Features

• 📄 Extract structured references from PDFs using GROBID
• 🔍 Automatic DOI resolution via CrossRef API
• 📌 Generate annotated PDFs with highlighted references
• 🔄 Checkpoint system for resuming interrupted processing
• 🚀 Parallel processing support
• 🌐 Full Unicode support

Quick Start

1. Start GROBID Docker:

   docker run --rm --init --ulimit core=0 -p 8070:8070 grobid/grobid:0.8.2

2. Install dependencies:

   pip install requests pandas tqdm lxml PyMuPDF

3. Run the pipeline:

   cd reference_extraction/scripts
   python master_pipeline.py --test  # Test mode
   python master_pipeline.py         # Process all PDFs

Documentation

• See reference_extraction/README.md for detailed usage
• See CLAUDE.md for development guidance

License

This project is for personal research use only.