ITEQS Paper Reviewer

An automated research paper review tool for the ITEQS workshop, powered by a local LLM via Ollama. The script converts PDF pages to images, reviews them in chunks, and synthesizes a structured final review.

---

HOW IT WORKS

1. The input PDF is split into page chunks and rendered as images (via PyMuPDF)
2. Each chunk is sent to the LLM with ITEQS-specific scope, topics, and review criteria
3. All chunk reviews are collected and synthesized into a single structured final review
4. The final review is saved as a .txt file alongside the generated images

---

REQUIREMENTS

• Python 3.9+
• Ollama installed and running locally (https://ollama.com)
• The target model pulled in Ollama (default: gemma3:27b)

Pull the default model:

  ollama pull gemma3:27b

---

INSTALLATION

1. Clone the repository

2. Create and activate a virtual environment (recommended)

   python -m venv .venv source .venv/bin/activate # Linux / macOS .venv\Scripts\activate # Windows

3. Install dependencies

   pip install -r requirements.txt

---

USAGE

python review_paper.py -p <path/to/paper.pdf> [options]

ARGUMENTS

-p, --paper (required) Path to the PDF file to review -c, --chunk-size (optional) Number of pages per chunk Default: 10 -o, --output (optional) Root folder for images and reviews Default: ../output_gemma_27b -m, --model (optional) Ollama model to use Default: gemma3:27b

EXAMPLES

Minimal — review a single paper with defaults:

  python review_paper.py -p paper1.pdf

Custom chunk size:

  python review_paper.py -p paper1.pdf -c 5

Full override:

  python review_paper.py -p ../papers/paper1.pdf -c 5 -o ../my_output -m gemma3:27b

---

OUTPUT STRUCTURE

output_gemma_27b/ └── paper1/ ├── chunk_1_10/ │ ├── page_1.jpg │ └── ... ├── chunk_11_20/ │ ├── page_11.jpg │ └── ... └── paper1_review.txt <-- final synthesized review

---

REVIEW CRITERIA

Each chunk is evaluated on:

- Summary                  What the chunk covers
- Relevance to ITEQS       Alignment with EFP testing scope
- Novelty                  Originality of contributions
- Technical soundness      Correctness and rigor
- Evaluation quality       Validity of experiments
- Strengths / Weaknesses
- Missing experiments or baselines
- Language quality         Grammar, spelling, clarity

The final synthesis produces:

- Overall Summary
- Strengths and Weaknesses
- Grammar / Language issues
- Missing experiments or claims
- Final Recommendation:
    Accept / Weak Accept / Borderline / Weak Reject / Reject

---

DEPENDENCIES

Package Version Purpose

---

PyMuPDF 1.27.2.2 PDF parsing and page rendering ollama 0.6.1 LLM inference via local Ollama server pillow 12.1.1 Image handling httpx 0.28.1 HTTP client for Ollama API pydantic 2.12.5 Data validation

Full pinned versions are listed in requirements.txt.

---

NOTES

• Processing time depends on the number of pages, chunk size, and hardware available for Ollama.

• Pages are rendered at 120 DPI by default. Increase this value in split_pdf_to_images() for higher fidelity at the cost of speed.

• The LLM is instructed to ignore any prompt injection embedded within paper images.