historical-hindex

Compute a researcher's h-index as it would have stood at the end of a given year, using Google Scholar data.

Scholar shows each paper's per-year citation histogram but doesn't expose a "h-index as of year Y" view. This tool scrapes the author profile and each paper's yearly citation graph, then re-runs the h-index computation using only citations accumulated up to the chosen cutoff year.

How it works

Two browser-driven phases — no API key, no login:

1. Profile scrape — load each author's Scholar profile and extract the full publication list (title, venue, year, current citations, citation_id) plus the year-by-year total citation counts.
2. Per-paper scrape — for each paper published in or before the cutoff year, load its Scholar page and extract the citations-per-year histogram.

Output lands in reports/<Name>_<year>.json, one file per author. Writes are incremental (after every paper), so the pipeline is resumable — re-run the same command to pick up after an interruption. At the end of the run, an Excel workbook is auto-generated (publications.xlsx).

Setup

pip install -r requirements.txt

Requires a recent Chrome installation. Selenium 4.20+ auto-downloads the matching chromedriver via Selenium Manager, so no manual driver setup.

Usage

Create an authors.json (see authors_example.json):

[
  {"name": "Sandeep P Chinchali", "scholar_id": "262ASa4AAAAJ", "year": 2024}
]

scholar_id is the user=... parameter from a Scholar profile URL. year is the cutoff year for that author.

Then:

python historical_hindex.py --input authors.json              # both phases, then Excel
python historical_hindex.py --input authors.json --phase 1    # profile only
python historical_hindex.py --input authors.json --phase 2    # per-paper only
python historical_hindex.py --input authors.json --fetch-all  # also fetch per-year citations for post-cutoff papers
python historical_hindex.py --input authors.json --visible    # show the browser (default: headless)
python historical_hindex.py --input authors.json --excel-out "" # skip Excel generation

Single-author:

python historical_hindex.py --author "Sandeep P Chinchali" --scholar-id 262ASa4AAAAJ --year 2024

If Scholar blocks you

Scholar rate-limits scrapers aggressively — expect an "unusual traffic" block roughly every 50–100 paper pages. When that happens, the script detects the block, saves progress, and prompts:

BLOCKED: Google Scholar is showing 'unusual traffic'.
Please switch your VPN / network node, then press ENTER to retry.

Switch nodes, press ENTER, and it retries the same request. Or Ctrl+C and re-run later to resume.

Output

reports/<Name>_<year>.json

One per author. Key fields:

• current — h-index and total citations as of now
• at_cutoff — h-index and total citations as of cutoff_year
• date_accessed — date the data was scraped
• publications[] — one entry per paper with title, venue, year, total_cites_now, cites_at_cutoff, cites_per_year

reports/summary.json aggregates all authors.

publications.xlsx

Auto-generated after every run. Two sheets per author:

• Summary — profile metrics, date accessed, total citations by year
• Papers — every publication with title, venue, year, total cites, cites at cutoff, status, and one column per citation year

Papers published after the cutoff year (and not fetched unless you pass --fetch-all) are explicitly labelled not fetched (post-cutoff), with only their total citation count shown.

Files

• historical_hindex.py — main pipeline (scrape + compute + export)
• excel_report.py — Excel report generator (runnable standalone)
• authors.json — your input list
• authors_example.json — example input
• reports/ — generated JSON reports
• publications.xlsx — generated Excel workbook