Harden verify.py + fix audit false-positive; sync docs

verify.py: add ERROR verdict kept distinct from NOT-FOUND so a transient
rate-limit/network failure can't masquerade as "does not exist" and drop a
real paper; prefetch arXiv ids in batches (id_list) to avoid the 429 bans
that caused 8 false NOT-FOUNDs on a ~90-paper run; accept int expect_year
(was a mid-run crash); degrade a malformed row to ERROR instead of aborting
the batch.

references.py: fix audit venue check false-flagging titles that end in ?/!
as "empty venue" (the title/venue splitter only recognized ". ").

families.py: accept assignment values case-insensitively and by display
name, so re-running off rows.json's stamped `family` field works.

docs: sync PLAYBOOK.md (Phase 3 verdicts + lessons incl. the strict-APA
sentence-case decision), tools/README.md, docs/phases.md, docs/tools.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: jackgallant

PLAYBOOK.md

@@ -203,6 +203,16 @@ In a previous run, the search agent fabricated 5 author lists, reversed
 one paper's conclusion, and invented a bioRxiv DOI that didn't exist.
 About 1 in 4 citations had errors. **Always verify before adding.**
 
+`verify.py` returns one verdict per citation: **OK**, **MISMATCH** (author/year),
+**NOT-FOUND**, or **ERROR**. NOT-FOUND and ERROR are NOT the same and must be
+handled differently: NOT-FOUND means every lookup completed and none matched
+(chase it down — likely fabricated); ERROR means a lookup could not complete
+(rate-limit / network), so **re-run those** rather than treating them as missing.
+On a big run this matters — arXiv rate-limits hard, so the tool prefetches all
+arXiv ids in batches (many per `id_list` call); a genuinely real preprint that
+would otherwise 429 into a false NOT-FOUND now comes back OK (or, if the batch
+still fails, ERROR to re-run).
+
 For each paper the agent returned:
 
 **3a. If a PMCID was given:** call NCBI esummary
@@ -637,9 +647,11 @@ theme:
    two-tier criteria (foundational vs recent), a capped target (~30–40), multiple
    query angles. One agent per theme.
 2. **Verify EVERY citation (Phase 3)** with `tools/verify.py`.
-   It now resolves arXiv/conference papers against the arXiv API — so a NOT-FOUND
-   is a real failure to investigate, never "an arXiv paper, skip it." Expect ~1
-   in 4 to need a fix (fabricated author lists, wrong arXiv ids, garbage DOIs).
+   It resolves arXiv/conference papers against the arXiv API (batched) — so a
+   NOT-FOUND is a real failure to investigate, never "an arXiv paper, skip it."
+   Re-run any ERROR verdicts (transient rate-limit/network — distinct from
+   NOT-FOUND). Expect ~1 in 4 to need a fix (fabricated author lists, wrong arXiv
+   ids, garbage DOIs).
 3. **Citation counts (Phase 5b)** with `tools/citations.py` for every field
    paper — bibliographies always carry counts.
 4. **Consolidate with two guarded steps (fail loud):**
@@ -682,6 +694,28 @@ that with the priority audit (contract rule 5; Phase 7).
   attention paper was actually Foster et al.; two Jain & Huth arXiv ids pointed
   at unrelated papers — all caught only because every citation, preprint
   included, was verified.)
+- **NOT-FOUND ≠ ERROR — a transient failure must never read as "missing."** On a
+  ~90-paper world_models run, 8 real preprints came back NOT-FOUND purely because
+  arXiv rate-limited (429) a per-paper loop into a temporary ban; trusting that
+  verdict would have dropped real papers. `verify.py` now (a) prefetches arXiv ids
+  in **batches** (many per `id_list` call, ~3s apart) so the ban doesn't happen,
+  and (b) reports a distinct **ERROR** verdict when a lookup can't complete, kept
+  separate from NOT-FOUND. Re-run ERRORs; only NOT-FOUND means "does not exist."
+  Pass `expect_year` as a string OR int — either is accepted (a JSON int no longer
+  crashes the run), and one malformed row degrades to ERROR instead of aborting.
+- **Reference titles are strict APA-7 sentence case; DOIs are the ground truth
+  for *location*, the APA string is just the bibliography display.** CrossRef and
+  arXiv return titles in inconsistent casing (arXiv and many publishers use Title
+  Case; Nature deposits sentence case), so `references.py` normalizes ALL-CAPS
+  titles but deliberately does NOT auto-transform Title Case → sentence case:
+  correct sentence-casing needs the proper-noun judgment APA bakes in (`Bayesian`,
+  `Atari`, `Weber`, `Tolman-Eichenbaum` stay capitalized; `Active`, `World`,
+  `Model` lowercase), and a mechanical caser silently mis-cases proper nouns —
+  which the audit can't catch. So after canon, **sentence-case new preprint titles
+  in a reviewed pass** (auto-protect all-caps acronyms, camelCase/digit model
+  names, and hyphen parts; keep a small proper-noun allowlist; eyeball every
+  changed title, e.g. a product name like `Matrix-Game`). This is a post-canon
+  hand-fix like the mojibake and compound-surname fixes below.
 
 ### On contextualizing a lab review (lab mode L4c)
 - **The outward search is a FULL topic-mode review, not a "context" add-on.**
@@ -824,7 +858,7 @@ outputs JSON/files. Run `python3 tools/<script>.py --help` for flags.
 
 | Script | Purpose |
 |--------|---------|
-| `tools/verify.py` | Verify a list of citations via PMC/PubMed/CrossRef + arXiv. Reports mismatches. |
+| `tools/verify.py` | Verify a list of citations via PMC/PubMed/CrossRef + arXiv (arXiv ids batched). Verdicts OK / MISMATCH / NOT-FOUND / ERROR — re-run ERROR, chase NOT-FOUND. |
 | `tools/references.py` | Phase 3f. Rebuild every `apa` from the verified DOI (CrossRef) or arXiv id into canonical APA-7; `--audit` gates the build (exit 1 on any defect). Both modes. |
 | `tools/citations.py` | Phase 5b. Fetch per-paper citation counts from OpenAlex (primary) + Semantic Scholar (secondary) by DOI. Google Scholar is not usable (no API / CAPTCHA). |
 | `tools/families.py` | Phase 6b. Validate an (agent-proposed, human-approved) family taxonomy, stamp `family` onto rows, emit `families.json` + `families.md`. `--digest` prints a corpus digest for the proposal step. |

docs/phases.md

@@ -65,8 +65,14 @@ python3 ../tools/verify.py --citations rows.json --out verify_report.json
     - inverted or misattributed findings,
     - entirely fabricated author lists for papers that genuinely exist.
 
-The report marks each citation `OK` / `MISMATCH` / `NOT-FOUND` so you (or the
-agent) can fix or drop it before anything downstream depends on it.
+The report marks each citation `OK` / `MISMATCH` / `NOT-FOUND` / `ERROR` so you
+(or the agent) can fix or drop it before anything downstream depends on it.
+`NOT-FOUND` and `ERROR` are **not** interchangeable: `NOT-FOUND` means every
+lookup completed and nothing matched (chase it — likely fabricated), while
+`ERROR` means a lookup could not complete (rate-limit / network) and must be
+**re-run**. To keep arXiv's aggressive rate-limiting from turning real preprints
+into false `NOT-FOUND`s, arXiv ids are prefetched in batches (`id_list`, many per
+call). `expect_year` may be a string or an int.
 
 ---
 

docs/tools.md

@@ -10,7 +10,7 @@ JSON I/O, DOI/arXiv parsing, APA building). Run any with `--help`.
 
 | Script | Phase | Purpose |
 |---|---|---|
-| [`verify.py`](https://github.com/gallantlab/literature-review-toolkit/blob/main/tools/verify.py) | 3 | Verify every citation via PMC/PubMed/CrossRef **+ the arXiv API** (preprints get a real verdict, not a misleading `NOT-FOUND`). Catches ~25% search-agent fabrications. |
+| [`verify.py`](https://github.com/gallantlab/literature-review-toolkit/blob/main/tools/verify.py) | 3 | Verify every citation via PMC/PubMed/CrossRef **+ the arXiv API** (arXiv ids batched to dodge rate-limit bans). Verdicts `OK` / `MISMATCH` / `NOT-FOUND` / `ERROR` — re-run `ERROR` (transient), chase `NOT-FOUND` (likely fake). Catches ~25% search-agent fabrications. |
 | [`references.py`](https://github.com/gallantlab/literature-review-toolkit/blob/main/tools/references.py) | 3f | Rebuild every reference from its verified DOI/arXiv id into canonical APA-7 (full authors, particles, casing, real venue incl. bioRxiv/PsyArXiv). `--audit` is a **hard gate**. Both modes. |
 | [`spreadsheet.py`](https://github.com/gallantlab/literature-review-toolkit/blob/main/tools/spreadsheet.py) | 5 | Build/rebuild the `.xlsx` from the accumulated JSON rows; auto-adds `Cite` / `Family` columns when present. |
 | [`citations.py`](https://github.com/gallantlab/literature-review-toolkit/blob/main/tools/citations.py) | 5b | Per-paper citation counts from OpenAlex (primary) + Semantic Scholar by DOI, with undercount reconciliation. |

tools/README.md

@@ -17,11 +17,17 @@ python3 tools/verify.py --citations cits.json --out report.json --email you@inst
 ```
 
 `cits.json` per item: `{label, pmcid?, pmid?, doi?, arxiv?, title?,
-expect_first_author?, expect_year?}`. arXiv papers (an `arxiv` id or a
-`10.48550/arXiv.<id>` DOI) route to the arXiv API first; otherwise looks up via
-PMC, then PubMed, then CrossRef, then title-search. Verdict per item: `OK`,
-`MISMATCH`, `NOT-FOUND`. **A NOT-FOUND is a real failure to chase down — never
-wave through an arXiv/conference paper as unverifiable.**
+expect_first_author?, expect_year?}` (`expect_year` may be a string or int).
+arXiv papers (an `arxiv` id or a `10.48550/arXiv.<id>` DOI) route to the arXiv
+API first; otherwise looks up via PMC, then PubMed, then CrossRef, then
+title-search. arXiv ids are **prefetched in batches** (`id_list`, many per call)
+because the API rate-limits a per-paper loop into a temporary ban. Verdict per
+item: `OK`, `MISMATCH`, `NOT-FOUND`, or `ERROR`. **`NOT-FOUND` and `ERROR` are
+different and must be handled differently:** NOT-FOUND = every lookup completed,
+none matched (chase it down — likely fabricated); ERROR = a lookup could not
+complete (rate-limit / network), so **re-run those** — never treat a throttled
+fetch as "does not exist." One malformed row degrades to ERROR rather than
+aborting the whole batch.
 
 ## `references.py` — canonical reference builder (Phase 3f)
 
@@ -128,7 +134,10 @@ python3 tools/families.py --rows rows.json --assign families_input.json --out fa
 ```
 
 `families_input.json`: `{principle, families:[{key,name,claim,lineage}],
-assignments:{ref:key}}`.
+assignments:{ref:key}}`. Assignment values are accepted case-insensitively and
+by display **name** as well as `key`, so you can re-run straight off the `family`
+field `spreadsheet.py` stamped into `rows.json` (which holds the display name,
+e.g. `"Infer"`) without first lowercasing it back to the key.
 
 ## `families_figure.py` — interactive HTML lineage figure (Phase 6b)
 

tools/families.py

@@ -92,6 +92,17 @@ def main():
                  "Topic column).")
     keyset, name_of = set(keys), {f["key"]: f["name"] for f in families}
 
+    # Accept assignment values case-insensitively and by display name, not just
+    # the exact lowercase key: rows.json stores the display name ("Infer"), so
+    # re-running families straight off the stamped `family` field would otherwise
+    # fail with "unknown family keys". Anything that doesn't resolve is left as-is
+    # and caught by the badkey check below.
+    resolve = {}
+    for f in families:
+        resolve[str(f["key"]).strip().lower()] = f["key"]
+        resolve[str(f["name"]).strip().lower()] = f["key"]
+    assign = {r: resolve.get(str(v).strip().lower(), v) for r, v in assign.items()}
+
     refs = [r["ref"] for r in rows]
     refset = set(refs)
     missing = [r for r in refs if r not in assign]

tools/references.py

@@ -144,7 +144,11 @@ def audit(apa, has_source):
     # Structure is "Authors (YEAR). Title. Venue...."; drop the year-paren and the
     # title sentence (its trailing ". ") and require something non-empty to remain.
     after_year = apa.split(").", 1)[1] if ")." in apa else ""
-    venue_part = after_year.split(". ", 1)[1].strip(" .") if ". " in after_year else ""
+    # The title/venue separator is the title's terminal sentence punctuation +
+    # space — usually ". " but a title ending in a question or exclamation mark
+    # ends with "? "/"! " instead, so split on any of [.?!] followed by space.
+    title_split = re.split(r"[.?!]\s", after_year, maxsplit=1)
+    venue_part = title_split[1].strip(" .") if len(title_split) > 1 else ""
     if has_source and not venue_part:
         defects.append("empty venue")
     return defects, notes