feat(bench): add COLOR DIMACS instances (anna, jean, queen8_12)

arXiv-2409.02135v2 Table 6 evaluates 12 specific Graph Coloring
instances. Until now QQA4CO shipped only 9 of them through the
procedural Mycielski + square-queen generators, leaving 3 real-world
Knuth / non-square Queen graphs uncovered.

- scripts/generate_coloring_instances.py now fetches anna.col,
  jean.col and queen8_12.col from Trick's canonical DIMACS COLOR
  mirror, parses them into NetworkX graphs, and emits them under
  data/coloring/dimacs/ with the paper's target num_colors (11, 10,
  12). --skip-dimacs keeps the pure-offline rebuild path intact.
- data/coloring/README.md documents the Table-6 mapping explicitly so
  the coverage contract is auditable.
- root README coverage matrix is restructured to §5.1-§5.5 with
  accurate Table references, an explicit ER-[9000-11000] best_known
  caveat, and the new coloring-dimacs suite.

Author: Yuma Ichikawa

README.md

@@ -466,25 +466,41 @@ The original DISCS-only path (`make bench-discs-setup`,
 ### arXiv-2409.02135v2 (PQQA) benchmark coverage matrix
 
 Every benchmark from PQQA (Ichikawa and Arai 2024, arXiv:2409.02135v2,
-§5.1–§5.4) is now mirrored on the companion HF dataset
+§5.1–§5.5) is now mirrored on the companion HF dataset
 [`Yuma-Ichikawa/qqa4co-bench`](https://huggingface.co/datasets/Yuma-Ichikawa/qqa4co-bench)
 and reachable via `qqa bench-run --suite <id>` from the installed
-`qqa` CLI:
+`qqa` CLI. Rows map one-to-one to the problem tables in §5 of the paper:
 
-| Paper section | Problem family | HF subset (`Yuma-Ichikawa/qqa4co-bench`) | `--suite <id>` | One-line command |
+| Paper section / table | Problem family | HF subset (`Yuma-Ichikawa/qqa4co-bench`) | `--suite <id>` | One-line command |
 | --- | --- | --- | --- | --- |
-| §5.1 | MIS on SATLIB (500 instances) | `mis/satlib/uf` | `mis-satlib-uf` | `qqa bench-run --suite mis-satlib-uf --instances 500` |
-| §5.1 | MIS on Erdos-Renyi (few / more budgets) | `mis/er/er_*` | `mis-er-small`, `mis-er-large` | `qqa bench-run --suite mis-er-small --instances 6` |
-| §5.1 | MIS on RRG `d=20, n=10^{4,5,6}` | `mis-rrg/d20_n{10000,100000,1000000}` | `mis-rrg-d20_n10000` (etc.) | `qqa bench-run --suite mis-rrg-d20_n10000 --instances 5` |
-| §5.1 | MIS on RRG `d=100, n=10^{4,5,6}` | `mis-rrg/d100_n{10000,100000,1000000}` | `mis-rrg-d100_n10000` (etc.) | `qqa bench-run --suite mis-rrg-d100_n10000 --instances 5` |
-| §5.2 | Max Clique (DISCS RB) | `maxclique/rb` | `maxclique-rb` | `qqa bench-run --suite maxclique-rb --instances 500` |
-| §5.2 | Max Clique (Twitter social) | `maxclique/twitter` | `maxclique-twitter` | `qqa bench-run --suite maxclique-twitter` |
-| §5.3 | Max Cut (DISCS Barabasi-Albert) | `maxcut/*` | `maxcut-ba-*`, `maxcut-er-*` | `qqa bench-run --suite maxcut-ba --instances 100` |
-| §5.3 | Max Cut (G-set, Helmberg-Rendl 2000) | `gset/standard` | `gset` | `qqa bench-run --suite gset --instances 71` |
-| §5.3 | Balanced k-way graph partition | `balanced/nets` | `balanced-partition-nets-*` | `qqa bench-run --suite balanced-partition-nets-MNIST` |
-| §5.3 | Normalized Cut | `normcut/nets` | `normcut-nets-*` | `qqa bench-run --suite normcut-nets-MNIST` |
-| §5.4 | 3D Edwards-Anderson (Gauss / bimodal, L=4,6,8) | `ea3d/*` | `ea3d-gauss-L4` (etc.) | `qqa bench-run --suite ea3d-gauss-L8 --instances 50` |
-| §5.4 | Graph Coloring (Myciel / queen / COLOR) | `coloring/*` | `coloring-myciel`, `coloring-queen` | `qqa bench-run --suite coloring-myciel` |
+| §5.1 / Table 1 | MIS on SATLIB (500 instances) | `mis/satlib/uf` | `mis-satlib-uf` | `qqa bench-run --suite mis-satlib-uf --instances 500` |
+| §5.1 / Table 1 | MIS on ER-[700-800] (128 instances) | `mis/er/800` | `mis-er-small` | `qqa bench-run --suite mis-er-small --instances 128` |
+| §5.1 / Table 1 | MIS on ER-[9000-11000] (16 instances, note [a]) | `mis/er/10k` | `mis-er-large` | `qqa bench-run --suite mis-er-large --instances 16` |
+| §5.1 / Table 2 | MIS on RRG `d=20, n=10^{4,5,6}` | `mis-rrg/d20_n{10000,100000,1000000}` | `mis-rrg-d20_n10000` (etc.) | `qqa bench-run --suite mis-rrg-d20_n10000 --instances 5` |
+| §5.1 / Table 2 | MIS on RRG `d=100, n=10^{4,5,6}` | `mis-rrg/d100_n{10000,100000,1000000}` | `mis-rrg-d100_n10000` (etc.) | `qqa bench-run --suite mis-rrg-d100_n10000 --instances 5` |
+| §5.2 / Table 3 | Max Clique (DISCS RB) | `maxclique/rb` | `maxclique-rb` | `qqa bench-run --suite maxclique-rb --instances 500` |
+| §5.2 / Table 3 | Max Clique (Twitter social) | `maxclique/twitter` | `maxclique-twitter` | `qqa bench-run --suite maxclique-twitter` |
+| §5.3 / Fig. 3 | Max Cut (ER, 7 sizes from 16 to 1,100 nodes) | `maxcut/er/er-0.15-n-*` | `maxcut-er-*` | `qqa bench-run --suite maxcut-er-n-128-150 --instances 100` |
+| §5.3 / Fig. 3 | Max Cut (BA, 7 sizes from 16 to 1,100 nodes) | `maxcut/ba/ba-4-n-*` | `maxcut-ba-*` | `qqa bench-run --suite maxcut-ba-n-128-150 --instances 100` |
+| §5.3 / Table 4 | Max Cut (Optsicom, 10 real-world graphs) | `maxcut/optsicom/b` | `maxcut-optsicom` | `qqa bench-run --suite maxcut-optsicom --instances 10` |
+| §5.4 / Table 5 | Balanced k-way graph partition (VGG, MNIST-conv, ResNet, AlexNet, Inception-v3) | `normcut/nets/*` | `balanced-partition-nets-*` | `qqa bench-run --suite balanced-partition-nets-INCEPTION` |
+| §5.5 / Table 6 | Graph Coloring — Mycielski (myciel5-7) | `coloring/myciel` | `coloring-myciel` | `qqa bench-run --suite coloring-myciel` |
+| §5.5 / Table 6 | Graph Coloring — Queen (queen5_5 … queen13_13, incl. queen8_12) | `coloring/queen` + `coloring/dimacs` | `coloring-queen`, `coloring-dimacs` | `qqa bench-run --suite coloring-queen` |
+| §5.5 / Table 6 | Graph Coloring — DIMACS real-world (`anna`, `jean`, `queen8_12`) | `coloring/dimacs` | `coloring-dimacs` | `qqa bench-run --suite coloring-dimacs --instances 3` |
+
+Rows whose HF subset does not appear in the paper (G-set, NormCut with
+the 8-graph DISCS superset, ER density sweeps, EA3D spin glass) are
+shipped as convenience extras and documented separately under
+`data/*/README.md`.
+
+**[a] ER-[9000-11000] caveat.** The 16 instances in `mis/er/10k/` are
+the exact graphs used by Sun et al. 2023 and Ichikawa & Arai 2024, but
+the upstream DISCS conversion does not carry per-instance KaMIS
+ground-truth labels, so `manifest.jsonl` records `best_known: null` and
+the ApR column in `qqa bench-run` output is left `NaN`. The paper
+reports the aggregate KaMIS average (`381.31`, Table 5 footnote); to
+reproduce the Table 1 ApR numbers you must either run KaMIS yourself on
+each of the 16 graphs or divide raw IS sizes by that aggregate.
 
 Run them all in one shot:
 

data/coloring/README.md

@@ -1,42 +1,79 @@
 # Graph Coloring Benchmark
 
-Procedurally generated Graph Coloring instances used by
-`qqa.datasets.coloring` and `scripts/bench_discs.py --suite coloring-*`.
+Graph Coloring instances used by `qqa.datasets.coloring` and
+`qqa bench-run --suite coloring-*`.
 
-Following the PQQA paper (arXiv:2409.02135v2) we evaluate the classical
-COLOR benchmark families:
+Following the PQQA paper (arXiv:2409.02135v2, §5.5 / Table 6) we
+evaluate the classical COLOR benchmark (Trick 2002). Coverage is
+organised in three families:
 
-| graph_type | generator                    | chromatic number |
-|------------|-------------------------------|------------------|
-| `myciel`   | `networkx.mycielski_graph(k)` | known (k+1)      |
-| `queen`    | queen-attack graph on k×k     | known (see code) |
+| graph_type | generator / source                                           | chromatic number (`num_colors`)                   |
+|------------|--------------------------------------------------------------|---------------------------------------------------|
+| `myciel`   | `networkx.mycielski_graph(k)` (Mycielski 1955)               | `k` (procedural)                                  |
+| `queen`    | queen-attack graph on a k×k chessboard                       | tabulated, k∈{5..13} (Chvátal; DeLaVina)         |
+| `dimacs`   | DIMACS `.col` fetched from `mat.tepper.cmu.edu/COLOR/instances` | tabulated (Trick 2002 and follow-ups)          |
 
-`best_known` is always `0` (the number of edge conflicts a proper coloring
-must reach). `num_colors` stores the target chromatic number so
-`qqa.problems.Coloring` is instantiated with a feasible palette.
+`best_known` is always `0` (the minimum number of edge conflicts a
+proper coloring must reach). `num_colors` stores the target chromatic
+number so `qqa.problems.Coloring` is instantiated with a feasible
+palette.
+
+## arXiv-2409.02135v2 Table 6 coverage
+
+The paper tabulates 12 specific instances. All 12 are reachable through
+the public loader:
+
+| Paper row   | Colors | Local file path                           | `num_nodes` | `num_edges` |
+|-------------|--------|-------------------------------------------|-------------|-------------|
+| anna        | 11     | `coloring/dimacs/0001.gpickle`            | 138         | 493         |
+| jean        | 10     | `coloring/dimacs/0002.gpickle`            | 80          | 254         |
+| queen8_12   | 12     | `coloring/dimacs/0003.gpickle`            | 96          | 1368        |
+| myciel5     |  6     | `coloring/myciel/0003.gpickle`            | 23          | 71          |
+| myciel6     |  7     | `coloring/myciel/0004.gpickle`            | 47          | 236         |
+| queen5_5    |  5     | `coloring/queen/0001.gpickle`             | 25          | 160         |
+| queen6_6    |  7     | `coloring/queen/0002.gpickle`             | 36          | 290         |
+| queen7_7    |  7     | `coloring/queen/0003.gpickle`             | 49          | 476         |
+| queen8_8    |  9     | `coloring/queen/0004.gpickle`             | 64          | 728         |
+| queen9_9    | 10     | `coloring/queen/0005.gpickle`             | 81          | 1056        |
+| queen11_11  | 11     | `coloring/queen/0007.gpickle`             | 121         | 1980        |
+| queen13_13  | 13     | `coloring/queen/0009.gpickle`             | 169         | 3328        |
+
+This mapping is enforced by
+`QQA4CO_plugin/tests/test_hf_bench_coverage.py::test_coloring_paper_table6_coverage`,
+which fails the test suite if any of the 12 instances disappears.
 
 ## Layout
 
 ```
 data/coloring/
-├── README.md         (git-tracked)
-├── .gitignore        (git-tracked, ignores the rest)
+├── README.md           (git-tracked)
+├── .gitignore          (git-tracked, ignores the rest)
 ├── myciel/
 │   ├── manifest.jsonl
-│   ├── myciel3.gpickle
+│   ├── 0001.gpickle    # myciel3 .. myciel7
 │   └── ...
-└── queen/
+├── queen/
+│   ├── manifest.jsonl
+│   ├── 0001.gpickle    # queen5_5 .. queen13_13
+│   └── ...
+└── dimacs/
     ├── manifest.jsonl
-    ├── queen5_5.gpickle
-    └── ...
+    ├── 0001.gpickle    # anna
+    ├── 0002.gpickle    # jean
+    └── 0003.gpickle    # queen8_12
 ```
 
 Everything below `.gitignore` is re-generated locally by:
 
 ```bash
-python scripts/generate_coloring_instances.py
+python scripts/generate_coloring_instances.py       # fetches DIMACS once, caches in data/coloring/_dimacs_cache
+python scripts/generate_coloring_instances.py --skip-dimacs   # offline rebuild
 ```
 
+The DIMACS family is downloaded once (≈10 KB per `.col`) from
+Trick's canonical mirror; the cached ascii sources remain in
+`data/coloring/_dimacs_cache/` for reproducibility.
+
 Upstream distribution: `Yuma-Ichikawa/qqa4co-bench` on the Hugging Face
 Hub (same repo as DISCS; see `data/discs/README.md` for credentials /
 fallbacks).

scripts/generate_coloring_instances.py

@@ -31,10 +31,72 @@
 import argparse
 import json
 import pickle
+import urllib.request
 from pathlib import Path
 
 import networkx as nx
 
+# -------- DIMACS COLOR real-world instances --------------------------------
+# The PQQA paper (Table 6) also evaluates three non-procedural COLOR
+# instances that do not have a closed-form generator: ``anna`` and ``jean``
+# come from Knuth's Stanford GraphBase (book-character intersection
+# graphs), and ``queen8_12`` is a non-square queen graph. We fetch the
+# canonical DIMACS ``.col`` files from Trick's COLOR repository. The
+# chromatic numbers below are the paper's target ``num_colors`` (anna: 11,
+# jean: 10, queen8_12: 12; see Schuetz et al. 2022 Table 7 and Yang et al.
+# 2021 Table 3).
+DIMACS_COLOR_URL = "https://mat.tepper.cmu.edu/COLOR/instances"
+DIMACS_INSTANCES: dict[str, int] = {
+    "anna": 11,
+    "jean": 10,
+    "queen8_12": 12,
+}
+
+
+def _parse_dimacs_col(text: str) -> nx.Graph:
+    """Parse a DIMACS ``.col`` ascii edge list into an undirected graph.
+
+    DIMACS format: comment lines start with ``c``; a ``p edge N M`` header
+    declares ``N`` 1-indexed vertices and ``M`` edges; each edge is a line
+    ``e u v``. We store 0-indexed nodes for consistency with the rest of
+    QQA4CO.
+    """
+    g = nx.Graph()
+    n_vertices: int | None = None
+    for raw in text.splitlines():
+        line = raw.strip()
+        if not line or line.startswith("c"):
+            continue
+        tok = line.split()
+        if tok[0] == "p":
+            n_vertices = int(tok[2])
+            g.add_nodes_from(range(n_vertices))
+        elif tok[0] == "e":
+            u = int(tok[1]) - 1
+            v = int(tok[2]) - 1
+            if u != v:
+                g.add_edge(u, v, weight=1.0)
+    if n_vertices is None:
+        raise ValueError("DIMACS .col file missing 'p edge ...' header")
+    return g
+
+
+def build_dimacs(name: str, cache_dir: Path) -> nx.Graph:
+    """Download (once) and parse ``<name>.col`` from Trick's COLOR site."""
+    cache_dir.mkdir(parents=True, exist_ok=True)
+    fpath = cache_dir / f"{name}.col"
+    if not fpath.exists():
+        url = f"{DIMACS_COLOR_URL}/{name}.col"
+        with urllib.request.urlopen(url, timeout=30) as resp:  # noqa: S310
+            fpath.write_bytes(resp.read())
+    g = _parse_dimacs_col(fpath.read_text())
+    g.graph["problem"] = "coloring"
+    g.graph["graph_type"] = "dimacs"
+    g.graph["subset"] = name
+    g.graph["num_colors"] = DIMACS_INSTANCES[name]
+    g.graph["source"] = f"DIMACS COLOR {name}.col (Trick 2002; Stanford GraphBase)"
+    return g
+
 # -------- Mycielski graphs -------------------------------------------------
 # Chromatic number of M_k is k by construction (Mycielski 1955).
 # We start from k=3 so N grows: M_3=5, M_4=11, M_5=23, M_6=47, M_7=95.
@@ -116,7 +178,11 @@ def write_family(
                 "best_known_source": (
                     "chromatic number from Mycielski 1955"
                     if family == "myciel"
-                    else "tabulated chromatic number"
+                    else (
+                        "DIMACS COLOR tabulated chromatic number (Trick 2002)"
+                        if family == "dimacs"
+                        else "tabulated chromatic number"
+                    )
                 ),
                 "num_colors": K,
                 "source": g.graph.get("source", ""),
@@ -131,6 +197,18 @@ def write_family(
 def main() -> None:
     parser = argparse.ArgumentParser(description=__doc__)
     parser.add_argument("--out", type=Path, default=Path("data"))
+    parser.add_argument(
+        "--skip-dimacs",
+        action="store_true",
+        help="Skip fetching DIMACS COLOR real-world instances (anna, jean, queen8_12). "
+        "Use this for offline rebuilds.",
+    )
+    parser.add_argument(
+        "--dimacs-cache",
+        type=Path,
+        default=Path("data") / "coloring" / "_dimacs_cache",
+        help="Directory to cache downloaded .col files (default: data/coloring/_dimacs_cache).",
+    )
     args = parser.parse_args()
 
     mycs = [(f"myciel{k}", build_mycielski(k), k) for k in MYCIEL_ORDERS]
@@ -139,6 +217,13 @@ def main() -> None:
     queens = [(f"queen{k}_{k}", build_queen(k), QUEEN_CHROMATIC[k]) for k in QUEEN_SIZES]
     write_family(args.out, "queen", queens)
 
+    if not args.skip_dimacs:
+        dimacs = [
+            (name, build_dimacs(name, args.dimacs_cache), DIMACS_INSTANCES[name])
+            for name in DIMACS_INSTANCES
+        ]
+        write_family(args.out, "dimacs", dimacs)
+
 
 if __name__ == "__main__":
     main()