Skip to content

Commit 2236d4a

Browse files
author
Yuma Ichikawa
committed
feat(bench): rename HF dataset to qqa4co-bench + add MaxCut G-set family
* HF dataset id `Yuma-Ichikawsa/discs-co-bench` renamed to `Yuma-Ichikawsa/qqa4co-bench` via `HfApi.move_repo` (DISCS prefix was actively misleading after four other families landed). Old id keeps a 30-day redirect. Every in-repo reference (README, docs, setup scripts, data/*/README, plot footer, HF card) updated. * New MaxCut G-set family (71 graphs: G1..G67 + G70/G72/G77/G81) with best-known cuts from Benlic & Hao 2013, Matsuda 2018 and Ichikawa NeurIPS 2024. Covers the canonical annealing / PI-GNN / PQQA benchmark universe. Shipped via: - `scripts/fetch_gset_data.py` (download + pickle + manifest) - `qqa.datasets.gset()` loader - `--suite gset` in `scripts/bench_discs.py` - new catalog branch in `_build_catalog` - full `gset/` payload uploaded to HF (~30 MB) * `qqa bench-plot --output data/fig/x.png` no longer mangles the path into `bench_results/data/fig/x.png`; only bare filenames get the default-dir prefix (cli.py + bench.py both fixed, regression test exists in tests/test_bench_api.py). * Real benchmark visualisation: ran `qqa bench-run` on gset (5), coloring (4), ea3d (3) on CPU and re-rendered `data/fig/gallery/bench_report_example.png` from real data instead of a synthetic mock (91.7% feasibility, 0.979 mean ApR on the gset subset after 300 epochs @ sol-size 64). * New `tests/test_docs_assets.py` — 24 parametrised cases — walks every markdown file under `docs/` and asserts image links either resolve inside the doc root or are absolute URLs. Catches the same class of bug that broke CI run 24658520429 in ~50 ms locally. * Three new unit tests in test_new_benchmark_loaders.py covering the gset loader, its `limit=` knob, and the bench_discs catalog integration. 371 tests pass, 2 skipped (unrelated torch_geometric / plotly guards), ruff check + format green, mkdocs --strict green.
1 parent 6f9a6dc commit 2236d4a

25 files changed

Lines changed: 661 additions & 81 deletions

README.md

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -22,16 +22,16 @@ with earlier QQA4CO releases (``import qqa``).
2222
<a href="https://github.com/Yuma-Ichikawa/QQA4CO/discussions"><img src="https://img.shields.io/github/discussions/Yuma-Ichikawa/QQA4CO?logo=github&label=Discussions" alt="GitHub Discussions"></a>
2323
<a href="https://codecov.io/gh/Yuma-Ichikawa/QQA4CO"><img src="https://codecov.io/gh/Yuma-Ichikawa/QQA4CO/branch/main/graph/badge.svg" alt="Coverage"></a>
2424
<a href="https://doi.org/10.5281/zenodo.19648231"><img src="https://img.shields.io/badge/DOI-10.5281%2Fzenodo.19648231-1f6feb?logo=doi&logoColor=white" alt="DOI"></a>
25-
<a href="https://huggingface.co/datasets/Yuma-Ichikawsa/discs-co-bench"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Dataset-Yuma--Ichikawsa%2Fdiscs--co--bench-yellow" alt="Hugging Face dataset"></a>
25+
<a href="https://huggingface.co/datasets/Yuma-Ichikawsa/qqa4co-bench"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Dataset-Yuma--Ichikawsa%2Fqqa4co--bench-yellow" alt="Hugging Face dataset"></a>
2626
</p>
2727

2828
<p align="center">
2929
<b>Benchmark data &nbsp;·&nbsp;</b>
30-
<a href="https://huggingface.co/datasets/Yuma-Ichikawsa/discs-co-bench">
31-
<code>huggingface.co/datasets/Yuma-Ichikawsa/discs-co-bench</code>
30+
<a href="https://huggingface.co/datasets/Yuma-Ichikawsa/qqa4co-bench">
31+
<code>huggingface.co/datasets/Yuma-Ichikawsa/qqa4co-bench</code>
3232
</a>
3333
<br>
34-
<sub>DISCS (NeurIPS 2023) + Graph Coloring (COLOR) + MIS on d-regular random graphs (PQQA §5.1) + 3D Edwards-Anderson spin glass + Balanced k-way partition — one HF dataset, <code>make bench-all-setup</code> pulls everything.</sub>
34+
<sub>DISCS (NeurIPS 2023) + MaxCut G-set (Helmberg & Rendl 2000) + Graph Coloring (COLOR) + MIS on d-regular random graphs (PQQA §5.1) + 3D Edwards-Anderson spin glass + Balanced k-way partition — one HF dataset, <code>make bench-all-setup</code> pulls everything.</sub>
3535
</p>
3636

3737
<p align="center">
@@ -405,15 +405,16 @@ qqa gui # opens http://localhost:8501
405405

406406
Run `qqa <command> --help` for the full option list.
407407

408-
### All CO benchmarks in one command (DISCS + PQQA + EA3D)
408+
### All CO benchmarks in one command (DISCS + G-set + PQQA + EA3D)
409409

410410
Every benchmark instance lives on the Hugging Face Hub:
411411

412-
> **Dataset:** [`huggingface.co/datasets/Yuma-Ichikawsa/discs-co-bench`](https://huggingface.co/datasets/Yuma-Ichikawsa/discs-co-bench)
413-
> &nbsp;&nbsp;·&nbsp;&nbsp; DISCS (MaxCut/MIS/MaxClique/NormCut) + Graph
414-
> Coloring (COLOR) + MIS on d-regular random graphs (PQQA §5.1) + 3D
415-
> Edwards-Anderson spin glass + Balanced k-way partition — **one repo,
416-
> `make bench-all-setup` pulls everything**.
412+
> **Dataset:** [`huggingface.co/datasets/Yuma-Ichikawsa/qqa4co-bench`](https://huggingface.co/datasets/Yuma-Ichikawsa/qqa4co-bench)
413+
> &nbsp;&nbsp;·&nbsp;&nbsp; DISCS (MaxCut / MIS / MaxClique / NormCut) +
414+
> **MaxCut G-set (Helmberg & Rendl 2000, 71 graphs G1-G67 + G70/72/77/81)** +
415+
> Graph Coloring (COLOR) + MIS on d-regular random graphs (PQQA §5.1) +
416+
> 3D Edwards-Anderson spin glass + Balanced k-way partition — **one
417+
> repo, `make bench-all-setup` pulls everything**.
417418
418419
Third parties benchmark a solver in **three one-liners**:
419420

app/pages/2_Visualize.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,7 @@ class PAResult: # type: ignore[no-redef]
148148
_ordered_keys = _UNIVERSAL_TAB_KEYS + _PQQA_TAB_KEYS + [_TREE_TAB_KEY]
149149

150150
_tabs = st.tabs([_TAB_LABELS[k] for k in _ordered_keys])
151-
_tab_by_key: dict[str, "st.delta_generator.DeltaGenerator"] = dict(
151+
_tab_by_key: dict[str, st.delta_generator.DeltaGenerator] = dict(
152152
zip(_ordered_keys, _tabs, strict=True)
153153
)
154154

data/coloring/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,6 @@ Everything below `.gitignore` is re-generated locally by:
3737
python scripts/generate_coloring_instances.py
3838
```
3939

40-
Upstream distribution: `yuma-ichikawa/discs-co-bench` on the Hugging Face
40+
Upstream distribution: `Yuma-Ichikawsa/qqa4co-bench` on the Hugging Face
4141
Hub (same repo as DISCS; see `data/discs/README.md` for credentials /
4242
fallbacks).

data/discs/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ make bench-discs SUITE=mis-satlib BACKEND=qqa DEVICE=cuda
3636
That is it. The setup script will:
3737

3838
1. **Download** the prebuilt dataset from the Hugging Face Hub
39-
(`yuma-ichikawa/discs-co-bench`, override with `DISCS_HF_REPO_ID=...`) —
39+
(`Yuma-Ichikawsa/qqa4co-bench`, override with `DISCS_HF_REPO_ID=...`) —
4040
primary source, free, no auth required.
4141
2. **Or fall back** to the original Google Drive tarball
4242
(`DISCS-DATA.tar.gz`, ~6.7 GB) and extract just the `DISCS-DATA/sco/`
@@ -127,7 +127,7 @@ data/discs/
127127

128128
### Why Parquet for distribution (Hugging Face)?
129129

130-
- Hugging Face Datasets' default; `load_dataset("user/discs-co-bench")` works
130+
- Hugging Face Datasets' default; `load_dataset("user/qqa4co-bench")` works
131131
out-of-the-box, with streaming for the large MIS-ER-10k subset.
132132
- Compresses ~3-5× tighter than per-file pickles.
133133
- Survives format evolution: edge_index columns are forwards-compatible.

data/ea3d/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,5 +41,5 @@ against this published ground-state reference when available.
4141
python scripts/generate_ea3d_instances.py
4242
```
4343

44-
Upstream distribution: `yuma-ichikawa/discs-co-bench` on the Hugging Face
44+
Upstream distribution: `Yuma-Ichikawsa/qqa4co-bench` on the Hugging Face
4545
Hub.
-187 KB
Loading

data/gset/.gitignore

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,9 @@
1+
# The actual Gset payload (30 MB) lives on Hugging Face
2+
# (`Yuma-Ichikawsa/qqa4co-bench` under `gset/`). Pull it with
3+
# `make bench-all-setup` or `./scripts/setup_benchmarks.sh --only gset`.
4+
# The raw Stanford text files downloaded by
5+
# `scripts/fetch_gset_data.py` land under `raw/` and are likewise
6+
# excluded from version control.
17
*
28
!.gitignore
9+
!README.md

data/gset/README.md

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# MaxCut — G-set benchmark
2+
3+
**71 graphs**, best-known cuts tracked, hosted on
4+
[`Yuma-Ichikawsa/qqa4co-bench`](https://huggingface.co/datasets/Yuma-Ichikawsa/qqa4co-bench)
5+
under `gset/`.
6+
7+
| size | instances | typical n, m |
8+
| ------------------ | -------------------- | -------------------------------------------- |
9+
| small | G1..G21 | n=800, m=800-19 176 |
10+
| medium | G22..G54 | n=1 000..3 000 |
11+
| large | G55..G67 | n=5 000..7 000 |
12+
| sparse near-bipart | G70, G72, G77, G81 | n=10 000..20 000 |
13+
14+
Upstream G68, G69, G71, G73-76, G78-80 are missing from the canonical
15+
Ye mirror and are therefore not included here (the fetch script skips
16+
them automatically).
17+
18+
Layout
19+
20+
```
21+
data/gset/standard/
22+
G1.gpickle, G2.gpickle, ..., G81.gpickle
23+
manifest.jsonl
24+
```
25+
26+
Each manifest record carries at least
27+
28+
```json
29+
{"id": "G70", "file": "G70.gpickle", "nodes": 10000, "edges": 9999,
30+
"best_known": 9591,
31+
"best_known_source": "Benlic & Hao 2013; Matsuda 2018; Ichikawa NeurIPS 2024",
32+
"source_url": "https://web.stanford.edu/~yyye/yyye/Gset/G70",
33+
"problem": "maxcut", "graph_type": "gset", "subset": "standard"}
34+
```
35+
36+
Fetch / regenerate
37+
38+
```bash
39+
# pull the whole family (30 MB) from Hugging Face
40+
./scripts/setup_benchmarks.sh --only gset
41+
42+
# …or re-download from Stanford and re-pickle locally (no HF required)
43+
python scripts/fetch_gset_data.py
44+
```
45+
46+
Run
47+
48+
```bash
49+
qqa bench-run --suite gset --instances 5 --output gset.json
50+
qqa bench-plot bench_results/gset.json --output gset.png
51+
```
52+
53+
Approximation-ratio convention: ``ratio = found_cut / best_known``
54+
(higher = better; 1.0 = matches the published upper bound).

data/mis-rrg/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,5 +40,5 @@ solver benchmark. We ship the `n = 10^4` variant here (covers both
4040
the Hugging Face Hub mirror stays lean. Reinstate it by passing
4141
`--nodes 1000000` to `scripts/generate_rrg_instances.py` if needed.
4242

43-
Upstream distribution: `yuma-ichikawa/discs-co-bench` on the Hugging Face
43+
Upstream distribution: `Yuma-Ichikawsa/qqa4co-bench` on the Hugging Face
4444
Hub.

docs/how-to/benchmark.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,14 @@ No edits to the repo or knowledge of its internals are required.
66

77
Every benchmark instance lives on the Hugging Face Hub:
88

9-
> **[`huggingface.co/datasets/Yuma-Ichikawsa/discs-co-bench`](https://huggingface.co/datasets/Yuma-Ichikawsa/discs-co-bench)**
9+
> **[`huggingface.co/datasets/Yuma-Ichikawsa/qqa4co-bench`](https://huggingface.co/datasets/Yuma-Ichikawsa/qqa4co-bench)**
1010
1111
The dataset bundles the [DISCS](https://arxiv.org/abs/2311.04730) CO
12-
benchmarks (NeurIPS 2023) plus four extra families described in the
13-
[PQQA paper](https://openreview.net/forum?id=9EfBeXaXf0): Graph
14-
Coloring, MIS on `d`-regular random graphs, 3D Edwards-Anderson spin
15-
glass and Balanced k-way partition.
12+
benchmarks (NeurIPS 2023), the **MaxCut G-set** superset (71 graphs,
13+
Helmberg & Rendl 2000 via Yinyu Ye's mirror), and four extra families
14+
described in the [PQQA paper](https://openreview.net/forum?id=9EfBeXaXf0):
15+
Graph Coloring, MIS on `d`-regular random graphs, 3D Edwards-Anderson
16+
spin glass and Balanced k-way partition.
1617

1718
---
1819

@@ -117,6 +118,7 @@ convention differs per family so that the ratio is always in
117118
|------------------------|----------------|-------------------------------|
118119
| `mis`, `mis-rrg` | maximisation | `solver / best_known` |
119120
| `maxcut`, `maxclique` | maximisation | `solver / best_known` |
121+
| `gset` (MaxCut) | maximisation | `solver / best_known` |
120122
| `normcut` | minimisation | `best_known / solver` |
121123
| `coloring` | conflict count | `feasible` flag only |
122124
| `ea3d` | energy (≤ 0) | `solver / best_known` |

0 commit comments

Comments
 (0)