Publish GSE274113 tutorial dataset workflow

Author: HaochenW

CHANGELOG.md

@@ -19,12 +19,15 @@ after its first public release.
   labels retained in portable checkpoints.
 - Version-pinned, checksum-verified download command for the scDiffusion-X
   OpenProblem dataset.
+- Version-pinned, checksum-verified download command for the processed
+  GSE274113 perturbation dataset on Zenodo.
 - A concise H5MU-first tutorial and an implementation audit linking the public
   models to the executed research references.
 - End-to-end paper workflows for training the generation RNA VAE and
   multimodal AE, encoding raw paired profiles, and decoding generated states.
 - Leave-one-cell-type-out perturbation preparation and the executed
   training-only post-sampling latent mean-shift correction.
+- Public perturbation tutorial commands using the published GSE274113 H5MU.
 
 ### Changed
 

README.md

@@ -13,8 +13,8 @@ states. It supports cell-type-conditioned generation and
 perturbation-conditioned prediction while keeping the two modalities paired.
 
 > **Alpha release.** The flow model and raw-H5MU paper workflow are ready for
-> testing. The processed perturbation dataset is not yet hosted publicly; the
-> tutorial therefore requires a local path for that dataset.
+> testing. Both public datasets used by the tutorial are now available through
+> version-pinned downloads.
 
 ## Install
 
@@ -92,6 +92,27 @@ The upstream file contains the correct raw RNA counts and binary ATAC
 profiles. The tutorial splits it, trains the RNA VAE and ATAC AE, and runs
 `multiflow paper encode`; users do not create latent arrays manually.
 
+## GSE274113 perturbation data
+
+The processed paired perturbation data can also be downloaded directly:
+
+```bash
+multiflow data download gse274113 \
+  --output data/GSE274113_filtered.h5mu \
+  --accept-license
+```
+
+This is a version-pinned **31.15 GB** Zenodo download. MultiFlow verifies the
+published byte size and MD5 before making the file available at the requested
+path.
+
+- Version DOI: [10.5281/zenodo.21986866](https://doi.org/10.5281/zenodo.21986866)
+- All-versions DOI: [10.5281/zenodo.21986865](https://doi.org/10.5281/zenodo.21986865)
+- Source study: [GSE274113](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE274113)
+
+The complete leave-one-cell-type-out perturbation workflow is in the
+[tutorial](docs/tutorial.md#4-task-2-perturbation-prediction).
+
 ## Models
 
 - `cell-state` (default): bidirectional cross-attention MultiFlow.

docs/implementation_audit.md

@@ -57,8 +57,10 @@ CLI round trips.
   scDiffusion-X multimodal AE. Perturbation uses the multimodal AE for both raw
   RNA counts and binary ATAC. These task-specific scales are not interchangeable.
 - The release includes the audited RNA VAE architecture and wrappers around a
-  pinned scDiffusion-X checkout. It does not redistribute pretrained weights or
-  the processed GSE274113 H5MU.
+  pinned scDiffusion-X checkout. It does not bundle pretrained weights or large
+  data files. The processed GSE274113 H5MU is distributed separately through
+  the version-pinned Zenodo record
+  [10.5281/zenodo.21986866](https://doi.org/10.5281/zenodo.21986866).
 - `multiflow paper prepare-perturbation-fold` reproduces the executed notebook:
   it removes held-out non-control cells from flow training, retains held-out
   controls, and computes each cell-type context from all remaining flow-training

docs/tutorial.md

@@ -234,26 +234,27 @@ Bernoulli sample and therefore contains only 0 and 1.
 
 ### 4.1 Prepare the perturbation H5MU
 
-The paper uses processed GSE274113 paired multiome data. The processed H5MU is
-not currently hosted by this repository, so do not substitute OpenProblem:
+The paper uses processed GSE274113 paired multiome data. Download the exact
+version used by this tutorial from Zenodo. The file is 31.15 GB; the command
+supports resuming and verifies its byte size and MD5 before publishing it at
+the requested path:
 
 ```bash
-export PERTURBATION_H5MU=/path/to/GSE274113_filtered.h5mu
+multiflow data download gse274113 \
+  --output data/GSE274113_filtered.h5mu \
+  --accept-license
+
+export PERTURBATION_H5MU=data/GSE274113_filtered.h5mu
 multiflow data validate "$PERTURBATION_H5MU"
 ```
 
-For a public release, upload the processed file to Zenodo or Figshare, not
-GitHub. Upload these items together:
-
-- `GSE274113_filtered.h5mu`;
-- `SHA256SUMS.txt`;
-- `dataset_manifest.json` with cell, gene and peak counts, matrix scales,
-  feature identifiers, preprocessing version and required `obs` columns; and
-- `README_data.md` citing GSE274113 and describing peak merging and filtering.
-
-After the record receives a DOI, add its version-pinned URL, byte size and
-checksum to `src/multiflow_omics/datasets.py`. Until then, this tutorial uses a
-local path rather than a fabricated URL.
+The fixed dataset version is
+[Zenodo 10.5281/zenodo.21986866](https://doi.org/10.5281/zenodo.21986866);
+the DOI representing all current and future versions is
+[10.5281/zenodo.21986865](https://doi.org/10.5281/zenodo.21986865). The source
+study is [GSE274113](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE274113).
+Review the Zenodo record and source-data terms before passing
+`--accept-license`.
 
 ### 4.2 Train and run the perturbation encoder
 

src/multiflow_omics/datasets.py

@@ -30,7 +30,23 @@ class DatasetFile:
     license="CC BY 4.0",
 )
 
-DATASETS = {OPENPROBLEM.name: OPENPROBLEM}
+GSE274113 = DatasetFile(
+    name="gse274113",
+    filename="GSE274113_filtered.h5mu",
+    url=(
+        "https://zenodo.org/records/21986866/files/"
+        "GSE274113_filtered.h5mu?download=1"
+    ),
+    size_bytes=31_154_968_237,
+    md5="b1aecf4d4cacf3328c0b3147426f5d16",
+    doi="10.5281/zenodo.21986866",
+    license="See the Zenodo record and the GSE274113 source-data terms",
+)
+
+DATASETS = {
+    OPENPROBLEM.name: OPENPROBLEM,
+    GSE274113.name: GSE274113,
+}
 
 
 def md5_file(path: str | os.PathLike[str], *, chunk_size: int = 8 * 1024 * 1024) -> str:

tests/test_datasets.py

@@ -4,7 +4,7 @@
 
 import pytest
 
-from multiflow_omics.datasets import DatasetFile, verify_dataset_file
+from multiflow_omics.datasets import DATASETS, DatasetFile, verify_dataset_file
 
 
 def test_dataset_file_verification(tmp_path):
@@ -26,3 +26,14 @@ def test_dataset_file_verification(tmp_path):
             path,
             DatasetFile(**{**dataset.__dict__, "size_bytes": len(payload) + 1}),
         )
+
+
+def test_public_dataset_registry_is_version_pinned():
+    perturbation = DATASETS["gse274113"]
+    assert perturbation.filename == "GSE274113_filtered.h5mu"
+    assert perturbation.doi == "10.5281/zenodo.21986866"
+    assert perturbation.url.endswith(
+        "/records/21986866/files/GSE274113_filtered.h5mu?download=1"
+    )
+    assert perturbation.size_bytes == 31_154_968_237
+    assert perturbation.md5 == "b1aecf4d4cacf3328c0b3147426f5d16"