MOON CytoSig Benchmark

This repository contains the CytoSig ligand-perturbation benchmark for the MOON method described in manuscript Section 2.2 and Figure 2.

The main script to read and maintain is:

Rscript scripts/citosig_moon.R

That script is organized around the manuscript panels and should be treated as the canonical entry point for this repository. Other scripts are helpers, provenance, or exploratory diagnostics.

Rationale

CytoSig contains transcriptomic signatures measured after cytokine or ligand perturbations. For each CytoSig signature, this benchmark estimates TF activities from the gene-expression z-scores and then uses MOON to propagate those activities upstream through the COSMOS prior knowledge network.

The central expectation is simple: when a ligand was applied in a CytoSig experiment, the MOON score of that ligand should be positive. This gives a direct benchmark of whether MOON can recover known upstream perturbations from downstream transcriptional footprints.

The repository intentionally keeps the published paper snapshot frozen. The default script uses the original MOON cache that generated the manuscript figures. A separate 2024-03-19 corrected-network rerun is available for transparency, but it is not silently mixed into the paper figures.

Main Results

Using the frozen manuscript snapshot:

• MOON scores could be evaluated for 549 CytoSig ligand-signature pairs.
• These 549 scores covered 63 unique ligands.
• 47 of 63 ligands had a positive mean MOON score in the experiments where they were applied.
• In the exploratory per-ligand one-sample t-tests against zero, 16 ligands were significantly positive and 4 were significantly negative at p <= 0.05.
• For 44 of 63 ligands, the average within-experiment score quantile was higher in experiments where the ligand was applied than in experiments where other ligands were applied.
• The extracted MOON scoring networks had median sizes of 30 edges, 11 nodes, and 3 downstream TFs; the mean number of downstream TFs was 7.3.
• The largest extracted scoring network contained 1,766 edges and 263 nodes.

The corrected 2024-03-19 network snapshot is useful for interpreting known prior-knowledge issues. In particular, it changes the OSM/IL6-related behavior after correction of IL6ST downstream annotations in the COSMOS/Omnipath prior knowledge network.

Reproducing the Paper Snapshot

Run the main script with defaults:

Rscript scripts/citosig_moon.R

This uses:

results/citosig_moon_activties.RData
results/moon_res_df_long_target_only_GPTcleanedup_withMeanScore.csv
support/neta_network.RData

and writes the Figure 2 panel outputs to results/figures/.

For a dry run that builds the analysis objects and prints the sanity-check counts without rewriting PDFs:

CITOSIG_WRITE_FIGURES=false Rscript scripts/citosig_moon.R

In PowerShell:

$env:CITOSIG_WRITE_FIGURES = "false"
Rscript scripts/citosig_moon.R
Remove-Item Env:CITOSIG_WRITE_FIGURES

Optional Corrected-Network Snapshot

To generate plots from the 2024-03-19 corrected-network rerun without recomputing TF activities or MOON scores:

CITOSIG_SNAPSHOT=corrected_20240319 Rscript scripts/citosig_moon.R

In PowerShell:

$env:CITOSIG_SNAPSHOT = "corrected_20240319"
Rscript scripts/citosig_moon.R
Remove-Item Env:CITOSIG_SNAPSHOT

Corrected-snapshot figures are written under:

results/figures/corrected_20240319/

Figure 2C is skipped for this snapshot unless a matching corrected scoring-network-characteristics cache is added.

Snapshot Delta Plot

To compare applied-ligand performance between the frozen paper snapshot and the 2024-03-19 corrected snapshot:

Rscript scripts/plot_citosig_snapshot_delta.R

This writes:

results/figures/moon_score_delta_original_vs_20240319.pdf
results/figures/moon_score_delta_original_vs_20240319.csv

The plotted value is:

sign * (updated_score - original_score)

For activating perturbations, positive values mean that the corrected snapshot moved the applied-ligand score in the expected direction; negative values mean that the corrected snapshot moved it away from the expected direction.

Expensive Recomputations

The main script defaults to cached results. Full recomputation is intentionally opt-in because running MOON across all CytoSig signatures is slow.

Useful switches:

CITOSIG_RECOMPUTE_TF=true
CITOSIG_RECOMPUTE_MOON=true
CITOSIG_EXPORT_PANEL_D_NETWORKS=true
CITOSIG_COMPARE_CORRECTED=true

Do not set CITOSIG_RECOMPUTE_MOON=true unless you explicitly want to rerun MOON on all CytoSig signatures.

Repository Layout

scripts/citosig_moon.R
  Main manuscript Figure 2 workflow.

scripts/plot_citosig_snapshot_delta.R
  Original-vs-corrected snapshot comparison plot.

scripts/pval_benchmark.R
  Exploratory per-ligand t-test diagnostics used to understand whether ligand
  scores differ from zero.

scripts/cleanup_cytosig.R
  CytoSig preprocessing and ligand-name harmonization.

scripts/Standardizing _hours_ column entries.html
  Provenance snapshot for the manual/ChatGPT-assisted standardization of messy
  CytoSig time annotations.

support/
  Cached prior-knowledge networks and CollecTRI resource used by the benchmark.

results/
  Cached intermediate results and generated figures.

Data and Large Caches

Large CytoSig input files and MOON cache files are intentionally ignored by Git. Get the data files from:

https://www.dropbox.com/scl/fo/xhwdkqkllrgzx65qnbhig/AD0ZtNRy-a_-jJr7UzVqJR8?rlkey=664x498nrrmm7n2uguxbaa2wo&st=iihg3o65&dl=0

Expected local ignored paths include:

data/
results/citosig_moon_activties.RData
results/citosig_moon_activties_20240319.RData
2024.07.15.603538v3.full.pdf

The manuscript PDF can be placed locally as a reference, but it is not tracked in Git.