Skip to content

README.md

Latest commit

 

History

History
171 lines (138 loc) · 7.31 KB

README.md

File metadata and controls

171 lines (138 loc) · 7.31 KB
Raw

ReporterNet predicts regulatory activity for sequences in diverse reporter assays and enables systematic comparative analysis of high throughput reporter assays

This repository contains the complete codebase for the ReporterNet manuscript, including data processing, model training, and evaluation, alongside motif calling, variant effect prediction, and all downstream analyses.

ReporterNet preprint:

High-throughput reporter assays are widely used for directly measuring regulatory potential for candidate regulatory elements (cREs). There are several variations of reporter assays based on factors such as the chromatin context and the location of the test sequences. Despite apparent similarities, different reporter assays exhibit variable concordance in their functional readouts for identical sequences. Systematic comparison of these assays is challenging due to distinct input libraries and different experimental biases. Thus, the causal sequence syntax driving similarities and differences among these assays is not yet well understood. In this project, we trained ReporterNet, a convolutional neural network, on four varieties of reporter assays in the K562 cell line and systematically compared the resulting models in terms of genome-wide motif abundance and activity, predicted cCRE activities, and GWAS variant prioritization. Using ReporterNet, we uncovered consistent, assay-specific differences in TF motif landscapes, suggesting that the experimental design fundamentally impacts the reporter assay readouts.

Installation

ReporterNet requires specific dependencies for genomic data processing (e.g., MACS3, wiggletools) and a strictly versioned deep learning stack for model training and interpretation (TensorFlow 2.8, Python 3.8, TF-MoDISco).

To prevent dependency conflicts, we provide two separate Conda environments. We recommend using conda for installation.

1. Clone the Repository

Begin by cloning the repository to your local machine:

git clone https://github.com/ziwei-75/reporterNet.git
cd reporterNet

2. Choose Your Environment

Option A: Training & Interpretation Environment

Use this environment if you want to train models, run predictions, or execute the interpretability pipeline (SHAP, DeepLIFT, TF-MoDISco). This environment uses Python 3.8.

Navigate to the envs directory and create the environment:

cd envs
conda env create -f environment_training.yml
conda activate reporterNet
cd ..

Option B: Data Processing Environment

Use this environment if your goal is solely to process raw STARR-seq data. This environment uses Python 3.10.

Navigate to the envs directory and create the environment:

cd envs
conda env create -f environment_processing.yml
conda activate reporterNet_processing
cd ..

3. Verify Installation

Installing the package registers several command-line tools. If you are in the reporterNet (training) environment, you can verify the installation by testing the help commands for the entry points:

# Verify the training pipeline
reporterNet-train --help

# Verify the evaluation pipeline
reporterNet-eval --help

# Verify the interpretation pipeline
reporterNet-interpret --help

ReporterNet training (MPRA)

To train a new ReporterNet model on MPRA data, you can run the following command. Make sure to adjust the paths to point to your specific dataset and split files.

reporterNet-train \
    --assay_type 'mpra' \
    --mpra_table "./data/example_dataset/data_with_chromo.tsv" \
    --fold "./data/splits/fold_0.json" \
    --input_seq_len 230 \
    --num_filters 512 \
    --num_dilation_layers 8 \
    --batch_size 256 \
    --dilation_rates "2,4,8,16,32,32,32,32" \
    --dropout_rate 0.2 \
    --learning_rate 5e-4 \
    --activity_column 'mean_value' \
    --sequence_column 'seq' \
    --patience 8 \
    --max_epochs 80 \
    --conv1_kernel_size 7 \
    --output_prefix "./models/mpra_fold0_model"

ReporterNet training (STARR-seq)

To train a ReporterNet model on STARR-seq data, you will need to provide the relevant .narrowPeak, .bed, and BigWig (.bw) files, along with the reference genome.

reporterNet-train \
   --assay_type 'starr' \
   --fold "./data/splits/fold_0.json" \
   --input_seq_len 500 \
   --rna_peak "./data/wgs_starr/rna/peaks_filtered.narrowPeak" \
   --rna_nonpeak "./data/wgs_starr/rna/negatives_with_summit.bed" \
   --dna_bigwig "./data/wgs_starr/dna/example_dna.bw" \
   --rna_bigwig "./data/wgs_starr/rna/example_rna.bw" \
   --bw_output_size 100 \
   --dilation_rates "2,4,8,16,32,32,32,32" \
   --dropout_rate 0.1 \
   --learning_rate 5e-4 \
   --max_jitter 50 \
   --rna_nonpeak_ratio 0.5 \
   --batch_size 256 \
   --genome "./data/hg38/hg38.fasta" \
   --dna_factor 0.5 \
   --output_prefix "./models/starr_fold0_model"

ReporterNet evaluation (MPRA)

Once the model is trained, you can evaluate its performance on the same fold using the reporterNet-eval command:

reporterNet-eval \
    --assay_type 'mpra' \
    --mpra_table "./data/example_dataset/data_with_chromo.tsv" \
    --model_path "./models/mpra_fold0_model.h5" \
    --fold "./data/splits/fold_0.json" \
    --input_seq_len 230 \
    --activity_column 'mean_value' \
    --sequence_column 'seq' \
    --output_prefix "./models/eval_fold0" \
    --reverse_complement True

ReporterNet evaluation (STARR-seq)

Evaluate your trained STARR-seq model using the reporterNet-eval command with the same input files and reference genome:

reporterNet-eval \
    --assay_type 'starr' \
    --model_path "./models/starr_fold0_model.h5" \
    --fold "./data/splits/fold_0.json" \
    --input_seq_len 500 \
    --rna_peak "./data/wgs_starr/rna/peaks_filtered.narrowPeak" \
    --rna_nonpeak "./data/wgs_starr/rna/negatives_with_summit.bed" \
    --dna_bigwig "./data/wgs_starr/dna/example_dna.bw" \
    --rna_bigwig "./data/wgs_starr/rna/example_rna.bw" \
    --bw_output_size 100 \
    --genome "./data/hg38/hg38.fasta" \
    --output_prefix "./models/starr_eval_fold0" \
    --reverse_complement True \
    --dna_factor 0.5

ReporterNet interpretation

After training, you can run the interpretation pipeline to extract predictive features (such as transcription factor binding motifs) from specific genomic regions using your trained model.

Make sure to provide the reference genome and the .tsv or .bed file containing the regions you want to interpret.

reporterNet-interpret \
    --reference_genome "./data/hg38/hg38.fasta" \
    --input_seq_len 500 \
    --model "./models/starr_fold0_model.h5" \
    --bed_file "./data/atac/interpretation_regions.tsv" \
    --output_prefix "./models/interpretation/fold0_split0"

Motif Discovery

Motif pattern discovery was performed using TF-MoDISco (https://github.com/kundajelab/tfmodisco), and motif calling was performed with FiNeMo (https://github.com/kundajelab/Fi-NeMo). For examples of how to run these tools within our pipeline, please refer to scripts/wgs_starr/train/merge_atac_dnase_shap.sh and scripts/wgs_starr/train/call_motif.sh.

Process STARR-seq data

To process STARR-seq data and perform peak calling, refer to the example script located at scripts/wgs_starr/process_data/process_data.sh

How to Cite

A preprint for ReporterNet will be available soon. In the meantime, if you use ReporterNet in your work, please link to this repository. Check back shortly for the formal citation!