Skip to content

Commit abbc38c

Browse files
committed
Add deployable RNA-seq report portal
1 parent 02db5a1 commit abbc38c

10 files changed

Lines changed: 372 additions & 15 deletions

File tree

.github/workflows/ci.yml

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,14 +10,14 @@ jobs:
1010
test:
1111
runs-on: ubuntu-latest
1212
steps:
13-
- uses: actions/checkout@v4
13+
- uses: actions/checkout@v6
1414

1515
- uses: actions/setup-java@v4
1616
with:
1717
distribution: temurin
1818
java-version: 21
1919

20-
- uses: actions/setup-python@v5
20+
- uses: actions/setup-python@v6
2121
with:
2222
python-version: '3.12'
2323

@@ -53,3 +53,6 @@ jobs:
5353
python -m pip install --upgrade pip
5454
pip install -r cloud/report-portal/requirements.txt
5555
pytest cloud/report-portal/tests
56+
57+
- name: Build report portal image
58+
run: docker build -t rnaseq-report-portal cloud/report-portal

README.md

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ Designed around the [Himes et al. (2014)](https://doi.org/10.1371/journal.pone.0
1414
- Full synthetic smoke test in GitHub Actions, including containerised FastQC, fastp, HISAT2, samtools, featureCounts, DESeq2 and MultiQC.
1515
- Docker, Singularity and AWS Batch profiles in `nextflow.config`.
1616
- Containerised FastAPI report portal under `cloud/report-portal/` for S3-hosted reports and Postgres run metadata.
17+
- Render Blueprint at `render.yaml` for a deployable FastAPI plus Postgres report portal.
1718
- `nextflow_schema.json` for parameter discovery in Seqera Platform and other launch tooling.
1819
- Nextflow execution report, timeline, trace and DAG written to `results/pipeline_info/` on every run.
1920
- `scripts/validate_outputs.py` checks count matrices, DESeq2 output, plots, MultiQC and run metadata in CI.
@@ -125,14 +126,27 @@ nextflow run Ekin-Kahraman/rnaseq-nextflow-pipeline \
125126

126127
### Report portal
127128

128-
The optional [cloud report portal](cloud/report-portal/) registers cloud runs and returns signed S3 URLs for Nextflow reports, timelines, traces, DAGs and MultiQC output. It is a small FastAPI service backed by Postgres in production and SQLite for local testing.
129+
The optional [cloud report portal](cloud/report-portal/) registers cloud runs and returns signed S3 URLs for Nextflow reports, timelines, traces, DAGs and MultiQC output. It is a small FastAPI service backed by Postgres in production and SQLite for local testing. The root route renders a browser dashboard and `/docs` exposes the API.
129130

130131
```bash
131132
cd cloud/report-portal
132133
pip install -r requirements.txt
133134
uvicorn app.main:app --reload --port 8000
134135
```
135136

137+
Run the local Postgres stack:
138+
139+
```bash
140+
cd cloud/report-portal
141+
docker compose up --build
142+
```
143+
144+
Deploy shape:
145+
146+
```text
147+
render.yaml -> Docker FastAPI service + managed Postgres + S3 presigned report links
148+
```
149+
136150
## Parameters
137151

138152
| Parameter | Default | Description |
@@ -169,6 +183,7 @@ results/
169183
- **Docker and Singularity** - `-profile docker` for local, `-profile singularity` for HPC where Docker is typically unavailable.
170184
- **AWS Batch profile** - `-profile awsbatch` runs the same containerised workflow on managed cloud compute with S3 work and output paths.
171185
- **Report portal separated from compute** - Nextflow stays responsible for execution; the FastAPI portal only stores run metadata and signs S3 artefact links, which keeps the cloud proof small and auditable.
186+
- **Render Blueprint** - `render.yaml` defines the web service, managed Postgres database, demo seed run and AWS secret placeholders as reviewable infrastructure-as-code.
172187
- **Run metadata by default** - Nextflow report, timeline, trace and DAG are emitted on every run so failures and performance can be audited after the fact.
173188
- **Reverse-stranded default** - `--strandedness 2` because the airway dataset (and most modern Illumina dUTP protocols) produces reverse-stranded libraries. Users with older unstranded preps should set `--strandedness 0`.
174189
- **Configurable contrast** - `--ref_condition` sets the DESeq2 reference level. Defaults to "untreated" for the airway dataset.
@@ -178,7 +193,7 @@ results/
178193

179194
- **2 samples per condition in the demo** - underpowered for reliable DE. The DESeq2 step runs and produces output, but with n=2 the results are illustrative, not statistically robust. Proper analysis requires ≥3 replicates per condition.
180195
- **CI uses synthetic data** - the public CI proves the full software path, not the biological conclusion. Real Himes/GSE52778 runs require external FASTQs, GRCh38 HISAT2 index and Gencode annotation files.
181-
- **AWS Batch proof status** - the profile and report portal are implemented, but no public real AWS Batch run artefact is committed yet.
196+
- **AWS Batch proof status** - the profile and report portal are implemented, but no public real AWS Batch run artefact is committed yet. The report portal is the current cloud proof path until a real Batch run is published.
182197
- **No STAR option** - only HISAT2 is implemented. Adding STAR as an alternative aligner would allow benchmarking on the same data.
183198

184199
## Licence

cloud/report-portal/.dockerignore

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
.venv/
2+
__pycache__/
3+
*.pyc
4+
*.db
5+
*.sqlite
6+
.pytest_cache/
7+
tests/

cloud/report-portal/Dockerfile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,4 +12,4 @@ COPY app ./app
1212

1313
EXPOSE 8000
1414

15-
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
15+
CMD ["sh", "-c", "uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-8000}"]

cloud/report-portal/README.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@ Small FastAPI service for registering cloud RNA-seq runs and serving signed link
44

55
This is intentionally separate from the Nextflow pipeline. The pipeline remains responsible for compute and published artefacts; the portal gives reviewers and collaborators a minimal cloud-facing surface for run status and report access.
66

7+
The root route renders a small dashboard for browser review. The API remains available through `/docs`, `/runs` and `/runs/{id}/artifacts`.
8+
79
## Architecture
810

911
```text
@@ -25,6 +27,8 @@ Nextflow AWS Batch run
2527
| `DATABASE_URL` | `postgresql+psycopg://rnaseq:change_me@db:5432/rnaseq` | Metadata database. Defaults to local SQLite for development. |
2628
| `AWS_REGION` | `eu-west-2` | Region used by the AWS SDK. |
2729
| AWS credentials | IAM role, env vars, or workload identity | Required only for signed S3 URLs. |
30+
| `DEMO_RUN_ID` | `synthetic-ci-001` | Optional seed run for live demos. |
31+
| `DEMO_S3_PREFIX` | `s3://bucket/results/synthetic-ci-001` | Optional S3 prefix for the seeded demo run. |
2832

2933
## Local Smoke Run
3034

@@ -35,6 +39,8 @@ pip install -r requirements.txt
3539
uvicorn app.main:app --reload --port 8000
3640
```
3741

42+
Open <http://localhost:8000> for the dashboard or <http://localhost:8000/docs> for the OpenAPI UI.
43+
3844
Register a completed run:
3945

4046
```bash
@@ -63,6 +69,29 @@ docker run --rm -p 8000:8000 \
6369
rnaseq-report-portal
6470
```
6571

72+
Run the full local stack with Postgres:
73+
74+
```bash
75+
docker compose up --build
76+
```
77+
78+
The compose stack seeds a synthetic run so the dashboard has a visible record immediately.
79+
80+
## Render Blueprint
81+
82+
The repository root contains `render.yaml` for a reproducible Render deployment:
83+
84+
- Docker web service built from `cloud/report-portal/Dockerfile`.
85+
- Managed Postgres database connected through `DATABASE_URL`.
86+
- Optional demo run seeded through `DEMO_RUN_ID` and `DEMO_S3_PREFIX`.
87+
- AWS credentials stored as Dashboard secrets for S3 presigned URLs.
88+
89+
Open the Blueprint after the file is pushed:
90+
91+
```text
92+
https://dashboard.render.com/blueprint/new?repo=https://github.com/Ekin-Kahraman/rnaseq-nextflow-pipeline
93+
```
94+
6695
## Tests
6796

6897
```bash

0 commit comments

Comments
 (0)