Skip to content

Commit 7fcde02

Browse files
authored
Merge pull request #105 from seqeralabs/cursor/benchmark-report-tarball-input-b8c1
Add tarball input support for benchmark report
2 parents 625d093 + 91abed7 commit 7fcde02

60 files changed

Lines changed: 5142 additions & 374 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.editorconfig

Lines changed: 11 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -11,27 +11,20 @@ indent_style = space
1111
[*.{md,yml,yaml,html,css,scss,js}]
1212
indent_size = 2
1313

14-
# These files are edited and tested upstream in nf-core/modules
15-
[/modules/nf-core/**]
16-
charset = unset
17-
end_of_line = unset
18-
insert_final_newline = unset
19-
trim_trailing_whitespace = unset
20-
indent_style = unset
21-
[/subworkflows/nf-core/**]
22-
charset = unset
23-
end_of_line = unset
24-
insert_final_newline = unset
25-
trim_trailing_whitespace = unset
14+
# ignore python and markdown
15+
[*.{py,md}]
2616
indent_style = unset
2717

28-
[/assets/email*]
29-
indent_size = unset
18+
# ignore ro-crate metadata files
19+
[**/ro-crate-metadata.json]
20+
insert_final_newline = unset
3021

31-
# ignore python and markdown
32-
[*.{py,md}]
22+
# ignore auto-generated multiqc report
23+
[assets/multiqc_report.html]
3324
indent_style = unset
25+
indent_size = unset
3426

35-
# ignore multiqc example
36-
[/assets/multiqc_report.html]
27+
# ignore Jinja HTML template (CSS alignment requires non-standard indentation)
28+
[bin/benchmark_report_template.html]
3729
indent_style = unset
30+
indent_size = unset

.github/.dockstore.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Dockstore config version, not pipeline version
2+
version: 1.2
3+
workflows:
4+
- subclass: nfl
5+
primaryDescriptorPath: /nextflow.config
6+
publish: True

.github/CONTRIBUTING.md

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -23,31 +23,16 @@ If you're not used to this workflow with git, you can start with some [docs from
2323

2424
## Tests
2525

26-
You have the option to test your changes locally by running the pipeline. For receiving warnings about process selectors and other `debug` information, it is recommended to use the debug profile. Execute all the tests with the following command:
27-
28-
```bash
29-
nf-test test --profile debug,test,docker --verbose
30-
```
31-
3226
When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
3327
Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.
3428

35-
There are typically two types of tests that run:
36-
3729
### Lint tests
3830

3931
`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to.
4032
To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core pipelines lint <pipeline-directory>` command.
4133

4234
If any failures or warnings are encountered, please follow the listed URL for more documentation.
4335

44-
### Pipeline tests
45-
46-
Each `nf-core` pipeline should be set up with a minimal set of test-data.
47-
`GitHub Actions` then runs the pipeline on this data to ensure that it exits successfully.
48-
If there are any failures then the automated tests fail.
49-
These tests are run both with the latest available version of `Nextflow` and also the minimum required version that is stated in the pipeline code.
50-
5136
## Patch
5237

5338
:warning: Only in the unlikely and regretful event of a release happening with a bug.

.github/PULL_REQUEST_TEMPLATE.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,14 @@ These are the most common things requested on pull requests (PRs).
88
99
Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release.
1010
11-
Learn more about contributing: [CONTRIBUTING.md](https://github.com/seqeralabs/nf-aggregate/tree/master/.github/CONTRIBUTING.md)
11+
Learn more about contributing: [CONTRIBUTING.md](https://github.com/seqeralabs/nf-aggregate/tree/main/.github/CONTRIBUTING.md)
1212
-->
1313

1414
## PR checklist
1515

1616
- [ ] This comment contains a description of changes (with reason).
1717
- [ ] If you've fixed a bug or added code that should be tested, add tests!
18-
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/seqeralabs/nf-aggregate/tree/master/.github/CONTRIBUTING.md)
18+
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/seqeralabs/nf-aggregate/tree/main/.github/CONTRIBUTING.md)
1919
- [ ] Make sure your code lints (`nf-core pipelines lint`).
2020
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
2121
- [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`).

.github/workflows/branch.yml

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
name: nf-core branch protection
2+
# This workflow is triggered on PRs to `main`/`master` branch on the repository
3+
# It fails when someone tries to make a PR against the nf-core `main`/`master` branch instead of `dev`
4+
on:
5+
pull_request_target:
6+
branches:
7+
- main
8+
- master
9+
10+
jobs:
11+
test:
12+
runs-on: ubuntu-latest
13+
steps:
14+
# PRs to the nf-core repo main/master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
15+
- name: Check PRs
16+
if: github.repository == 'seqeralabs/nf-aggregate'
17+
run: |
18+
{ [[ ${{github.event.pull_request.head.repo.full_name }} == seqeralabs/nf-aggregate ]] && [[ $GITHUB_HEAD_REF == "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
19+
20+
# If the above check failed, post a comment on the PR explaining the failure
21+
# NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets
22+
- name: Post PR comment
23+
if: failure()
24+
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
25+
with:
26+
message: |
27+
## This PR is against the `${{github.event.pull_request.base.ref}}` branch :x:
28+
29+
* Do not close this PR
30+
* Click _Edit_ and change the `base` to `dev`
31+
* This CI test will remain failed until you push a new commit
32+
33+
---
34+
35+
Hi @${{ github.event.pull_request.user.login }},
36+
37+
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) ${{github.event.pull_request.base.ref}} branch.
38+
The ${{github.event.pull_request.base.ref}} branch on nf-core repositories should always contain code from the latest release.
39+
Because of this, PRs to ${{github.event.pull_request.base.ref}} are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
40+
41+
You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page.
42+
Note that even after this, the test will continue to show as failing until you push a new commit.
43+
44+
Thanks again for your contribution!
45+
repo-token: ${{ secrets.GITHUB_TOKEN }}
46+
allow-repeats: false

.github/workflows/clean-up.yml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
name: "Close user-tagged issues and PRs"
2+
on:
3+
schedule:
4+
- cron: "0 0 * * 0" # Once a week
5+
6+
jobs:
7+
clean-up:
8+
runs-on: ubuntu-latest
9+
permissions:
10+
issues: write
11+
pull-requests: write
12+
steps:
13+
- uses: actions/stale@28ca1036281a5e5922ead5184a1bbf96e5fc984e # v9
14+
with:
15+
stale-issue-message: "This issue has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment otherwise this issue will be closed in 20 days."
16+
stale-pr-message: "This PR has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment if it is still useful."
17+
close-issue-message: "This issue was closed because it has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor and then staled for 20 days with no activity."
18+
days-before-stale: 30
19+
days-before-close: 20
20+
days-before-pr-close: -1
21+
any-of-labels: "awaiting-changes,awaiting-feedback"
22+
exempt-issue-labels: "WIP"
23+
exempt-pr-labels: "WIP"
24+
repo-token: "${{ secrets.GITHUB_TOKEN }}"
Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
name: Test successful pipeline download with 'nf-core pipelines download'
2+
3+
# Run the workflow when:
4+
# - dispatched manually
5+
# - when a PR is opened or reopened to main/master branch
6+
# - the head branch of the pull request is updated, i.e. if fixes for a release are pushed last minute to dev.
7+
on:
8+
workflow_dispatch:
9+
inputs:
10+
testbranch:
11+
description: "The specific branch you wish to utilize for the test execution of nf-core pipelines download."
12+
required: true
13+
default: "dev"
14+
pull_request:
15+
types:
16+
- opened
17+
- edited
18+
- synchronize
19+
branches:
20+
- main
21+
- master
22+
pull_request_target:
23+
branches:
24+
- main
25+
- master
26+
27+
env:
28+
NXF_ANSI_LOG: false
29+
30+
jobs:
31+
configure:
32+
runs-on: ubuntu-latest
33+
outputs:
34+
REPO_LOWERCASE: ${{ steps.get_repo_properties.outputs.REPO_LOWERCASE }}
35+
REPOTITLE_LOWERCASE: ${{ steps.get_repo_properties.outputs.REPOTITLE_LOWERCASE }}
36+
REPO_BRANCH: ${{ steps.get_repo_properties.outputs.REPO_BRANCH }}
37+
steps:
38+
- name: Get the repository name and current branch
39+
id: get_repo_properties
40+
run: |
41+
echo "REPO_LOWERCASE=${GITHUB_REPOSITORY,,}" >> "$GITHUB_OUTPUT"
42+
echo "REPOTITLE_LOWERCASE=$(basename ${GITHUB_REPOSITORY,,})" >> "$GITHUB_OUTPUT"
43+
echo "REPO_BRANCH=${{ github.event.inputs.testbranch || 'dev' }}" >> "$GITHUB_OUTPUT"
44+
45+
download:
46+
runs-on: ubuntu-latest
47+
needs: configure
48+
steps:
49+
- name: Install Nextflow
50+
uses: nf-core/setup-nextflow@v2
51+
52+
- name: Disk space cleanup
53+
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
54+
55+
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5
56+
with:
57+
python-version: "3.12"
58+
architecture: "x64"
59+
60+
- name: Setup Apptainer
61+
uses: eWaterCycle/setup-apptainer@4bb22c52d4f63406c49e94c804632975787312b3 # v2.0.0
62+
with:
63+
apptainer-version: 1.3.4
64+
65+
- name: Install dependencies
66+
run: |
67+
python -m pip install --upgrade pip
68+
pip install git+https://github.com/nf-core/tools.git@dev
69+
70+
- name: Make a cache directory for the container images
71+
run: |
72+
mkdir -p ./singularity_container_images
73+
74+
- name: Download the pipeline
75+
env:
76+
NXF_SINGULARITY_CACHEDIR: ./singularity_container_images
77+
run: |
78+
nf-core pipelines download ${{ needs.configure.outputs.REPO_LOWERCASE }} \
79+
--revision ${{ needs.configure.outputs.REPO_BRANCH }} \
80+
--outdir ./${{ needs.configure.outputs.REPOTITLE_LOWERCASE }} \
81+
--compress "none" \
82+
--container-system 'singularity' \
83+
--container-library "quay.io" -l "docker.io" -l "community.wave.seqera.io/library/" \
84+
--container-cache-utilisation 'amend' \
85+
--download-configuration 'yes'
86+
87+
- name: Inspect download
88+
run: tree ./${{ needs.configure.outputs.REPOTITLE_LOWERCASE }}
89+
90+
- name: Inspect container images
91+
run: tree ./singularity_container_images | tee ./container_initial

.github/workflows/fix-linting.yml

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
name: Fix linting from a comment
2+
on:
3+
issue_comment:
4+
types: [created]
5+
6+
jobs:
7+
fix-linting:
8+
# Only run if comment is on a PR with the main repo, and if it contains the magic keywords
9+
if: >
10+
contains(github.event.comment.html_url, '/pull/') &&
11+
contains(github.event.comment.body, '@nf-core-bot fix linting') &&
12+
github.repository == 'seqeralabs/nf-aggregate'
13+
runs-on: ubuntu-latest
14+
steps:
15+
# Use the @nf-core-bot token to check out so we can push later
16+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
17+
with:
18+
token: ${{ secrets.nf_core_bot_auth_token }}
19+
20+
# indication that the linting is being fixed
21+
- name: React on comment
22+
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
23+
with:
24+
comment-id: ${{ github.event.comment.id }}
25+
reactions: eyes
26+
27+
# Action runs on the issue comment, so we don't get the PR by default
28+
# Use the gh cli to check out the PR
29+
- name: Checkout Pull Request
30+
run: gh pr checkout ${{ github.event.issue.number }}
31+
env:
32+
GITHUB_TOKEN: ${{ secrets.nf_core_bot_auth_token }}
33+
34+
# Install and run pre-commit
35+
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5
36+
with:
37+
python-version: "3.12"
38+
39+
- name: Install pre-commit
40+
run: pip install pre-commit
41+
42+
- name: Run pre-commit
43+
id: pre-commit
44+
run: pre-commit run --all-files
45+
continue-on-error: true
46+
47+
# indication that the linting has finished
48+
- name: react if linting finished succesfully
49+
if: steps.pre-commit.outcome == 'success'
50+
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
51+
with:
52+
comment-id: ${{ github.event.comment.id }}
53+
reactions: "+1"
54+
55+
- name: Commit & push changes
56+
id: commit-and-push
57+
if: steps.pre-commit.outcome == 'failure'
58+
run: |
59+
git config user.email "core@nf-co.re"
60+
git config user.name "nf-core-bot"
61+
git config push.default upstream
62+
git add .
63+
git status
64+
git commit -m "[automated] Fix code linting"
65+
git push
66+
67+
- name: react if linting errors were fixed
68+
id: react-if-fixed
69+
if: steps.commit-and-push.outcome == 'success'
70+
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
71+
with:
72+
comment-id: ${{ github.event.comment.id }}
73+
reactions: hooray
74+
75+
- name: react if linting errors were not fixed
76+
if: steps.commit-and-push.outcome == 'failure'
77+
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
78+
with:
79+
comment-id: ${{ github.event.comment.id }}
80+
reactions: confused
81+
82+
- name: react if linting errors were not fixed
83+
if: steps.commit-and-push.outcome == 'failure'
84+
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4
85+
with:
86+
issue-number: ${{ github.event.issue.number }}
87+
body: |
88+
@${{ github.actor }} I tried to fix the linting errors, but it didn't work. Please fix them manually.
89+
See [CI log](https://github.com/seqeralabs/nf-aggregate/actions/runs/${{ github.run_id }}) for more details.

.github/workflows/linting.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,10 +14,10 @@ jobs:
1414
pre-commit:
1515
runs-on: ubuntu-latest
1616
steps:
17-
- uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
17+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
1818

1919
- name: Set up Python 3.12
20-
uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5
20+
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5
2121
with:
2222
python-version: "3.12"
2323

@@ -31,12 +31,12 @@ jobs:
3131
runs-on: ubuntu-latest
3232
steps:
3333
- name: Check out pipeline code
34-
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4
34+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
3535

3636
- name: Install Nextflow
3737
uses: nf-core/setup-nextflow@v2
3838

39-
- uses: actions/setup-python@82c7e631bb3cdc910f68e0081d67478d79c6982d # v5
39+
- uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5
4040
with:
4141
python-version: "3.12"
4242
architecture: "x64"
@@ -74,7 +74,7 @@ jobs:
7474

7575
- name: Upload linting log file artifact
7676
if: ${{ always() }}
77-
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4
77+
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4
7878
with:
7979
name: linting-logs
8080
path: |

0 commit comments

Comments
 (0)