Skip to content

Add wget wait time for retries#325

Merged
muffato merged 6 commits into
nf-core:devfrom
maxibor:wget_wait
Apr 24, 2026
Merged

Add wget wait time for retries#325
muffato merged 6 commits into
nf-core:devfrom
maxibor:wget_wait

Conversation

@maxibor

@maxibor maxibor commented Nov 29, 2024

Copy link
Copy Markdown
Member

When downloading from EBI via ftp, I get frequent retries (both via wget, and via process retries) because of a failed: Connection refused.

Adding a waiting time seems to help a little bit

excerpt from wget manual

-w seconds
       --wait=seconds
           Wait the specified number of seconds between the retrievals.  Use of this option is recommended, as it lightens the
           server load by making the requests less frequent.  Instead of in seconds, the time can be specified in minutes using
           the "m" suffix, in hours using "h" suffix, or in days using "d" suffix.

           Specifying a large value for this option is useful if the network or the destination host is down, so that Wget can
           wait long enough to reasonably expect the network error to be fixed before the retry.  The waiting interval
           specified by this function is influenced by "--random-wait", which see.

PR checklist

  • This comment contains a description of changes (with reason).
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@maxibor maxibor changed the title Wget wait Add wget wait time for retries Nov 29, 2024
@github-actions

github-actions Bot commented Nov 29, 2024

Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 6869637

+| ✅ 181 tests passed       |+
#| ❔  14 tests were ignored |#
!| ❗   1 tests had warnings |!
Details

❗ Test warnings:

❔ Tests ignored:

  • files_exist - File is ignored: .github/workflows/awsfulltest.yml
  • files_exist - File is ignored: .github/workflows/awstest.yml
  • files_exist - File is ignored: assets/multiqc_config.yml
  • files_exist - File is ignored: conf/igenomes.config
  • files_exist - File is ignored: conf/igenomes_ignored.config
  • files_exist - File is ignored: conf/modules.config
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: assets/sendmail_template.txt
  • files_unchanged - File ignored due to lint config: assets/nf-core-fetchngs_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-fetchngs_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-fetchngs_logo_dark.png
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/fetchngs/fetchngs/.github/workflows/awstest.yml
  • multiqc_config - multiqc_config
  • modules_config - modules_config

✅ Tests passed:

Run details

  • nf-core/tools version 3.3.2
  • Run at 2026-04-24 14:16:51

@maxibor maxibor requested a review from drpatelh December 5, 2024 12:00
@nf-core-bot

Copy link
Copy Markdown
Member

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.0.2.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

@maxulysse maxulysse added this to the 1.13.0 milestone May 19, 2025
@muffato muffato merged commit bc80aaf into nf-core:dev Apr 24, 2026
6 checks passed
@muffato

muffato commented Apr 24, 2026

Copy link
Copy Markdown
Member

That's useful to have. Thank you !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants