Skip to content

Commit dfe535f

Browse files
committed
ingest/ncbi: Add retries for rules that fetch data
Allow rules that fetch data to retry in case of network errors. Allow 5 retries, which is the usual number we use for the `fetch_ncbi_dataset_package` rule across ingest workflows.¹ Resolves #154 ¹ <https://github.com/nextstrain/pathogen-repo-guide/blob/a227dc9485b0291f0d0848168d81a48e15e600ac/ingest/rules/fetch_from_ncbi.smk#L44-L45>
1 parent 7123c75 commit dfe535f

2 files changed

Lines changed: 4 additions & 0 deletions

File tree

ingest/build-configs/ncbi/rules/fetch_from_ncbi.smk

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,8 @@ rule fetch_from_ncbi_virus:
3131
ncbi_taxon_id=config["ncbi_taxon_id"],
3232
ncbi_collection_date_filter=_get_date_filter(),
3333
ncbi_virus_filters=" ".join(f"{filter!r}" for filter in config["ncbi_virus_filters"]),
34+
# Allow retries in case of network errors
35+
retries: 5
3436
shell:
3537
"""
3638
./build-configs/ncbi/bin/fetch-from-ncbi-virus \

ingest/build-configs/ncbi/rules/ingest_andersen_lab.smk

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ from the Andersen Lab's avian-influenza repo
77
rule fetch_andersen_lab_repo:
88
output:
99
andersen_lab_repo = temp("andersen-lab/data/avian-influenza.tar.gz")
10+
# Allow retries in case of network errors
11+
retries: 5
1012
shell:
1113
"""
1214
curl -fsSL \

0 commit comments

Comments
 (0)