Support for running on AWS HealthOmics and/or an AWS EC2 VM

[phillip-richmond-alamya]

Hello,

I'm looking to run this pipeline on AWS HealthOmics, or just on a basic EC2 VM that I've spun up with suitable resources.

The AWS HealthOmics how-to is a "TBD", and when I try to treat the EC2 VM like an HPC, it's throwing errors which I can't interpret, e.g.:

2025-12-12 19:19:18.525 miniwdl-run read configuration file :: path: "/home/ec2-user/.config/miniwdl.cfg"
2025-12-12 19:19:19.085 wdl.w:humanwgs_singleton workflow start :: name: "humanwgs_singleton", source: "/scratch/HiFi-human-WGS-WDL//workflows/singleton.wdl", line: 11, column: 1, dir: "/scratch/TestRun1/20251212_191919_humanwgs_singleton"
2025-12-12 19:19:19.099 miniwdl-run.CallCache call cache miss :: cache_file: "/scratch/TestRun1/miniwdl_call_cache/humanwgs_singleton/3jn67asldcpbsyizerumkhdsfd23cffm/fgzuah6dz6r4pkg2hz7olelsxdc46kud.json"
2025-12-12 19:19:19.100 wdl.w:humanwgs_singleton miniwdl :: version: "v1.13.1", uname: "Linux ip-172-31-2-223.us-west-1.compute.internal 6.1.38-59.109.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jul 11 23:51:29 UTC 2023 x86_64"
2025-12-12 19:19:19.101 wdl.w:humanwgs_singleton task thread pool initialized :: task_concurrency: 64
2025-12-12 19:19:19.105 wdl.w:humanwgs_singleton visit :: node: "decl-backend", values: {"backend": "HPC"}
2025-12-12 19:19:19.105 wdl.w:humanwgs_singleton visit :: node: "decl-container_registry", values: {"container_registry": null}
2025-12-12 19:19:19.105 wdl.w:humanwgs_singleton visit :: node: "decl-cpuPlatform", values: {"cpuPlatform": null}
2025-12-12 19:19:19.105 wdl.w:humanwgs_singleton visit :: node: "decl-debug_version", values: {"debug_version": null}
2025-12-12 19:19:19.105 wdl.w:humanwgs_singleton visit :: node: "decl-fail_reads", values: {"fail_reads": null}
2025-12-12 19:19:19.105 wdl.w:humanwgs_singleton visit :: node: "decl-gpu", values: {"gpu": false}
2025-12-12 19:19:19.105 wdl.w:humanwgs_singleton visit :: node: "decl-gpuType", values: {"gpuType": null}
2025-12-12 19:19:19.105 wdl.w:humanwgs_singleton visit :: node: "decl-hifi_reads", values: {"hifi_reads": ["/scratch/Data/F160849_m84185_250917_205836_s1.hifi_reads.bam"]}
2025-12-12 19:19:19.105 wdl.w:humanwgs_singleton visit :: node: "decl-max_reads_per_alignment_chunk", values: {"max_reads_per_alignment_chunk": 500000}
2025-12-12 19:19:19.106 wdl.w:humanwgs_singleton visit :: node: "decl-pedigree_sex", values: {"pedigree_sex": {"MALE": "1", "FEMALE": "2", "": "."}}
2025-12-12 19:19:19.106 wdl.w:humanwgs_singleton visit :: node: "decl-pharmcat_min_coverage", values: {"pharmcat_min_coverage": 10}
2025-12-12 19:19:19.106 wdl.w:humanwgs_singleton visit :: node: "decl-phenotypes", values: {"phenotypes": "HP:0000001"}
2025-12-12 19:19:19.106 wdl.w:humanwgs_singleton visit :: node: "decl-preemptible", values: {"preemptible": true}
2025-12-12 19:19:19.106 wdl.w:humanwgs_singleton visit :: node: "decl-ref_map_file", values: {"ref_map_file": "/scratch/hifi-wdl-resources-v3.1.0/GRCh38.ref_map.v3p1p0.template.tsv"}
2025-12-12 19:19:19.106 wdl.w:humanwgs_singleton visit :: node: "decl-sample_id", values: {"sample_id": "F160849"}
2025-12-12 19:19:19.106 wdl.w:humanwgs_singleton visit :: node: "decl-sex", values: {"sex": "MALE"}
2025-12-12 19:19:19.106 wdl.w:humanwgs_singleton visit :: node: "decl-tertiary_map_file", values: {"tertiary_map_file": "/scratch/hifi-wdl-resources-v3.1.0/GRCh38.tertiary_map.v3p1p0.template.tsv"}
2025-12-12 19:19:19.106 wdl.w:humanwgs_singleton visit :: node: "decl-zones", values: {"zones": null}
2025-12-12 19:19:19.106 wdl.w:humanwgs_singleton visit :: node: "output-workflow_name", values: {"workflow_name": "humanwgs_singleton"}
2025-12-12 19:19:19.107 wdl.w:humanwgs_singleton ready :: job: "call-backend_configuration", callee: "backend_configuration"
2025-12-12 19:19:19.107 wdl.w:humanwgs_singleton input :: job: "call-backend_configuration", values: {"container_registry": null, "gpuType": null, "cpuPlatform": null, "zones": null, "backend": "HPC"}
2025-12-12 19:19:19.107 wdl.w:humanwgs_singleton subworkflow thread pool initialized :: subworkflow_concurrency: 64, call_depth: 0
2025-12-12 19:19:19.108 wdl.w:humanwgs_singleton.w:call-backend_configuration workflow start :: name: "backend_configuration", source: "wdl-common/wdl/workflows/backend_configuration/backend_configuration.wdl", line: 7, column: 1, dir: "/scratch/TestRun1/20251212_191919_humanwgs_singleton/call-backend_configuration"
2025-12-12 19:19:19.110 wdl.w:humanwgs_singleton workflow humanwgs_singleton (/scratch/HiFi-human-WGS-WDL//workflows/singleton.wdl Ln 11 Col 1) failed :: dir: "/scratch/TestRun1/20251212_191919_humanwgs_singleton", error: "EvalError", message: "function evaluation failed, read_map(): each line must have two fields", node: "decl-ref_map", pos: {"source": "/scratch/HiFi-human-WGS-WDL/workflows/singleton.wdl", "line": 112, "column": 33}
2025-12-12 19:19:19.111 wdl.w:humanwgs_singleton aborting workflow
2025-12-12 19:19:19.111 miniwdl-run.CallCache call cache hit :: run_dir: "/scratch/TestRun1/20251212_191310_humanwgs_singleton/call-backend_configuration", cache_file: "/scratch/TestRun1/miniwdl_call_cache/backend_configuration/d7522xdekeylaozzmrjhg4rlmu6alv4d/wli4zllozrwkmn3hqlqnfbb3ll5brdau.json"
2025-12-12 19:19:19.111 wdl.w:humanwgs_singleton.w:call-backend_configuration cached output :: name: "spot_runtime_attributes", value: {"backend": "HPC", "preemptible_tries": 0, "max_retries": 3, "zones": "", "cpuPlatform": "", "gpuType": "", "container_registry": "quay.io/pacbio", "container_namespace": null}
2025-12-12 19:19:19.112 wdl.w:humanwgs_singleton.w:call-backend_configuration cached output :: name: "on_demand_runtime_attributes", value: {"backend": "HPC", "preemptible_tries": 0, "max_retries": 3, "zones": "", "cpuPlatform": "", "gpuType": "", "container_registry": "quay.io/pacbio", "container_namespace": null}
2025-12-12 19:19:19.112 wdl.w:humanwgs_singleton.w:call-backend_configuration done (cached)
2025-12-12 19:19:19.112 miniwdl-run function evaluation failed, read_map(): each line must have two fields :: error: "EvalError", node: "decl-ref_map", pos: {"source": "/scratch/HiFi-human-WGS-WDL/workflows/singleton.wdl", "line": 112, "column": 33}, dir: "/scratch/TestRun1/20251212_191919_humanwgs_singleton"

My singleton json:

{
  "humanwgs_singleton.sample_id": "F160849",
  "humanwgs_singleton.sex": "MALE",
  "humanwgs_singleton.hifi_reads": [
    "/scratch/Data/F160849_m84185_250917_205836_s1.hifi_reads.bam"
  ],
  "humanwgs_singleton.ref_map_file": "/scratch/hifi-wdl-resources-v3.1.0/GRCh38.ref_map.v3p1p0.template.tsv",
  "humanwgs_singleton.tertiary_map_file": "/scratch/hifi-wdl-resources-v3.1.0/GRCh38.tertiary_map.v3p1p0.template.tsv",
  "humanwgs_singleton.backend": "HPC",
  "humanwgs_singleton.preemptible": true
}

I've got the resource files set (ref map template)

name    GRCh38
fasta   /scratch/hifi-wdl-resources-v3.1.0/GRCh38/human_GRCh38_no_alt_analysis_set.fasta
fasta_index	/scratch/hifi-wdl-resources-v3.1.0/GRCh38/human_GRCh38_no_alt_analysis_set.fasta.fai
trgt_tandem_repeat_bed	/scratch/hifi-wdl-resources-v3.1.0/GRCh38/trgt/adotto_strchive_20250827.hg38.bed.gz
sawfish_exclude_bed	/scratch/hifi-wdl-resources-v3.1.0/GRCh38/sawfish/annotation_and_common_cnv.hg38.bed.gz
sawfish_exclude_bed_index	/scratch/hifi-wdl-resources-v3.1.0/GRCh38/sawfish/annotation_and_common_cnv.hg38.bed.gz.tbi
sawfish_expected_bed_male	/scratch/hifi-wdl-resources-v3.1.0/GRCh38/sawfish/expected_cn.hg38.XY.bed
sawfish_expected_bed_female	/scratch/hifi-wdl-resources-v3.1.0/GRCh38/sawfish/expected_cn.hg38.XX.bed
methbat_region_tsv	/scratch/hifi-wdl-resources-v3.1.0/GRCh38/methbat/cpgIslandExt.sorted.hg38.tsv
pharmcat_positions_vcf	/scratch/hifi-wdl-resources-v3.1.0/GRCh38/pharmcat/pharmcat_positions_2.15.4.vcf.bgz
pharmcat_positions_vcf_index	/scratch/hifi-wdl-resources-v3.1.0/GRCh38/pharmcat/pharmcat_positions_2.15.4.vcf.bgz.csi

tertiary map template

slivar_js	/scratch//hifi-wdl-resources-v3.1.0/slivar/slivar-functions.v0.2.8.js
ensembl_gff	/scratch//hifi-wdl-resources-v3.1.0/GRCh38/ensembl.GRCh38.101.reformatted.gff3.gz
lof_lookup	/scratch//hifi-wdl-resources-v3.1.0/slivar/lof.gnomadv4p1.lookup
clinvar_lookup	/scratch//hifi-wdl-resources-v3.1.0/slivar/clinvar_gene_desc.20250618T144412.txt
slivar_gnotate_files	/scratch//hifi-wdl-resources-v3.1.0/GRCh38/slivar_gnotate/gnomad.hg38.v4.1.custom.v1.zip,/scratch//hifi-wdl-resources-v3.1.0/GRCh38/slivar_gnotate/CoLoRSdb.GRCh38.v1.2.0.deepvariant.glnexus.zip
slivar_gnotate_prefixes	gnomad,colors
slivar_max_af	0.03
slivar_max_nhomalt	4
slivar_max_ac	4
slivar_min_gq	5
svpack_pop_vcfs	/scratch//hifi-wdl-resources-v3.1.0/GRCh38/sv_pop_vcfs/gnomad.v4.1.sv.sites.pass.vcf.gz,/scratch//hifi-wdl-resources-v3.1.0/GRCh38/sv_pop_vcfs/CoLoRSdb.GRCh38.v1.2.0.pbsv.jasmine.vcf.gz
svpack_pop_vcf_indices	/scratch//hifi-wdl-resources-v3.1.0/GRCh38/sv_pop_vcfs/gnomad.v4.1.sv.sites.pass.vcf.gz.tbi,/scratch//hifi-wdl-resources-v3.1.0/GRCh38/sv_pop_vcfs/CoLoRSdb.GRCh38.v1.2.0.pbsv.jasmine.vcf.gz.tbi

Any support would be grateful!

Thanks,
Phil

[williamrowell]

Hi Phil,

I'm going to split this into two problems, and we'll tackle them differently.

1. running on AWS HealthOmics

Please reach out to support@pacb.com, reference this issue and mention that the question should get targeted to Matt Seetin and Billy. We'll help you out.

2. "hpc" read_map(): each line must have two fields

Thanks for providing the log and inputs! It's hard to be sure, because cut and paste could be playing tricks here, but is it possible that one or more lines in your your ref_map file are delimited by multiple spaces instead of a single tab? Maybe cat -A {ref_map} to see if there are multiple delimiters/whitespaces on any of the lines (specifically the name and fasta lines).