Skip to content

Reporting slowness of local miniwdl run #140

@pichuan

Description

@pichuan

My experimental run was from roughly 11 days ago. @adamnovak asked me to write down what I have observed so far.

So, roughly:

  1. I read on https://github.com/vgteam/vg_wdl/tree/gbz#giraffe-workflow
  2. Then I made a local setup for myself on a full WGS FASTQ files. I ran as root. I downloaded files to local filesystems first. My json file is params/giraffe-pichuan.json, which looks like:
root@pichuan-cpu:/home/pichuan/vg_wdl# cat params/giraffe-pichuan.json
{
  "Giraffe.INPUT_READ_FILE_1": "/root/giraffe-pichuan-input/HG002.novaseq.pcr-free.30x.R1.fastq.gz",
  "Giraffe.INPUT_READ_FILE_2": "/root/giraffe-pichuan-input/HG002.novaseq.pcr-free.30x.R2.fastq.gz",
  "Giraffe.GBZ_FILE": "/root/giraffe-pichuan-input/hprc-v1.0-mc-chm13-minaf.0.1.gbz",
  "Giraffe.REFERENCE_PREFIX": "GRCh38.",
  "Giraffe.SAMPLE_NAME": "s0",
  "Giraffe.MIN_FILE": "/root/giraffe-pichuan-input/hprc-v1.0-mc-chm13-minaf.0.1.min",
  "Giraffe.DIST_FILE": "/root/giraffe-pichuan-input/hprc-v1.0-mc-chm13-minaf.0.1.dist",
  "Giraffe.OUTPUT_GAF": true,
  "Giraffe.OUTPUT_CALLING_BAMS": true,
  "Giraffe.OUTPUT_SINGLE_BAM": true
}

The command I ran was:

time miniwdl run --as-me workflows/giraffe.wdl -i params/giraffe-pichuan.json

The strange thing is that the vg process step didn't finish after 3 days. @jmonlong asked me to look at the line that says:

Achieved ??? reads per CPU-second (including output)

From that number, it somehow seems like the speed of my run was 27x slower than what @jmonlong was seeing.

Since then, I've moved to just running vg giraffe directly instead of using miniwdl. Directly running vg giraffe worked well for me so far. I'm reporting this to @adamnovak so that he can make sure the miniwdl path works fine.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions