Skip to content

cwltool Freezes When Not Using Inputs in Scatter Parallelization #2031

Open
@gen740

Description

@gen740

Expected Behavior

Successfully runs.

Actual Behavior

cwltool freezes.

Workflow Code

cwlVersion: v1.2
class: Workflow
inputs:
  targets:
    type: string[]
requirements:
  ScatterFeatureRequirement: {}
steps:
  job:
    in:
      target: targets
    scatter: target
    scatterMethod: dotproduct
    run:
      inputs:
        target:
          type: string
      class: CommandLineTool
      baseCommand: ls
      outputs: []
    out: []
outputs: []
  • with cwltool --parallel --cachedir=./cache repro.cwl --targets "A" --targets "B"

Full Traceback

INFO /Users/gen/home/tmp/venv/bin/cwltool 3.1.20240708091337
INFO Resolved 'test.cwl' to 'file:///Users/gen/home/tmp/test.cwl'
WARNING Error checking CUDA version with nvidia-smi: [Errno 2] No such file or directory: 'nvidia-smi'
DEBUG Parsed job order from command line: {
    "__id": "test.cwl",
    "targets": [
        "A",
        "B"
    ]
}
DEBUG [workflow ] initialized from file:///Users/gen/home/tmp/test.cwl
DEBUG [workflow ] inputs {
    "targets": [
        "A",
        "B"
    ]
}
INFO [workflow ] starting step job
INFO [step job] start
DEBUG job: <cwltool.workflow_job.WorkflowJob object at 0x105162510>, runtime_context: <cwltool.context.RuntimeContext object at 0x102414610>, TMPDIR_LOCK: <unlocked _thread.lock object at 0x104cabc80>
INFO [workflow ] start
DEBUG [job job] keydictstr is {"cmdline":["ls"]} -> a09766136709aa2e858984e35fe3ca63
INFO [job job] Output of job will be cached in /Users/gen/home/tmp/.cache/a09766136709aa2e858984e35fe3ca63
DEBUG [job job] initializing from _:0b038dc5-e3da-45de-8f98-d4b5ce005d90 as part of step job
DEBUG [job job] {
    "target": "A"
}
DEBUG [job job] path mappings is {}
DEBUG [job job] command line bindings is [
    {
        "position": [
            -1000000,
            0
        ],
        "datum": "ls"
    }
]
INFO [step job] start
DEBUG job: CommandLineJob(job), runtime_context: <cwltool.context.RuntimeContext object at 0x102414610>, TMPDIR_LOCK: <unlocked _thread.lock object at 0x104cabc80>
DEBUG [job job] initial work dir {}
INFO [job job] /Users/gen/home/tmp/.cache/a09766136709aa2e858984e35fe3ca63$ ls
DEBUG [job job_2] keydictstr is {"cmdline":["ls"]} -> a09766136709aa2e858984e35fe3ca63
DEBUG Could not collect memory usage, job ended before monitoring began.
INFO [job job] completed success
DEBUG [job job] outputs {}

Your Environment

  • cwltool version: /Users/gen/home/tmp/venv/bin/cwltool 3.1.20240708091337
  • macOS Sonoma 14.6.1
  • python 3.12.4, 3.11.9

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions