Cannot run CWL workflow with DockerRequirement: Not enough available locations: job /0 requires 1 locations on deployment

[kinow]

Hi,

I have this workflow that runs OK with cwltool and Toil:

#!/usr/bin/env cwl-runner

# SPDX-License-Identifier: CC-BY-4.0
#
# This file is licensed under CC BY 4.0
# https://creativecommons.org/licenses/by/4.0/
#
# Copyright (c) 2026 Bruno de Paula Kinoshita

cwlVersion: v1.2

# --- Metadata ---

$namespaces:
  s: https://schema.org/
  xsd: http://www.w3.org/2001/XMLSchema#

$schemas:
  - https://schema.org/version/latest/schemaorg-current-http.rdf

s:author:
  class: s:Person
  s:identifier: https://orcid.org/0000-0001-8250-4074
  s:email: mailto:bruno.depaulakinoshita@bsc.es
  s:name: Bruno de Paula Kinoshita

s:dateCreated:
  "@type": "xsd:date"
  "@value": "2026-02-13"

s:license: https://creativecommons.org/licenses/by/4.0/

s:codeRepository: https://github.com/kinow/cwl-mpi

# --- CWL Tool ---

id: mpich-mpicc
class: CommandLineTool
label: mpich-mpicc
baseCommand: mpicc
doc: >
  An example CWL tool that compiles a C program using the `mpicc`
  compiler wrapper.
  
  By default, it compiles the `sr.c` source file using the `mpicc`
  compiler wrapper. The tool can also be used to compile other
  C programs by providing a different value for source.
  
  It uses the MPICH 4.3.2 Docker image as a hint.

hints:
  DockerRequirement:
    dockerPull: mfisherman/mpich:4.3.2

requirements:
  InlineJavascriptRequirement: { }
  NetworkAccess:
    networkAccess: false
  ResourceRequirement:
    coresMin: 1
    ramMin: 512

inputs:
  source:
    type: File
    inputBinding:
      position: 1
    default:
      class: File
      path: sr.c
  output_name:
    type: string
    default: sr
    inputBinding:
      prefix: -o
      position: 2

outputs:
  executable:
    type: File
    outputBinding:
      glob: $(inputs.output_name)

cwltool

When I run it with cwltool with no containers:

(venv) kinow@ranma:~/Development/python/workspace/cwl-mpi/workflows/mpich-sr$ cwltool mpicc.cwl 
INFO /home/kinow/Development/python/workspace/cwltool/venv/bin/cwltool 3.1.20260315121657
INFO Resolved 'mpicc.cwl' to 'file:///home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/mpicc.cwl'
WARNING [job mpich-mpicc] Skipping Docker software container '--memory' limit despite presence of ResourceRequirement with ramMin and/or ramMax setting. Consider running with --strict-memory-limit for increased portability assurance.
WARNING [job mpich-mpicc] Skipping Docker software container '--cpus' limit despite presence of ResourceRequirement with coresMin and/or coresMax setting. Consider running with --strict-cpu-limit for increased portability assurance.
INFO [job mpich-mpicc] /tmp/m4fvyhd4$ docker \
    run \
    -i \
    --mount=type=bind,source=/tmp/m4fvyhd4,target=/RLrgaK \
    --mount=type=bind,source=/tmp/ahmdbpvt,target=/tmp \
    --mount=type=bind,source=/home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr.c,target=/var/lib/cwl/stg999c8e16-4552-40b8-a3ef-f8cfcf979da6/sr.c,readonly \
    --workdir=/RLrgaK \
    --read-only=true \
    --net=none \
    --user=1000:1000 \
    --rm \
    --cidfile=/tmp/mi_6gc8w/20260608200458-351234.cid \
    --env=TMPDIR=/tmp \
    --env=HOME=/RLrgaK \
    mfisherman/mpich:4.3.2 \
    mpicc \
    /var/lib/cwl/stg999c8e16-4552-40b8-a3ef-f8cfcf979da6/sr.c \
    -o \
    sr
INFO [job mpich-mpicc] completed success
{
    "executable": {
        "location": "file:///home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr",
        "basename": "sr",
        "class": "File",
        "checksum": "sha1$0ec2227060df40b5ce79cc59c60cdaacaf19bfb6",
        "size": 16512,
        "path": "/home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr"
    }
}INFO Final process status is success

With containers:

(venv) kinow@ranma:~/Development/python/workspace/cwl-mpi/workflows/mpich-sr$ cwltool --singularity mpicc.cwl 
INFO /home/kinow/Development/python/workspace/cwltool/venv/bin/cwltool 3.1.20260315121657
INFO Resolved 'mpicc.cwl' to 'file:///home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/mpicc.cwl'
INFO Using local copy of Singularity image mfisherman_mpich:4.3.2.sif found in /home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr
INFO [job mpich-mpicc] /tmp/vc9bqkk_$ singularity \
    --quiet \
    run \
    --contain \
    --ipc \
    --cleanenv \
    --no-eval \
    --userns \
    --home \
    /tmp/vc9bqkk_:/uwTdCe \
    --mount=type=bind,source=/tmp/hz7i2lql,target=/tmp \
    --mount=type=bind,source=/home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr.c,target=/var/lib/cwl/stg5acb05de-927d-413a-a9b5-fb57cd26ef20/sr.c,readonly \
    --pwd \
    /uwTdCe \
    --net \
    --network \
    none \
    /home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/mfisherman_mpich:4.3.2.sif \
    mpicc \
    /var/lib/cwl/stg5acb05de-927d-413a-a9b5-fb57cd26ef20/sr.c \
    -o \
    sr
INFO [job mpich-mpicc] completed success
{
    "executable": {
        "location": "file:///home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr",
        "basename": "sr",
        "class": "File",
        "checksum": "sha1$0ec2227060df40b5ce79cc59c60cdaacaf19bfb6",
        "size": 16512,
        "path": "/home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr"
    }
}INFO Final process status is success

StreamFlow 0.2.0rc2

It works when I run without Singularity with StreamFlow 0.2.0rc2:

version: v1.0
workflows:
  workflow_name:
    type: cwl
    config:
      file: "/home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/mpicc.cwl"

$ streamflow run test_nocontainer.yml 
2026-06-08 20:07:22.980 INFO     Processing workflow 161df4b4-3bfd-410b-8cae-e3896ba71d74
2026-06-08 20:07:22.980 INFO     Building workflow execution plan
2026-06-08 20:07:23.638 INFO     COMPLETED building of workflow execution plan
2026-06-08 20:07:23.638 INFO     EXECUTING workflow 161df4b4-3bfd-410b-8cae-e3896ba71d74
2026-06-08 20:07:23.666 INFO     DEPLOYING e4a67d4f-7f76-42e7-aa57-adb568114c25
2026-06-08 20:07:23.894 INFO     COMPLETED deployment of e4a67d4f-7f76-42e7-aa57-adb568114c25
2026-06-08 20:07:23.903 INFO     COPYING from /home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr.c on local file-system to /tmp/streamflow/d87b3949-d48a-4ae9-9b6f-ccdedd832329/ccd52065-1c27-4541-8d10-38a497a2094a/sr.c on location e4a67d4f-7f76-42e7-aa57-adb568114c25/3e21bd808c8a5f3fd1f5a2a8e96c97fbd897a666e51d1d89d922bb8be3fd170c
2026-06-08 20:07:23.955 INFO     COMPLETED copy from /home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr.c on local file-system to /tmp/streamflow/d87b3949-d48a-4ae9-9b6f-ccdedd832329/ccd52065-1c27-4541-8d10-38a497a2094a/sr.c on location e4a67d4f-7f76-42e7-aa57-adb568114c25/3e21bd808c8a5f3fd1f5a2a8e96c97fbd897a666e51d1d89d922bb8be3fd170c
2026-06-08 20:07:23.959 INFO     EXECUTING step / (job /0) on location e4a67d4f-7f76-42e7-aa57-adb568114c25/3e21bd808c8a5f3fd1f5a2a8e96c97fbd897a666e51d1d89d922bb8be3fd170c into directory /tmp/streamflow/6676b295-8bb7-4bfd-8e81-fb5540da8795:
mpicc \
	/tmp/streamflow/d87b3949-d48a-4ae9-9b6f-ccdedd832329/ccd52065-1c27-4541-8d10-38a497a2094a/sr.c \
	-o \
	sr
2026-06-08 20:07:24.085 INFO     COMPLETED Step /
2026-06-08 20:07:24.085 INFO     COPYING from /tmp/streamflow/6676b295-8bb7-4bfd-8e81-fb5540da8795/sr on location e4a67d4f-7f76-42e7-aa57-adb568114c25/3e21bd808c8a5f3fd1f5a2a8e96c97fbd897a666e51d1d89d922bb8be3fd170c to /home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr on local file-system
2026-06-08 20:07:24.143 INFO     COMPLETED copy from /tmp/streamflow/6676b295-8bb7-4bfd-8e81-fb5540da8795/sr on location e4a67d4f-7f76-42e7-aa57-adb568114c25/3e21bd808c8a5f3fd1f5a2a8e96c97fbd897a666e51d1d89d922bb8be3fd170c to /home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr on local file-system
2026-06-08 20:07:24.146 INFO     COMPLETED workflow execution
{
    "executable": {
        "basename": "sr",
        "checksum": "sha1$0ec2227060df40b5ce79cc59c60cdaacaf19bfb6",
        "class": "File",
        "dirname": "/home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr",
        "location": "file:///home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr",
        "nameext": "",
        "nameroot": "sr",
        "path": "/home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr",
        "size": 16512
    }
}
2026-06-08 20:07:24.146 INFO     UNDEPLOYING e4a67d4f-7f76-42e7-aa57-adb568114c25
2026-06-08 20:07:24.195 INFO     COMPLETED undeployment of e4a67d4f-7f76-42e7-aa57-adb568114c25

However, using the example at the bottom of the CWL Docker Requirement from StreamFlow 0.2 docs, https://streamflow.di.unito.it/documentation/0.2/cwl/docker-requirement.html:

version: v1.0
workflows:
  workflow_name:
    type: cwl
    config:
      file: "/home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/mpicc.cwl"
      docker:
        - step: /
          deployment:
            type: singularity
            config: {}

It hangs running with the container, and complains about the lack of locations? I searched a bit the code and docs, but I couldn't understand how to solve it.

(venv) kinow@ranma:~/Development/python/workspace/cwl-mpi/workflows/mpich-sr$ streamflow run --debug test.yml 
2026-06-08 20:09:07.831 INFO     Processing workflow 9e3a72c4-fdae-4e21-b7f8-44fe7ab2b4f1
2026-06-08 20:09:07.831 INFO     Building workflow execution plan
2026-06-08 20:09:08.239 DEBUG    Translating CommandLineTool /
2026-06-08 20:09:08.492 INFO     COMPLETED building of workflow execution plan
2026-06-08 20:09:08.493 INFO     EXECUTING workflow 9e3a72c4-fdae-4e21-b7f8-44fe7ab2b4f1
2026-06-08 20:09:08.517 DEBUG    Step /source-cmd-default-transformer-injector received inputs ['0']
2026-06-08 20:09:08.517 DEBUG    Step /source-cmd-default-transformer received inputs ['0']
2026-06-08 20:09:08.518 DEBUG    Step /output_name-cmd-default-transformer-injector received inputs ['0']
2026-06-08 20:09:08.518 DEBUG    Step /output_name-cmd-default-transformer received inputs ['0']
2026-06-08 20:09:08.518 DEBUG    Retrieving available locations for job /source-cmd-default-transformer-injector/0 on __LOCAL__.
2026-06-08 20:09:08.518 DEBUG    Available locations for job /source-cmd-default-transformer-injector/0 on __LOCAL__ are ['__LOCAL__'].
2026-06-08 20:09:08.518 DEBUG    Job /source-cmd-default-transformer-injector/0 allocated locally
2026-06-08 20:09:08.518 DEBUG    Retrieving available locations for job /output_name-cmd-default-transformer-injector/0 on __LOCAL__.
2026-06-08 20:09:08.519 DEBUG    Available locations for job /output_name-cmd-default-transformer-injector/0 on __LOCAL__ are ['__LOCAL__'].
2026-06-08 20:09:08.519 DEBUG    Job /output_name-cmd-default-transformer-injector/0 allocated locally
2026-06-08 20:09:08.519 DEBUG    COMPLETED Step __deploy__/644c4bd9-33c8-4ccd-9d82-a8726d781d98
2026-06-08 20:09:08.519 DEBUG    COMPLETED Step __deploy__/__LOCAL__
2026-06-08 20:09:08.519 DEBUG    Job /source-cmd-default-transformer-injector/0 changed status to RUNNING
2026-06-08 20:09:08.520 DEBUG    Job /output_name-cmd-default-transformer-injector/0 changed status to RUNNING
2026-06-08 20:09:08.520 DEBUG    COMPLETED Step /source-cmd-default-transformer-injector/__schedule__
2026-06-08 20:09:08.521 DEBUG    COMPLETED Step /output_name-cmd-default-transformer-injector/__schedule__
2026-06-08 20:09:08.521 DEBUG    Job /output_name-cmd-default-transformer-injector/0 changed status to COMPLETED
2026-06-08 20:09:08.521 DEBUG    Step /output_name-cmd-default-transformer received inputs ['0']
2026-06-08 20:09:08.521 DEBUG    Step /output_name-cmd-default-transformer-injector received termination token with Status COMPLETED
2026-06-08 20:09:08.521 DEBUG    Job /source-cmd-default-transformer-injector/0 changed status to COMPLETED
2026-06-08 20:09:08.521 DEBUG    Step /source-cmd-default-transformer received inputs ['0']
2026-06-08 20:09:08.521 DEBUG    Step /source-cmd-default-transformer-injector received termination token with Status COMPLETED
2026-06-08 20:09:08.521 DEBUG    Step /output_name-cmd-default-transformer received termination token with Status COMPLETED
2026-06-08 20:09:08.521 DEBUG    COMPLETED Step /output_name-cmd-default-transformer-injector
2026-06-08 20:09:08.521 DEBUG    Step /source-token-transformer received inputs ['0', '0']
2026-06-08 20:09:08.521 DEBUG    Processing /home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr.c on location __LOCAL__/__LOCAL__.
2026-06-08 20:09:08.521 DEBUG    Step /output_name-token-transformer received inputs ['0', '0']
2026-06-08 20:09:08.521 DEBUG    Step /source-cmd-default-transformer received termination token with Status COMPLETED
2026-06-08 20:09:08.522 DEBUG    COMPLETED Step /output_name-cmd-default-transformer
2026-06-08 20:09:08.522 DEBUG    COMPLETED Step /source-cmd-default-transformer-injector
2026-06-08 20:09:08.522 DEBUG    COMPLETED Step /source-cmd-default-transformer
2026-06-08 20:09:08.522 DEBUG    Step /__transfer__/output_name received inputs ['0']
2026-06-08 20:09:08.522 DEBUG    Step /output_name-token-transformer received termination token with Status COMPLETED
2026-06-08 20:09:08.522 DEBUG    Step /__transfer__/source received inputs ['0']
2026-06-08 20:09:08.522 DEBUG    Step /__schedule__ received inputs ['0', '0']
2026-06-08 20:09:08.522 DEBUG    Step /source-token-transformer received termination token with Status COMPLETED
2026-06-08 20:09:08.522 DEBUG    Retrieving available locations for job /0 on 644c4bd9-33c8-4ccd-9d82-a8726d781d98/mfisherman/mpich:4.3.2.
2026-06-08 20:09:08.522 INFO     DEPLOYING 644c4bd9-33c8-4ccd-9d82-a8726d781d98
2026-06-08 20:09:08.522 DEBUG    EXECUTING command singularity --version 2>&1 on __LOCAL__/__LOCAL__ 
2026-06-08 20:09:08.523 DEBUG    COMPLETED Step /output_name-token-transformer
2026-06-08 20:09:08.523 DEBUG    COMPLETED Step /source-token-transformer
2026-06-08 20:09:08.535 DEBUG    Using apptainer version 1.4.5.
2026-06-08 20:09:08.535 DEBUG    EXECUTING command singularity instance start    --bind  "/tmp/streamflow:/tmp/streamflow"     --cleanenv    --contain                       --net   --network  "none"                         docker://mfisherman/mpich:4.3.2 6c8d7a7e-64de-40b2-b970-374475f59147  2>&1 on __LOCAL__/__LOCAL__ 
2026-06-08 20:09:10.089 DEBUG    EXECUTING command singularity instance list --json 2>&1 on __LOCAL__/__LOCAL__ 
2026-06-08 20:09:10.107 DEBUG    EXECUTING command nproc 2>&1 on SubprocessShell with command `singularity exec instance://6c8d7a7e-64de-40b2-b970-374475f59147 sh`
2026-06-08 20:09:10.163 DEBUG    EXECUTING command cat /proc/meminfo | grep MemTotal | awk '{print $2}' 2>&1 on SubprocessShell with command `singularity exec instance://6c8d7a7e-64de-40b2-b970-374475f59147 sh`
2026-06-08 20:09:10.171 DEBUG    EXECUTING command cat /proc/1/mountinfo 2>&1 on SubprocessShell with command `singularity exec instance://6c8d7a7e-64de-40b2-b970-374475f59147 sh`
2026-06-08 20:09:10.172 DEBUG    Container binds: {'/tmp/streamflow': '/tmp/streamflow', '/etc/localtime': '/usr/share/zoneinfo/Europe/Madrid', '/sys/firmware/efi/efivars': '/sys/firmware/efi/efivars/'}
2026-06-08 20:09:10.172 DEBUG    EXECUTING command df -aT | tail -n +2 | awk 'NF == 1 {device = $1; getline; $0 = device $0} {print $7, $2, $5}' 2>&1 on SubprocessShell with command `singularity exec instance://6c8d7a7e-64de-40b2-b970-374475f59147 sh`
2026-06-08 20:09:10.182 INFO     COMPLETED deployment of 644c4bd9-33c8-4ccd-9d82-a8726d781d98
2026-06-08 20:09:10.182 DEBUG    EXECUTING command test -e /tmp/streamflow && readlink -f /tmp/streamflow 2>&1 on SubprocessShell with command `singularity exec instance://6c8d7a7e-64de-40b2-b970-374475f59147 sh`
2026-06-08 20:09:10.184 DEBUG    Not enough available locations: job /0 requires 1 locations on deployment 644c4bd9-33c8-4ccd-9d82-a8726d781d98/mfisherman/mpich:4.3.2, but only 0 are available.

Any clue what I'm doing wrong?

Thanks!

[kinow]

I had a look at the file used in CWL Conformance Tests, https://github.com/alpha-unito/streamflow/blob/master/tests/cwl-conformance/streamflow-singularity.yml, and the only difference I could see was the database option. I added that and tried again, but no luck.

[LanderOtto]

I tried on some machines, but I was not able to reproduce the error. I created a new StreamFlow branch (logging-satisfy) which logs more information. Can you re-execute the workflow with the debug flag using this branch?

[kinow]

Hi @LanderOtto ! 👋

Sure, here's the output on my local laptop:

Executing: streamflow run --debug /tmp/tmp.FYCrRUS7qB
2026-06-09 20:26:27.211 INFO     Processing workflow 2125b9d9-658d-4c52-a723-c2049593b110
2026-06-09 20:26:27.211 INFO     Building workflow execution plan
2026-06-09 20:26:27.929 DEBUG    Translating CommandLineTool /
2026-06-09 20:26:28.215 INFO     COMPLETED building of workflow execution plan
2026-06-09 20:26:28.215 INFO     EXECUTING workflow 2125b9d9-658d-4c52-a723-c2049593b110
2026-06-09 20:26:28.231 DEBUG    Step /source-cmd-default-transformer-injector received inputs ['0']
2026-06-09 20:26:28.231 DEBUG    Step /output_name-cmd-default-transformer-injector received inputs ['0']
2026-06-09 20:26:28.231 DEBUG    Step /__empty_unbound_inputs__-injector received inputs ['0']
2026-06-09 20:26:28.232 DEBUG    Retrieving available locations for job /source-cmd-default-transformer-injector/0 on __LOCAL__.
2026-06-09 20:26:28.232 DEBUG    Available locations for job /source-cmd-default-transformer-injector/0 on __LOCAL__ are ['__LOCAL__'].
2026-06-09 20:26:28.232 DEBUG    Job /source-cmd-default-transformer-injector/0 allocated locally
2026-06-09 20:26:28.232 DEBUG    Retrieving available locations for job /output_name-cmd-default-transformer-injector/0 on __LOCAL__.
2026-06-09 20:26:28.232 DEBUG    Available locations for job /output_name-cmd-default-transformer-injector/0 on __LOCAL__ are ['__LOCAL__'].
2026-06-09 20:26:28.232 DEBUG    Job /output_name-cmd-default-transformer-injector/0 allocated locally
2026-06-09 20:26:28.233 DEBUG    Retrieving available locations for job /__empty_unbound_inputs__-injector/0 on __LOCAL__.
2026-06-09 20:26:28.233 DEBUG    Available locations for job /__empty_unbound_inputs__-injector/0 on __LOCAL__ are ['__LOCAL__'].
2026-06-09 20:26:28.233 DEBUG    Job /__empty_unbound_inputs__-injector/0 allocated locally
2026-06-09 20:26:28.234 DEBUG    COMPLETED Step __deploy__/f0497b3b-9fe3-422a-8bb9-98122f36b3f5
2026-06-09 20:26:28.234 DEBUG    COMPLETED Step __deploy__/__LOCAL__
2026-06-09 20:26:28.234 DEBUG    Job /source-cmd-default-transformer-injector/0 changed status to RUNNING
2026-06-09 20:26:28.235 DEBUG    Job /output_name-cmd-default-transformer-injector/0 changed status to RUNNING
2026-06-09 20:26:28.235 DEBUG    COMPLETED Step /source-cmd-default-transformer-injector/__schedule__
2026-06-09 20:26:28.236 DEBUG    Job /__empty_unbound_inputs__-injector/0 changed status to RUNNING
2026-06-09 20:26:28.236 DEBUG    COMPLETED Step /output_name-cmd-default-transformer-injector/__schedule__
2026-06-09 20:26:28.236 DEBUG    COMPLETED Step /__empty_unbound_inputs__-injector/__schedule__
2026-06-09 20:26:28.236 DEBUG    Job /output_name-cmd-default-transformer-injector/0 changed status to COMPLETED
2026-06-09 20:26:28.236 DEBUG    Step /output_name-cmd-default-transformer-injector received termination token with Status COMPLETED
2026-06-09 20:26:28.236 DEBUG    Job /source-cmd-default-transformer-injector/0 changed status to COMPLETED
2026-06-09 20:26:28.237 DEBUG    Job /__empty_unbound_inputs__-injector/0 changed status to COMPLETED
2026-06-09 20:26:28.237 DEBUG    Step /__empty_unbound_inputs__-bcast received inputs ['0']
2026-06-09 20:26:28.237 DEBUG    Step /source-cmd-default-transformer-injector received termination token with Status COMPLETED
2026-06-09 20:26:28.237 DEBUG    Step /__empty_unbound_inputs__-injector received termination token with Status COMPLETED
2026-06-09 20:26:28.237 DEBUG    COMPLETED Step /output_name-cmd-default-transformer-injector
2026-06-09 20:26:28.237 DEBUG    COMPLETED Step /source-cmd-default-transformer-injector
2026-06-09 20:26:28.237 DEBUG    COMPLETED Step /__empty_unbound_inputs__-injector
2026-06-09 20:26:28.237 DEBUG    Step /source-cmd-default-transformer received inputs ['0']
2026-06-09 20:26:28.237 DEBUG    Step /source-cmd-default-transformer received inputs ['0']
2026-06-09 20:26:28.238 DEBUG    Step /output_name-cmd-default-transformer received inputs ['0']
2026-06-09 20:26:28.238 DEBUG    Step /__empty_unbound_inputs__-bcast received termination token with Status COMPLETED
2026-06-09 20:26:28.238 DEBUG    Step /output_name-cmd-default-transformer received inputs ['0']
2026-06-09 20:26:28.238 DEBUG    Step /source-cmd-default-transformer received termination token with Status COMPLETED
2026-06-09 20:26:28.238 DEBUG    COMPLETED Step /__empty_unbound_inputs__-bcast
2026-06-09 20:26:28.238 DEBUG    COMPLETED Step /source-cmd-default-transformer
2026-06-09 20:26:28.238 DEBUG    Step /source-token-transformer received inputs ['0', '0']
2026-06-09 20:26:28.238 DEBUG    Processing /home/kinow/Development/python/workspace/cwl-mpi/workflows/mpich-sr/sr.c on location __LOCAL__/__LOCAL__.
2026-06-09 20:26:28.238 DEBUG    Step /output_name-token-transformer received inputs ['0', '0']
2026-06-09 20:26:28.238 DEBUG    Step /output_name-cmd-default-transformer received termination token with Status COMPLETED
2026-06-09 20:26:28.239 DEBUG    COMPLETED Step /output_name-cmd-default-transformer
2026-06-09 20:26:28.239 DEBUG    Step /__transfer__/output_name received inputs ['0']
2026-06-09 20:26:28.239 DEBUG    Step /output_name-token-transformer received termination token with Status COMPLETED
2026-06-09 20:26:28.239 DEBUG    Step /__transfer__/source received inputs ['0']
2026-06-09 20:26:28.239 DEBUG    Step /__schedule__ received inputs ['0', '0']
2026-06-09 20:26:28.239 DEBUG    Step /source-token-transformer received termination token with Status COMPLETED
2026-06-09 20:26:28.239 DEBUG    Retrieving available locations for job /0 on f0497b3b-9fe3-422a-8bb9-98122f36b3f5/mfisherman/mpich:4.3.2.
2026-06-09 20:26:28.239 INFO     DEPLOYING f0497b3b-9fe3-422a-8bb9-98122f36b3f5
2026-06-09 20:26:28.239 DEBUG    EXECUTING command singularity --version 2>&1 on __LOCAL__/__LOCAL__ 
2026-06-09 20:26:28.240 DEBUG    COMPLETED Step /output_name-token-transformer
2026-06-09 20:26:28.240 DEBUG    COMPLETED Step /source-token-transformer
2026-06-09 20:26:28.252 DEBUG    Using apptainer version 1.4.5.
2026-06-09 20:26:28.252 DEBUG    EXECUTING command singularity instance start    --bind  "/tmp/streamflow:/tmp/streamflow"     --cleanenv    --contain                       --net   --network  "none"                         docker://mfisherman/mpich:4.3.2 2afd9ffe-c464-4c47-a6a2-a0ecf2e7bdb8  2>&1 on __LOCAL__/__LOCAL__ 
2026-06-09 20:26:29.602 DEBUG    EXECUTING command singularity instance list --json 2>&1 on __LOCAL__/__LOCAL__ 
2026-06-09 20:26:29.620 DEBUG    EXECUTING command nproc 2>&1 on SubprocessShell with command `singularity exec instance://2afd9ffe-c464-4c47-a6a2-a0ecf2e7bdb8 sh`
2026-06-09 20:26:29.674 DEBUG    EXECUTING command cat /proc/meminfo | grep MemTotal | awk '{print $2}' 2>&1 on SubprocessShell with command `singularity exec instance://2afd9ffe-c464-4c47-a6a2-a0ecf2e7bdb8 sh`
2026-06-09 20:26:29.681 DEBUG    Host mount points: ['/', '/sys/firmware/efi/efivars', '/proc/sys/fs/binfmt_misc', '/boot', '/snap/bare/5', '/snap/blender/7480', '/snap/core18/2999', '/snap/core20/2866', '/snap/core22/2411', '/snap/core24/1643', '/snap/desktop-security-center/151', '/snap/firefox/8387', '/snap/firefox/8417', '/snap/firmware-updater/226', '/snap/gnome-3-34-1804/93', '/snap/gnome-3-28-1804/198', '/snap/gnome-3-38-2004/143', '/snap/gnome-42-2204/247', '/snap/gnome-46-2404/153', '/snap/gtk-common-themes/1535', '/snap/mesa-2404/1165', '/snap/prompting-client/222', '/snap/slack/244', '/snap/snap-store/1216', '/snap/snapd/26865', '/snap/snapd-desktop-integration/361', '/snap/thunderbird/1117', '/snap/thunderbird/1138', '/snap/vlc/3777', '/boot/efi', '/proc/sys/fs/binfmt_misc', '/run/snapd/ns/snapd-desktop-integration.mnt', '/run/docker/netns/ingress_sbox', '/run/docker/netns/1-pdvcq6wckm', '/run/user/1000/doc', '/run/snapd/ns/prompting-client.mnt', '/run/user/1000/gvfs', '/run/snapd/ns/firefox.mnt', '/run/snapd/ns/firmware-updater.mnt', '/run/snapd/ns/slack.mnt', '/run/snapd/ns/thunderbird.mnt']
2026-06-09 20:26:29.681 DEBUG    EXECUTING command singularity buildcfg | grep CONF_FILE | cut -d'=' -f2 | xargs cat | grep "^bind path" 2>&1 on f0497b3b-9fe3-422a-8bb9-98122f36b3f5/2afd9ffe-c464-4c47-a6a2-a0ecf2e7bdb8 
2026-06-09 20:26:29.698 DEBUG    EXECUTING command cat /proc/1/mountinfo 2>&1 on SubprocessShell with command `singularity exec instance://2afd9ffe-c464-4c47-a6a2-a0ecf2e7bdb8 sh`
2026-06-09 20:26:29.700 DEBUG    Container binds: {'/etc/localtime': '/usr/share/zoneinfo/Europe/Madrid', '/sys/firmware/efi/efivars': '/sys/firmware/efi/efivars/'}
2026-06-09 20:26:29.700 DEBUG    EXECUTING command df -aT | tail -n +2 | awk 'NF == 1 {device = $1; getline; $0 = device $0} {print $7, $2, $5}' 2>&1 on SubprocessShell with command `singularity exec instance://2afd9ffe-c464-4c47-a6a2-a0ecf2e7bdb8 sh`
2026-06-09 20:26:29.710 INFO     COMPLETED deployment of f0497b3b-9fe3-422a-8bb9-98122f36b3f5
2026-06-09 20:26:29.710 DEBUG    EXECUTING command test -e /tmp/streamflow && readlink -f /tmp/streamflow 2>&1 on SubprocessShell with command `singularity exec instance://2afd9ffe-c464-4c47-a6a2-a0ecf2e7bdb8 sh`
2026-06-09 20:26:29.712 INFO     Hardware unsatisfied: insufficient storage at '/' (has 63.98046875, requires 2048).
2026-06-09 20:26:29.712 DEBUG    Not enough available locations: job /0 requires 1 locations on deployment f0497b3b-9fe3-422a-8bb9-98122f36b3f5/mfisherman/mpich:4.3.2, but only 0 are available.

I think this might be the problem??

2026-06-09 20:26:29.712 INFO Hardware unsatisfied: insufficient storage at '/' (has 63.98046875, requires 2048).

Although not sure where it's coming from

[kinow]

Well, changing Hardware.satisfies to this

    def satisfies(self, other: Any) -> bool:
        """Check if this hardware has enough resources to satisfy the requirement."""
        return True

made my workflow pass. So maybe there's something with how Singularity containers are loaded in StreamFlow, and its volumes/storages values are loaded and parsed/used.

[LanderOtto]

I believe the problem stems from an optimization in StreamFlow that skips certain filesystem types. It might be too restrictive in what it filters out. Could you run the following command to confirm if this is the issue?
singularity exec --bind "/tmp/streamflow:/tmp/streamflow" --cleanenv --contain --net --network "none" docker://mfisherman/mpich:4.3.2 df -aT

[kinow]

I believe the problem stems from an optimization in StreamFlow that skips certain filesystem types. It might be too restrictive in what it filters out. Could you run the following command to confirm if this is the issue? singularity exec --bind "/tmp/streamflow:/tmp/streamflow" --cleanenv --contain --net --network "none" docker://mfisherman/mpich:4.3.2 df -aT

Sure thing, @LanderOtto . Here it goes:

kinow@ranma:~$ singularity exec   --bind  "/tmp/streamflow:/tmp/streamflow"  --cleanenv  --contain  --net   --network  "none" docker://mfisherman/mpich:4.3.2 df -aT
INFO:    Using cached SIF image
INFO:    gocryptfs not found, will not be able to use gocryptfs
Filesystem                        Type        1K-blocks      Used Available Use% Mounted on
overlay                           overlay         65536        20     65516   1% /
tmpfs                             tmpfs           65536        20     65516   1% /dev
tmpfs                             tmpfs        15809040         0  15809040   0% /dev/shm
devpts                            devpts              0         0         0    - /dev/pts
devpts                            devpts              0         0         0    - /dev/console
udev                              devtmpfs     15705224         0  15705224   0% /dev/tty
udev                              devtmpfs     15705224         0  15705224   0% /dev/null
udev                              devtmpfs     15705224         0  15705224   0% /dev/zero
udev                              devtmpfs     15705224         0  15705224   0% /dev/random
udev                              devtmpfs     15705224         0  15705224   0% /dev/urandom
tmpfs                             tmpfs           65536        20     65516   1% /etc/hosts
/dev/mapper/ubuntu--vg-ubuntu--lv ext4        957134040 548464184 359976404  61% /etc/localtime
proc                              proc                0         0         0    - /proc
systemd-1                         -                   -         -         -    - /proc/sys/fs/binfmt_misc
binfmt_misc                       binfmt_misc         0         0         0    - /proc/sys/fs/binfmt_misc
sysfs                             sysfs               0         0         0    - /sys
securityfs                        securityfs          0         0         0    - /sys/kernel/security
cgroup2                           cgroup2             0         0         0    - /sys/fs/cgroup
none                              pstore              0         0         0    - /sys/fs/pstore
efivarfs                          efivarfs          148        56        88  39% /sys/firmware/efi/efivars
bpf                               bpf                 0         0         0    - /sys/fs/bpf
debugfs                           debugfs             0         0         0    - /sys/kernel/debug
tracefs                           tracefs             0         0         0    - /sys/kernel/tracing
configfs                          configfs            0         0         0    - /sys/kernel/config
fusectl                           fusectl             0         0         0    - /sys/fs/fuse/connections
tmpfs                             tmpfs           65536        20     65516   1% /home/kinow
tmpfs                             tmpfs           65536        20     65516   1% /tmp
tmpfs                             tmpfs           65536        20     65516   1% /var/tmp
tmpfs                             tmpfs           65536        20     65516   1% /etc/resolv.conf
tmpfs                             tmpfs           65536        20     65516   1% /etc/passwd
tmpfs                             tmpfs           65536        20     65516   1% /etc/group
tmpfs                             tmpfs        15809040     29684  15779356   1% /tmp/streamflow

Thanks!

[kinow]

Oh, I'm launching these experiments from my home directory, on tmpfs in this session on Singularity. Maybe that's the problem? Because we are not mapping $HOME and excluding everything so that is probably automatically mounted into tmpfs, and StreamFlow ignores that type of FS?