Skip to content

Commit 5fabd03

Browse files
committed
CWL Conformance tests docs for Fugaku updated
1 parent ab4e9da commit 5fabd03

1 file changed

Lines changed: 9 additions & 36 deletions

File tree

  • cwl-conformance-tests/output/cwltool/fugaku

‎cwl-conformance-tests/output/cwltool/fugaku/README.md‎

Lines changed: 9 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -5,39 +5,12 @@ the tests performed for the CWL + MPI master's thesis.
55

66
`local` is the same as in other HPCs, a login node in Fugaku HPC.
77

8-
`pjm` refers to the execution where, using the same environment used for
9-
the login node, tests are submitted via a batch scheduler. Different from
10-
the other HPCs (CESGA FT3, CSC LUMI, BSC MN5), Fugaku uses PJM.
11-
12-
`pjm_install` is the execution also using PJM, but instead of using the same
13-
environment from the login node, the Python virtual environment is created
14-
within the compute node.
15-
16-
`pjm` and `pjm_install` are used because Fugaku (at the time of writing, 2026)
17-
has login nodes with `x86_64` architecture, while PJM compute nodes have
18-
`aarch64` architecture CPUs.
19-
20-
In the `pjm` folder, the execution fails because `cwltool` has a dependency
21-
on `psutil`, which ships with pre-compiled Linux static object. The `psutil`
22-
package has an `.so` created for `x86_64`, which fails to load on PJM.
23-
24-
```bash
25-
ImportError: /vol0006/mdt1/home/[REDACTED]/venv/lib64/python3.11/site-packages/
26-
psutil/_psutil_linux.abi3.so: cannot open shared object file: No such file
27-
or directory
28-
```
29-
30-
`pjm_install` tries to solve this by doing everything in the compute
31-
node. There are different errors, but one that's recurrent in the logs is
32-
the incorrect architecture of the container image. Again, `x86_64` images
33-
are being used in the PJM compute nodes, causing Singularity to fail running
34-
the CWL conformance tests.
35-
36-
```bash
37-
<system-err>FATAL: While checking image: could not open image
38-
/vol0006/mdt1/home/[REDACTED]/common-workflow-language/v1.0/python:2-slim.sif:
39-
the image's architecture (amd64) could not run on the host's (arm64)
40-
...
41-
```
42-
43-
The Fugaku tests were executed directly on a login node, using `cwltest`.
8+
`pjm` refers to the execution where the CWL conformance tests were executed
9+
within an interactive PJM allocation. **NOTE**: Previously, there were errors
10+
launching the tests in this configuration because the `local` tests produced
11+
`x86_64` containers and `pip` dependencies that were cached and then re-used
12+
during the execution on `aarch64` PJM. The micromamba environment was created
13+
completely within the interactive PJM session.
14+
15+
The conformance tests were executed directly on a Fugaku login node, using
16+
`cwltest`. All tests passed successfully 🎉🥳!

0 commit comments

Comments
 (0)