Open
Description
With the 0.14 release line, I see sporadic race conditons on conda-forge with macOS (w/ OpenMPI 4.1.2) of the form:
-------------------------------------------------------------------------------
$SRC_DIR/test/ParallelIOTest.cpp:431
...............................................................................
$SRC_DIR/test/ParallelIOTest.cpp:431: FAILED:
due to unexpected exception with message:
ERROR: File ../samples/parallelExtendDataset.bp could not be found within the
0.000000s timeout: errno=2: ERROR: couldn't open file ../samples/
parallelExtendDataset.bp/md.idx, in call to POSIX open: errno = 2: No such
file or directory: unspecified iostream_category error: unspecified
iostream_category error
...
===============================================================================
test cases: 21 | 20 passed | 1 failed
assertions: 2031 | 2030 passed | 1 failed
It might be possible that the network filesystem in CI is not fully synced/flushed yet, although we do an MPI barrier between write and read...
https://github.com/openPMD/openPMD-api/blob/0.14.4/test/ParallelIOTest.cpp#L413-L416
Maybe we need to do a sync or simple sleep here. It's a bit surprising that an MPI barrier is not sufficient here, because all ranks write from the same node.
The following NEW packages will be INSTALLED:
adios: 1.13.1-mpi_openmpi_h7b7cf31_1021 conda-forge
adios2: 2.7.1-mpi_openmpi_py38hd301022_3 conda-forge
blosc: 1.21.0-he49afe7_0 conda-forge
bzip2: 1.0.8-h0d85af4_4 conda-forge
c-ares: 1.18.1-h0d85af4_0 conda-forge
ca-certificates: 2021.10.8-h033912b_0 conda-forge
catch2: 2.13.8-h940c156_0 conda-forge
hdf5: 1.12.1-mpi_openmpi_hbcbd0bb_3 conda-forge
krb5: 1.19.2-h289aae4_3 conda-forge
libblas: 3.9.0-13_osx64_openblas conda-forge
libcblas: 3.9.0-13_osx64_openblas conda-forge
libcurl: 7.81.0-h97da3c1_0 conda-forge
libcxx: 12.0.1-habf9029_1 conda-forge
libedit: 3.1.20191231-h0678c8f_2 conda-forge
libev: 4.33-haf1e3a3_1 conda-forge
libffi: 3.4.2-h0d85af4_5 conda-forge
libgfortran: 5.0.0-9_3_0_h6c81a4c_23 conda-forge
libgfortran5: 9.3.0-h6c81a4c_23 conda-forge
liblapack: 3.9.0-13_osx64_openblas conda-forge
libnghttp2: 1.43.0-hfd382f3_1 conda-forge
libopenblas: 0.3.18-openmp_h3351f45_0 conda-forge
libpng: 1.6.37-h7cec526_2 conda-forge
libsodium: 1.0.18-hbcb3906_1 conda-forge
libssh2: 1.10.0-hd3787cc_2 conda-forge
libzlib: 1.2.11-h9173be1_1013 conda-forge
llvm-openmp: 12.0.1-hda6cdc1_1 conda-forge
mpi: 1.0-openmpi conda-forge
mpi4py: 3.1.3-py38h1087cf1_0 conda-forge
ncurses: 6.3-he49afe7_0 conda-forge
numpy: 1.22.0-py38h9d72dae_1 conda-forge
openmpi: 4.1.2-hd3cd54c_0 conda-forge
openssl: 3.0.0-h0d85af4_2 conda-forge
pybind11: 2.9.0-py38h12bbefe_0 conda-forge
pybind11-global: 2.9.0-py38h12bbefe_0 conda-forge
python: 3.8.12-h43ca1e7_2_cpython conda-forge
python_abi: 3.8-2_cp38 conda-forge
readline: 8.1-h05e3726_0 conda-forge
sqlite: 3.37.0-h23a322b_0 conda-forge
tk: 8.6.11-h5dbffcc_1 conda-forge
xz: 5.2.5-haf1e3a3_1 conda-forge
zeromq: 4.3.4-he49afe7_1 conda-forge
zfp: 0.5.5-h4a89273_8 conda-forge
zlib: 1.2.11-h9173be1_1013 conda-forge
Package Plan
environment location: /Users/runner/miniforge3/conda-bld/openpmd-api_1642979656479/_build_env
The following NEW packages will be INSTALLED:
bzip2: 1.0.8-h0d85af4_4 conda-forge
c-ares: 1.18.1-h0d85af4_0 conda-forge
ca-certificates: 2021.10.8-h033912b_0 conda-forge
cctools_osx-64: 973.0.1-h609b070_5 conda-forge
clang: 11.1.0-h694c41f_1 conda-forge
clang-11: 11.1.0-default_he082bbe_1 conda-forge
clang_osx-64: 11.1.0-hc8532ff_6 conda-forge
clangxx: 11.1.0-default_he082bbe_1 conda-forge
clangxx_osx-64: 11.1.0-h2c55bc3_6 conda-forge
cmake: 3.21.3-hdbd2f3e_0 conda-forge
compiler-rt: 11.1.0-h654b07c_0 conda-forge
compiler-rt_osx-64: 11.1.0-h8c5fa43_0 conda-forge
curl: 7.81.0-h97da3c1_0 conda-forge
expat: 2.4.3-he49afe7_0 conda-forge
krb5: 1.19.2-h289aae4_3 conda-forge
ld64_osx-64: 609-h6adca69_5 conda-forge
libclang-cpp11.1: 11.1.0-default_he082bbe_1 conda-forge
libcurl: 7.81.0-h97da3c1_0 conda-forge
libcxx: 12.0.1-habf9029_1 conda-forge
libedit: 3.1.20191231-h0678c8f_2 conda-forge
libev: 4.33-haf1e3a3_1 conda-forge
libiconv: 1.16-haf1e3a3_0 conda-forge
libllvm11: 11.1.0-hd011deb_2 conda-forge
libllvm12: 12.0.1-hd011deb_2 conda-forge
libnghttp2: 1.43.0-hfd382f3_1 conda-forge
libssh2: 1.10.0-hd3787cc_2 conda-forge
libuv: 1.43.0-h0d85af4_0 conda-forge
libzlib: 1.2.11-h9173be1_1013 conda-forge
llvm-tools: 11.1.0-hd011deb_2 conda-forge
lz4-c: 1.9.3-he49afe7_1 conda-forge
make: 4.3-h22f3db7_1 conda-forge
ncurses: 6.3-he49afe7_0 conda-forge
openssl: 3.0.0-h0d85af4_2 conda-forge
pkg-config: 0.29.2-h31203cd_1008 conda-forge
rhash: 1.4.1-h35c211d_0 conda-forge
sigtool: 0.1.2-h45d4a6f_1 conda-forge
tapi: 1100.0.11-h9ce4665_0 conda-forge
tk: 8.6.11-h5dbffcc_1 conda-forge
xz: 5.2.5-haf1e3a3_1 conda-forge
zlib: 1.2.11-h9173be1_1013 conda-forge
zstd: 1.5.2-h582d3a0_0 conda-forge