Hang when running replicates

[jaclark5]

I have a set of tasks that completed successfully "AlchemicalNetwork-55961080ba1805b112aff83fca84b15f-openff-openff_2_3_0_release-freesolv", and I wanted to run two more replicates of each.

I added 2 repeat tasks per freesolv transformation with:

an_sk = open("alchemicalnetwork_scopekey.txt").read()
tf_sks = asc.get_network_transformations(an_sk)
tasks = asc.create_transformations_tasks(tf_sks * 2)

However, it ran for 4 hours without output or update. In a different notebook I checked the status of the same scope and found that the replicates were created and "waiting", no additional tasks were made.

I'm using an M3 Mac with MacOS 15.1:

Python env is slightly customized to allow use of alchemiscale with alchemical archives before stack update.

name: alchemiscale-client
channels:

• conda-forge
  dependencies:
• _openmp_mutex=4.5=7_kmp_llvm
• _python_abi3_support=1.0=hd8ed1ab_2
• adjusttext=1.3.0=pyhd8ed1ab_0
• adwaita-icon-theme=49.0=unix_0
• alchemiscale-client=0.7.2=pyhd8ed1ab_0
• ambertools=24.8=cuda_None_nompi_py312h59db364_101
• annotated-types=0.7.0=pyhd8ed1ab_1
• anyio=4.13.0=pyhcf101f3_0
• appnope=0.1.4=pyhd8ed1ab_1
• argon2-cffi=25.1.0=pyhd8ed1ab_0
• argon2-cffi-bindings=25.1.0=py312h4409184_2
• arpack=3.9.1=nompi_h1f29f7c_102
• arrow=1.4.0=pyhcf101f3_0
• asttokens=3.0.1=pyhd8ed1ab_0
• async-lru=2.3.0=pyhcf101f3_0
• atk-1.0=2.38.0=hd03087b_2
• attrs=26.1.0=pyhcf101f3_0
• babel=2.18.0=pyhcf101f3_1
• backports.zstd=1.3.0=py312h44dc372_0
• beautifulsoup4=4.14.3=pyha770c72_0
• biopython=1.87=py312h2bbb03f_0
• bleach=6.3.0=pyhcf101f3_1
• bleach-with-css=6.3.0=hbca2aae_1
• blosc=1.21.6=h7dd00d9_1
• brotli=1.2.0=h7d5ae5b_1
• brotli-bin=1.2.0=hc919400_1
• brotli-python=1.2.0=py312h0dfefe5_1
• bson=0.5.10=pyhd8ed1ab_0
• bzip2=1.0.8=hd037594_9
• c-ares=1.34.6=hc919400_0
• c-blosc2=2.23.1=hf9886e1_0
• ca-certificates=2026.2.25=hbd8a1cb_0
• cached-property=1.5.2=hd8ed1ab_1
• cached_property=1.5.2=pyha770c72_1
• cachetools=7.0.5=pyhd8ed1ab_0
• cairo=1.18.4=h6a3b0d2_0
• certifi=2026.2.25=pyhd8ed1ab_0
• cffi=2.0.0=py312h1b4d9a2_1
• cftime=1.6.5=py312hf57c059_1
• charset-normalizer=3.4.6=pyhd8ed1ab_0
• cinnabar=0.5.0=pyhd8ed1ab_2
• click=8.3.1=pyh8f84b5b_1
• click-option-group=0.5.6=pyhd8ed1ab_0
• colorama=0.4.6=pyhd8ed1ab_1
• comm=0.2.3=pyhe01879c_0
• contourpy=1.3.3=py312h3093aea_4
• cpython=3.12.13=py312hd8ed1ab_0
• cryptography=46.0.5=py312h3fef973_0
• cuda-version=11.8=h70ddcb2_3
• cycler=0.12.1=pyhcf101f3_2
• debugpy=1.8.20=py312h6510ced_0
• decorator=5.2.1=pyhd8ed1ab_0
• defusedxml=0.7.1=pyhd8ed1ab_0
• dill=0.4.1=pyhcf101f3_0
• diskcache=5.6.3=pyhd8ed1ab_1
• epoxy=1.5.10=hc919400_2
• exceptiongroup=1.3.1=pyhd8ed1ab_0
• executing=2.2.1=pyhd8ed1ab_0
• feflow=0.2.0=pyhd8ed1ab_0
• fftw=3.3.10=nompi_haf1500d_112
• filelock=3.25.2=pyhd8ed1ab_0
• flexcache=0.3=pyhd8ed1ab_1
• flexparser=0.4=pyhd8ed1ab_1
• font-ttf-dejavu-sans-mono=2.37=hab24e00_0
• font-ttf-inconsolata=3.000=h77eed37_0
• font-ttf-source-code-pro=2.038=h77eed37_0
• font-ttf-ubuntu=0.83=h77eed37_3
• fontconfig=2.17.1=h2b252f5_0
• fonts-conda-ecosystem=1=0
• fonts-conda-forge=1=hc364b38_1
• fonttools=4.62.0=py312h04c11ed_0
• fqdn=1.5.1=pyhd8ed1ab_1
• freetype=2.14.3=hce30654_0
• freetype-py=2.3.0=pyhd8ed1ab_0
• fribidi=1.0.16=hc919400_0
• fsspec=2026.3.0=pyhd8ed1ab_0
• gdk-pixbuf=2.44.6=h4e57454_0
• glib-tools=2.86.4=h60c1bae_1
• gmp=6.3.0=h7bae524_2
• gmpy2=2.3.0=py312he1ee7cc_1
• graphite2=1.3.14=hec049ff_2
• graphviz=13.1.2=hcd33d8b_0
• greenlet=3.3.2=py312h6510ced_0
• griddataformats=1.1.0=pyhd8ed1ab_1
• gsd=4.2.0=h9bff6d5_0
• gtk3=3.24.43=h5febe37_6
• gts=0.7.6=he42f4ea_4
• h11=0.16.0=pyhcf101f3_1
• h2=4.3.0=pyhcf101f3_0
• h5py=3.16.0=nompi_py312hdd01ddf_101
• harfbuzz=12.2.0=haf38c7b_0
• hdf4=4.2.15=h2ee6834_7
• hdf5=1.14.6=nompi_had3affe_108
• hicolor-icon-theme=0.17=hce30654_3
• hpack=4.1.0=pyhd8ed1ab_0
• httpcore=1.0.9=pyh29332c3_0
• httpx=0.28.1=pyhd8ed1ab_0
• hyperframe=6.1.0=pyhd8ed1ab_0
• icu=75.1=hfee45f7_0
• idna=3.11=pyhd8ed1ab_0
• importlib-metadata=8.8.0=pyhcf101f3_0
• importlib_resources=6.5.2=pyhd8ed1ab_0
• iniconfig=2.3.0=pyhd8ed1ab_0
• ipycytoscape=1.3.3=pyhd8ed1ab_1
• ipykernel=7.2.0=pyh5552912_1
• ipython=9.12.0=pyhecfbec7_0
• ipython_pygments_lexers=1.1.1=pyhd8ed1ab_0
• ipywidgets=8.1.8=pyhd8ed1ab_0
• isoduration=20.11.0=pyhd8ed1ab_1
• jax=0.9.0=pyhd8ed1ab_0
• jaxlib=0.9.0=cpu_py312h82b1990_0
• jedi=0.19.2=pyhd8ed1ab_1
• jinja2=3.1.6=pyhcf101f3_1
• joblib=1.5.3=pyhd8ed1ab_0
• jq=1.8.1=hbc156a2_0
• json5=0.14.0=pyhd8ed1ab_0
• jsonpointer=3.1.1=pyhcf101f3_0
• jsonschema=4.26.0=pyhcf101f3_0
• jsonschema-specifications=2025.9.1=pyhcf101f3_0
• jsonschema-with-format-nongpl=4.26.0=hcf101f3_0
• jupyter-lsp=2.3.0=pyhcf101f3_0
• jupyter_client=8.8.0=pyhcf101f3_0
• jupyter_core=5.9.1=pyhc90fa1f_0
• jupyter_events=0.12.0=pyhe01879c_0
• jupyter_server=2.17.0=pyhcf101f3_0
• jupyter_server_terminals=0.5.4=pyhcf101f3_0
• jupyterlab=4.5.6=pyhd8ed1ab_0
• jupyterlab_pygments=0.3.0=pyhd8ed1ab_2
• jupyterlab_server=2.28.0=pyhcf101f3_0
• jupyterlab_widgets=3.0.16=pyhcf101f3_1
• kartograf=1.2.0=pyhd8ed1ab_0
• khronos-opencl-icd-loader=2024.10.24=h5505292_1
• kiwisolver=1.5.0=py312h3093aea_0
• konnektor=0.2.1=pyhcf101f3_0
• krb5=1.22.2=h385eeb1_0
• lark=1.3.1=pyhd8ed1ab_0
• lcms2=2.18=hdfa7624_0
• lerc=4.1.0=h1eee2c3_0
• leveldb=1.23=hd80cfd0_2
• libabseil=20250512.1=cxx17_hd41c47c_0
• libaec=1.1.5=h8664d51_0
• libblas=3.11.0=6_h51639a9_openblas
• libboost=1.86.0=hf493ff8_4
• libboost-python=1.86.0=py312hdeff4eb_5
• libbrotlicommon=1.2.0=hc919400_1
• libbrotlidec=1.2.0=hc919400_1
• libbrotlienc=1.2.0=hc919400_1
• libcblas=3.11.0=6_hb0561ab_openblas
• libcurl=8.19.0=hd5a2499_0
• libcxx=22.1.2=h55c6f16_0
• libdeflate=1.25=hc11a715_0
• libedit=3.1.20250104=pl5321hafb1f1b_0
• libev=4.33=h93a5062_2
• libexpat=2.7.5=hf6b4638_0
• libffi=3.5.2=he5f378a_0
• libfreetype=2.14.3=hce30654_0
• libfreetype6=2.14.3=hdfa99f5_0
• libgcc=15.2.0=hcbb3090_18
• libgd=2.3.3=hb2c3a21_11
• libgfortran=15.2.0=h07b0088_18
• libgfortran5=15.2.0=hdae7583_18
• libglib=2.86.4=he378b5c_1
• libgrpc=1.73.1=h3063b79_1
• libiconv=1.18=h23cfdf5_2
• libintl=0.25.1=h493aca8_0
• libjpeg-turbo=3.1.2=hc919400_0
• liblapack=3.11.0=6_hd9741b5_openblas
• liblzma=5.8.2=h8088a28_0
• libnetcdf=4.9.2=nompi_h2d3d5cf_118
• libnghttp2=1.68.1=h8f3e76b_0
• libopenblas=0.3.32=openmp_he657e61_0
• libpng=1.6.56=h132b30e_0
• libpq=16.13=h795a7be_0
• libprotobuf=6.31.1=h98f38fd_4
• librdkit=2024.09.2=he5d30ef_0
• libre2-11=2025.11.05=h91c62da_0
• librsvg=2.58.4=h266df6f_3
• libsodium=1.0.21=h1a92334_3
• libsqlite=3.48.0=h3f77e49_1
• libssh2=1.11.1=h1590b86_0
• libtiff=4.7.1=h4030677_1
• libtorch=2.8.0=cpu_generic_h853542e_2
• libuv=1.51.0=h6caf38d_1
• libwebp-base=1.6.0=h07db88b_0
• libxcb=1.17.0=hdb1d25a_0
• libxml2=2.13.9=h4a9ca0c_0
• libxslt=1.1.43=h429d6fd_0
• libzip=1.11.2=h1336266_0
• libzlib=1.3.2=h8088a28_2
• lightning-utilities=0.15.3=pyhd8ed1ab_0
• llvm-openmp=22.1.2=hc7d1edf_0
• llvmlite=0.46.0=py312hc82e5dd_0
• lomap2=3.2.1=pyhd8ed1ab_3
• lxml=6.0.2=py312hfcb44ad_0
• lz4-c=1.10.0=h286801f_1
• markdown-it-py=4.0.0=pyhd8ed1ab_0
• markupsafe=3.0.3=py312h04c11ed_1
• matplotlib-base=3.10.8=py312h605b88b_0
• matplotlib-inline=0.2.1=pyhd8ed1ab_0
• mda-xdrlib=0.2.0=pyhd8ed1ab_1
• mdanalysis=2.10.0=py312h5978115_1
• mdtraj=1.11.1=np2py312h2df5d9b_1
• mdurl=0.1.2=pyhd8ed1ab_1
• mistune=3.2.0=pyhcf101f3_0
• ml_dtypes=0.5.4=np2py312h60fbb24_1
• mmtf-python=1.1.3=pyhd8ed1ab_0
• mpc=1.4.0=h169892a_0
• mpfr=4.2.2=h6bc93b0_0
• mpiplus=v0.0.2=pyhd8ed1ab_0
• mpmath=1.4.1=pyhd8ed1ab_0
• mrcfile=1.5.4=pyhd8ed1ab_0
• msgpack-python=1.1.2=py312h84eede6_1
• munkres=1.1.4=pyhd8ed1ab_1
• narwhals=2.18.1=pyhcf101f3_1
• nbclient=0.10.4=pyhd8ed1ab_0
• nbconvert-core=7.17.0=pyhcf101f3_0
• nbformat=5.10.4=pyhd8ed1ab_1
• ncurses=6.5=h5e97a16_3
• nest-asyncio=1.6.0=pyhd8ed1ab_1
• netcdf-fortran=4.6.2=nompi_h170865b_101
• netcdf4=1.7.2=nompi_py312hd50420b_103
• networkx=3.6.1=pyhcf101f3_0
• nomkl=1.0=h5ca1d4c_0
• notebook=7.5.5=pyhcf101f3_0
• notebook-shim=0.2.4=pyhd8ed1ab_1
• numba=0.64.0=py312h2d3d6e9_0
• numexpr=2.14.1=py312h3de7d89_1
• numpy=2.2.6=py312h7c1f314_0
• ocl_icd_wrapper_apple=1.0.0=h27ca646_0
• oniguruma=6.9.10=h5505292_0
• opencl-headers=2025.06.13=hf6b4638_0
• openfe-analysis=0.3.1=pyhd8ed1ab_0
• openfe-base=1.8.0=pyhd8ed1ab_1
• openff-amber-ff-ports=2025.09.0=pyhd8ed1ab_0
• openff-forcefields=2026.01.0=pyhd8ed1ab_0
• openff-interchange=0.4.11=pyhd8ed1ab_1
• openff-interchange-base=0.4.11=pyhd8ed1ab_1
• openff-nagl=0.5.4=pyhd8ed1ab_1
• openff-nagl-base=0.5.4=pyhd8ed1ab_1
• openff-nagl-models=2025.9.0=pyhd8ed1ab_0
• openff-toolkit=0.18.0=pyhd8ed1ab_3
• openff-toolkit-base=0.18.0=pyhd8ed1ab_3
• openff-units=0.3.1=pyhd8ed1ab_2
• openff-utilities=0.1.16=pyhd8ed1ab_1
• openjpeg=2.5.4=hd9e9057_0
• openmm=8.2.0=py312h505867f_3_khronos
• openmmforcefields=0.15.1=pyhcf101f3_1
• openmmtools=0.25.3=pyhd8ed1ab_0
• openssl=3.6.1=hd24854e_1
• opt_einsum=3.4.0=pyhd8ed1ab_1
• optree=0.19.0=py312h766f71e_0
• overrides=7.7.0=pyhd8ed1ab_1
• packaging=26.0=pyhcf101f3_0
• pandas=2.3.3=py312h5978115_2
• pandocfilters=1.5.0=pyhd8ed1ab_0
• pango=1.56.4=h875632e_0
• parmed=4.3.1=py312h0ec3ceb_0
• parso=0.8.6=pyhcf101f3_0
• patsy=1.0.2=pyhcf101f3_0
• pcre2=10.47=h30297fc_0
• pdbfixer=1.12=pyhd8ed1ab_0
• perl=5.32.1=7_h4614cfb_perl5
• pexpect=4.9.0=pyhd8ed1ab_1
• pillow=12.1.1=py312h4e908a4_0
• pint=0.24.4=pyhe01879c_2
• pip=26.0.1=pyh8b19718_0
• pixman=0.46.4=h81086ad_1
• platformdirs=4.9.4=pyhcf101f3_0
• plotly=6.6.0=pyhd8ed1ab_0
• plugcli=0.2.1=pyhd8ed1ab_0
• pluggy=1.6.0=pyhf9edf01_1
• plyvel=1.5.1=py312h84eede6_1
• pontibus=0.4.0=pyhd8ed1ab_0
• pooch=1.9.0=pyhd8ed1ab_0
• prometheus_client=0.24.1=pyhd8ed1ab_0
• prompt-toolkit=3.0.52=pyha770c72_0
• psutil=7.1.3=py312h37e1c23_0
• pthread-stubs=0.4=hd74edd7_1002
• ptyprocess=0.7.0=pyhd8ed1ab_1
• pure_eval=0.2.3=pyhd8ed1ab_1
• py-cpuinfo=9.0.0=pyhd8ed1ab_1
• py3dmol=2.5.4=pyhd8ed1ab_0
• pybind11=2.13.6=pyhff21bea_3
• pybind11-abi=4=hd8ed1ab_3
• pybind11-global=2.13.6=pyh217bc35_3
• pycairo=1.29.0=py312h2437029_0
• pycparser=2.22=pyh29332c3_1
• pydantic=2.11.10=pyh3cfb1c2_0
• pydantic-core=2.33.2=py312hd3c0895_0
• pydantic-settings=2.13.1=pyhd8ed1ab_0
• pyedr=0.8.0=pyhd8ed1ab_1
• pygments=2.20.0=pyhd8ed1ab_0
• pygraphviz=1.14=py312he738534_2
• pymbar=4.2.0=pyha770c72_2
• pymbar-core=4.2.0=pyhd8ed1ab_2
• pyobjc-core=12.1=py312h19bbe71_0
• pyobjc-framework-cocoa=12.1=py312h1de3e18_0
• pyparsing=3.3.2=pyhcf101f3_0
• pysocks=1.7.1=pyha55dd90_7
• pytables=3.11.1=py312h53cfef5_1
• pytest=9.0.2=pyhcf101f3_0
• pytest-datadir=1.8.0=pyhd8ed1ab_0
• pytest-regressions=2.10.0=pyhc455866_0
• pytest-rerunfailures=16.1=pyhd8ed1ab_0
• python=3.12.9=hc22306f_0_cpython
• python-constraint=1.4.0=pyhff2d567_1
• python-dateutil=2.9.0.post0=pyhe01879c_2
• python-dotenv=1.2.2=pyhcf101f3_0
• python-fastjsonschema=2.21.2=pyhe01879c_0
• python-gil=3.12.13=hd8ed1ab_0
• python-json-logger=2.0.7=pyhd8ed1ab_0
• python-tzdata=2025.3=pyhd8ed1ab_0
• python_abi=3.12=8_cp312
• pytng=0.3.4=py312he040d82_0
• pytorch=2.8.0=cpu_generic_py312_h05d1414_2
• pytorch-lightning=2.6.1=pyhcf101f3_0
• pytz=2026.1.post1=pyhcf101f3_0
• pyyaml=6.0.3=py312h04c11ed_1
• pyzmq=27.1.0=py312h022ad19_2
• qhull=2020.2=h420ef59_5
• rdkit=2024.09.2=py312h2f1d5c7_0
• re2=2025.11.05=h64b956e_0
• readline=8.3=h46df422_0
• referencing=0.37.0=pyhcf101f3_0
• reportlab=4.4.10=pyhcf101f3_1
• requests=2.33.1=pyhcf101f3_0
• rfc3339-validator=0.1.4=pyhd8ed1ab_1
• rfc3986-validator=0.1.1=pyh9f0ad1d_0
• rfc3987-syntax=1.1.0=pyhe01879c_1
• rich=14.3.3=pyhcf101f3_0
• rlpycairo=0.4.0=pyh6c17108_0
• rpds-py=0.30.0=py312h6ef9ec0_0
• scikit-learn=1.8.0=np2py312he7bfc6a_1
• scikit-mol=0.6.1=pyhd8ed1ab_0
• scipy=1.17.1=py312h0f234b1_0
• seaborn=0.13.2=hd8ed1ab_3
• seaborn-base=0.13.2=pyhd8ed1ab_3
• send2trash=2.1.0=pyh5552912_1
• setuptools=82.0.1=pyh332efcf_0
• six=1.17.0=pyhe01879c_1
• sleef=3.9.0=hb028509_0
• snappy=1.2.2=hada39a4_1
• sniffio=1.3.1=pyhd8ed1ab_2
• soupsieve=2.8.3=pyhd8ed1ab_0
• spectate=1.0.1=pyhd8ed1ab_0
• sqlalchemy=2.0.48=py312hb3ab3e3_0
• stack_data=0.6.3=pyhd8ed1ab_1
• statsmodels=0.14.6=py312ha11c99a_0
• stratocaster=0.1.2=pyhd8ed1ab_0
• sympy=1.14.0=pyh2585a3b_106
• terminado=0.18.1=pyhc90fa1f_1
• threadpoolctl=3.6.0=pyhecae5ae_0
• tidynamics=1.1.2=pyhd8ed1ab_0
• tinycss2=1.4.0=pyhd8ed1ab_0
• tinydb=4.8.2=pyhd8ed1ab_1
• tk=8.6.13=h892fb3f_3
• tomli=2.4.1=pyhcf101f3_0
• torchmetrics=1.9.0=pyhd8ed1ab_0
• tornado=6.5.5=py312h2bbb03f_0
• tqdm=4.67.3=pyh8f84b5b_0
• traitlets=5.14.3=pyhd8ed1ab_1
• typing-extensions=4.15.0=h396c80c_0
• typing-inspection=0.4.2=pyhd8ed1ab_1
• typing_extensions=4.15.0=pyhcf101f3_0
• typing_utils=0.1.0=pyhd8ed1ab_1
• tzdata=2025c=hc9c84f9_1
• unicodedata2=17.0.1=py312h2bbb03f_0
• uri-template=1.3.0=pyhd8ed1ab_1
• urllib3=2.6.3=pyhd8ed1ab_0
• validators=0.35.0=pyhd8ed1ab_0
• wcwidth=0.6.0=pyhd8ed1ab_0
• webcolors=25.10.0=pyhd8ed1ab_0
• webencodings=0.5.1=pyhd8ed1ab_3
• websocket-client=1.9.0=pyhd8ed1ab_0
• wheel=0.46.3=pyhd8ed1ab_0
• widgetsnbextension=4.0.15=pyhd8ed1ab_0
• xmltodict=1.0.2=pyhcf101f3_0
• xorg-libice=1.1.2=h5505292_0
• xorg-libsm=1.2.6=h5505292_0
• xorg-libx11=1.8.13=hf948f5a_0
• xorg-libxau=1.0.12=hc919400_1
• xorg-libxdmcp=1.1.5=hc919400_1
• xorg-libxext=1.3.7=h84a0fba_0
• xorg-libxt=1.3.1=h5505292_0
• yaml=0.2.5=h925e9cb_3
• zeromq=4.3.5=h4818236_10
• zipp=3.23.0=pyhcf101f3_1
• zlib=1.3.2=h8088a28_2
• zlib-ng=2.3.3=hed4e4f5_1
• zstandard=0.25.0=py312h37e1c23_1
• zstd=1.5.7=hbf9d68e_6
• pip:
  • alchemiscale_fah==0.3.1
  • gufe==1.9.0
  • openfe==1.8.1

prefix: "/Users/jenniferclark/mamba/envs/alchemiscale-client"