Skip to content

spack update to 0.23.1#1413

Merged
cyrush merged 14 commits into
developfrom
task/2025_03_update_uberenv
Apr 2, 2025
Merged

spack update to 0.23.1#1413
cyrush merged 14 commits into
developfrom
task/2025_03_update_uberenv

Conversation

@cyrush
Copy link
Copy Markdown
Member

@cyrush cyrush commented Mar 31, 2025

No description provided.

@cyrush
Copy link
Copy Markdown
Member Author

cyrush commented Mar 31, 2025

Cannot satisfy 'cmake@3.12:' and 'cmake@3.21.1
        required because uberenv-conduit@develop%gcc~adios~doc~mpi~parmetis~python~silo ^cmake@3.21.1 requested explicitly 
        required because hdf5 depends on cmake@3.12: 
          required because uberenv-conduit depends on hdf5@1.8.0:1.8 when +hdf5+hdf5_compat 
            required because uberenv-conduit@develop%gcc~adios~doc~mpi~parmetis~python~silo ^cmake@3.21.1 requested explicitly 

Seems to me that 3.21.1 > 3.12 ...

@cyrush
Copy link
Copy Markdown
Member Author

cyrush commented Mar 31, 2025

==> Error: failed to concretize `uberenv-conduit@develop%gcc~adios~doc~mpi~parmetis~python~silo ^cmake@3.21.1` for the following reasons:
     1. Cannot satisfy 'cmake@3.21.1'
     2. Cannot satisfy 'cmake@3.21.1'
        required because uberenv-conduit@develop%gcc~adios~doc~mpi~parmetis~python~silo ^cmake@3.21.1 requested explicitly
     3. Cannot satisfy 'cmake@3.21.0:' and 'cmake@3.21.1
        required because uberenv-conduit@develop%gcc~adios~doc~mpi~parmetis~python~silo ^cmake@3.21.1 requested explicitly
        required because uberenv-conduit depends on cmake@3.21.0: when @0.9.0:
          required because uberenv-conduit@develop%gcc~adios~doc~mpi~parmetis~python~silo ^cmake@3.21.1 requested explicitly
     4. Cannot satisfy 'cmake@3.14.1:' and 'cmake@3.21.1
        required because uberenv-conduit@develop%gcc~adios~doc~mpi~parmetis~python~silo ^cmake@3.21.1 requested explicitly
        required because uberenv-conduit depends on cmake@3.14.1:
          required because uberenv-conduit@develop%gcc~adios~doc~mpi~parmetis~python~silo ^cmake@3.21.1 requested explicitly
     5. Cannot satisfy 'cmake@3.12:' and 'cmake@3.21.1
        required because hdf5 depends on cmake@3.12:
          required because uberenv-conduit depends on hdf5@1.8.0:1.8 when +hdf5+hdf5_compat
            required because uberenv-conduit@develop%gcc~adios~doc~mpi~parmetis~python~silo ^cmake@3.21.1 requested explicitly
        required because uberenv-conduit@develop%gcc~adios~doc~mpi~parmetis~python~silo ^cmake@3.21.1 requested explicitly

@cyrush
Copy link
Copy Markdown
Member Author

cyrush commented Mar 31, 2025

Issue was that spack no longer has version entry for cmake@3.21.1

@cyrush cyrush added this to the 0.9.4 milestone Mar 31, 2025
@cyrush
Copy link
Copy Markdown
Member Author

cyrush commented Apr 2, 2025

a key difference between conduit and ascent spack package logic:

https://github.com/spack/spack/blob/2bfcc69fa870d3c6919be87593f22647981b648a/var/spack/repos/builtin/packages/conduit/package.py#L311

this one line change made it so that the host config wasn't copied during the host config stage, but instead only during final install.

This undermined the uberenv-pkg mode, which depends on the host config being generated earlier -- since it stops after the host config stage ...

I added to add an extra copy to the derived uberenv-ascent package.

Hunting this down took about a day of builds and sleuthing.

Here is the ascent logic for ref:
https://github.com/spack/spack/blob/2bfcc69fa870d3c6919be87593f22647981b648a/var/spack/repos/builtin/packages/ascent/package.py#L377

remaining failures:

minimal failure: MPI is off, but we are trying to build conduit relay mpi for some reason
static build failure: spack silo defaults to +python, however silo fails to build with static & python combo
other failures:
need :

depends_on("py-pip", type=("build", "run"))

In the conduit package proper in spack, py-pip is only listed as a build dep.

This differs from ascent (another subtle difference that we should resolve)

@cyrush
Copy link
Copy Markdown
Member Author

cyrush commented Apr 2, 2025

From the spack lock file:

{"name":"hdf5","version":"1.8.23","arch":{"platform":"linux","platform_os":"ubuntu20.04","target":{"name":"zen2","vendor":"AuthenticAMD","features":["abm","aes","avx","avx2","bmi1","bmi2","clflushopt","clwb","clzero","cx16","f16c","fma","fsgsbase","mmx","movbe","pclmulqdq","popcnt","rdseed","sse","sse2","sse4_1","sse4_2","sse4a","ssse3","xsavec","xsaveopt"],"generation":0,"parents":["zen"],"cpupart":""}},"compiler":{"name":"gcc","version":"9.4.0"},"namespace":"builtin","parameters":{"api":"default","build_system":"cmake","build_type":"Release","cxx":false,"fortran":false,"generator":"make","hl":false,"ipo":false,"mpi":true,"patches":

We are getting hdf5+mpi, however we are requesting hdf5~mpi ...

@cyrush
Copy link
Copy Markdown
Member Author

cyrush commented Apr 2, 2025

confirming it is HDF5 that is flipping MPI_FOUND to true:

-- 3d start MPI_FOUND 
-- Using yyjson for JSON parsing
-- ZLIB_LIBRARIES: /home/user/conduit/uberenv_libs/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-9.4.0/zlib-ng-2.2.1-jv7dc55ozib6236evmw47kezajlmmt6j/lib/libz.so
-- Looking for HDF5 at: /home/user/conduit/uberenv_libs/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-9.4.0/hdf5-1.8.23-4avd2iazscpkeqx7pfm47pa4txhayapy
-- HDF5_DIR_REAL=/home/user/conduit/uberenv_libs/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-9.4.0/hdf5-1.8.23-4avd2iazscpkeqx7pfm47pa4txhayapy
-- Checking that found HDF5_INCLUDE_DIRS are in HDF5_DIR
-- HDF5_INCLUDE_DIRS=/home/user/conduit/uberenv_libs/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-9.4.0/hdf5-1.8.23-4avd2iazscpkeqx7pfm47pa4txhayapy/include
--  /home/user/conduit/uberenv_libs/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-9.4.0/hdf5-1.8.23-4avd2iazscpkeqx7pfm47pa4txhayapy/include includes HDF5_DIR (/home/user/conduit/uberenv_libs/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-9.4.0/hdf5-1.8.23-4avd2iazscpkeqx7pfm47pa4txhayapy)
--  /home/user/conduit/uberenv_libs/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-9.4.0/hdf5-1.8.23-4avd2iazscpkeqx7pfm47pa4txhayapy/include includes HDF5_REAL_DIR ()
-- Attempting to find libhdf5.settings in HDF5_REAL_DIR...
-- Found HDF5 settings file: /home/user/conduit/uberenv_libs/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-9.4.0/hdf5-1.8.23-4avd2iazscpkeqx7pfm47pa4txhayapy/lib/libhdf5.settings
-- Reading 'HDF5_SETTINGS_FILE' to determine hdf5 config settings
-- HDF5 Include Dirs: /home/user/conduit/uberenv_libs/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-9.4.0/hdf5-1.8.23-4avd2iazscpkeqx7pfm47pa4txhayapy/include
-- HDF5 Libraries:    hdf5-shared;m;dl;/usr/lib/x86_64-linux-gnu/libmpich.so;/home/user/conduit/uberenv_libs/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-9.4.0/zlib-ng-2.2.1-jv7dc55ozib6236evmw47kezajlmmt6j/lib/libz.so
-- HDF5 Definitions:  
-- HDF5 is parallel:  ON
-- HDF5 Thirdparty Include Flags: 
-- HDF5 Thirdparty Link Flags:   -lm -ldl /usr/lib/x86_64-linux-gnu/libmpich.so /home/user/conduit/uberenv_libs/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-9.4.0/zlib-ng-2.2.1-jv7dc55ozib6236evmw47kezajlmmt6j/lib/libz.so
-- HDF5 using hdf5-shared target
-- 3d end MPI_FOUND TRUE

@cyrush
Copy link
Copy Markdown
Member Author

cyrush commented Apr 2, 2025

hdf5+mpi this was an issue with the spack spec not getting passed correctly in our ci

@cyrush
Copy link
Copy Markdown
Member Author

cyrush commented Apr 2, 2025

More fun:

Will try a different candidate, due to conflict:
    sphinxcontrib-qthelp 2.0.0 depends on Python >=3.9
ERROR: Package 'sphinxcontrib-qthelp' requires a different Python: 3.8.16 not in '>=3.9'

Bummer about this message is that is is not shown in the console, I had to read the log to see what the underlying error was.

I also think this is an error during pip install and solve -- not with spack constraints -- so this slipped past the spack defenses.

I am going to swing to python 3.9.

@cyrush
Copy link
Copy Markdown
Member Author

cyrush commented Apr 2, 2025

No more mysteries!

It was the spack concretizer, with the candlestick, in the cloud.

@cyrush cyrush merged commit 0099ba4 into develop Apr 2, 2025
28 checks passed
@cyrush cyrush deleted the task/2025_03_update_uberenv branch April 2, 2025 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant