Update macos.platform and docs

[buchanankerswell]

This PR adds some basic checks to the default macOS toolchain:

1. checks if cmake is installed locally or via homebrew. If not, exit candi with a hint.
2. checks for incompatible (non-mpi enabled) hdf5 installed via homebrew. If hdf5 is installed via homebrew, exit candi with a hint.
3. uses Trilinos version 16.1 by default to avoid compilations errors on macOS arm64 stemming from a known Kokkos bug Use Trilinos 16.1.0 to fix Kokkos compilation error #401

[buchanankerswell]

The new PR commit moves cmake checks from macos.platform to candi.sh. It also fixes typos and updates urls in the macOS.platform header.

[tjhei]

ping me once you have it updated to not require cmake.

[buchanankerswell]

Hi Timo,

I am working on a couple more changes on this PR, so don't merge it yet. I will ping you when it's ready for review.

[buchanankerswell]

Timo, this is ready for review now. I tested a fresh install from the ASPECT wiki instructions:

git clone https://github.com/buchanankerswell/candi.git
cd candi
git checkout update-macos-platform
curl -O https://raw.githubusercontent.com/geodynamics/aspect/main/contrib/install/local.cfg
./candi.sh -j 8

CMake is no longer required as a prerequisite because candi can now build it for macOS. The only prerequisite is MPI, which I added to the ASPECT wiki page. You could update the deal.II wiki page to be consistent (e.g., add the curl line for local.cfg).

[tjhei]

CMake is no longer required as a prerequisite

I prefer installing cmake via homebrew (or other system package managers). This is because it will end up in the path in all cases. Additionally installing via candi can cause conflicts if you happen to have more than one CMake version installed.

I also saw that you removed gcc11: We do not need gcc11 specifically, but I wanted to avoid homebrew always choosing the latest version, which might not work (new errors/warnings/etc.). Maybe update to something newer than 11 but fix the version?

[buchanankerswell]

Hi Timo, yeah I understand the cmake issue. I'll can add cmake and gcc@13 to the brew install command in the macos.platform header and ASPECT wiki page

# - Install the following via Homebrew
# $  brew install cmake gcc@13 openmpi

Then the question is what to do about aspect/contrib/install/local.cfg

PACKAGES="load:dealii-prepare once:cmake once:astyle once:hdf5 once:netcdf once:sundials once:p4est once:trilinos dealii"

NATIVE_OPTIMIZATIONS=ON
BUILD_EXAMPLES=OFF
USE_DEAL_II_CMAKE_MPI_COMPILER=ON
DEAL_II_CONFOPTS="-DDEAL_II_WITH_COMPLEX_VALUES=OFF"

As currently written, candi will install and use the bundled cmake on macOS. One option is to add some logic to macos.platform for detecting and removing once:make from the packages list?

# Remove once:cmake from the PACKAGES list
PACKAGES="${PACKAGES//once:cmake /}"

What do you think? Does it make sense to add brew install cmake gcc@13 openmpi to the macos.platform header and PACKAGES="${PACKAGES//once:cmake /}" to the macos.platform logic?

[buchanankerswell]

@tjhei, I added a check in macos.platform that removes cmake or once:cmake from the candi PACKAGES list.

  if [[ "$PACKAGES" == *"once:cmake"* || "$PACKAGES" == *"cmake"* ]]; then
    # Remove cmake from the PACKAGES list because the preferred install method
    # for cmake on macOS is via Homebrew (so cmake will always be in the path)
    if [[ "$PACKAGES" == *"once:cmake"* ]]; then
      cecho ${INFO} "Found 'once:cmake' in PACKAGES!"
      PACKAGES="${PACKAGES//once:cmake /}"
    elif [[ "$PACKAGES" == *"cmake"* ]]; then
      cecho ${INFO} "Found 'cmake' in PACKAGES!"
      PACKAGES="${PACKAGES//cmake /}"
    fi
    cecho ${INFO} "candi will skip the cmake install because the preferred method"
    cecho ${INFO} "for installing cmake on macOS is via Homebrew."
  fi

The PACKAGE list in aspect/contrib/install/local.cfg includes once:make, so this addition to macos.platform ensures that candi does not install cmake for macOS users.

I updated the macos.platform header docs and updated gcc@11 to gcc@13 for Intel Mac users. These changes are also reflected in the ASPECT wiki page.

[buchanankerswell]

@tjhei,

I realize that if we decide to skip cmake install for macOS users, the changes to deal.II-toolchain/packages/cmake.package are unnecessary.

[tjhei]

Can you link to the deal.II wiki instead? Candi is not ASPECT specific.

[buchanankerswell]

Hi @tjhei,

Thanks for getting back to this.

I will link https://github.com/dealii/dealii/wiki/Apple-ARM-M1-OSX instead, but it should be updated to be consistent with https://github.com/geodynamics/aspect/wiki/Installation-on-macOS-arm64.

In fact, the dealii wiki page could be updated in a few places for Mac users. I would make these changes myself, but I cannot edit the dealii wiki.

[tjhei]

Ok. If you share the proposed text with me, I can make the edits.

[buchanankerswell]

This document describes installation instructions for deal.II on macOS arm64 devices (M1–M4 Macs).

**Please note:** while deal.II was found to be working using the instructions below, please keep in mind that it is not a commonly used platform by the core deal.II team and not everything is guaranteed to work. Future updates to macOS might also break what was once working. Additionally, some parts of deal.II are not optimized on arm64 and are likely more performant on x64 machines.

## 1. Set up system

Open Terminal and check that the command ``machine`` returns ``arm64e``. If not, you are running under Rosetta emulation.

Next, in the terminal, install Xcode command line tools via:

```bash
xcode-select --install
```

Check if Xcode command line tools are installed by running:

```bash
clang -v
```

Install Homebrew (a package manager for macOS) via:

```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```

## 2. Install CMake and MPI

The only prerequisites needed are CMake and MPI, and the easiest way to get them on macOS is via Homebrew:

```bash
brew install cmake open-mpi
```

## 3. Install deal.II

Install deal.II and its dependencies with candi:

```bash
git clone https://github.com/dealii/candi
cd candi
curl -O https://raw.githubusercontent.com/geodynamics/aspect/main/contrib/install/local.cfg
./candi.sh -j 8
```

[buchanankerswell]

Hi @tjhei,

Here is a summary of the PR changes:

1. removed the check in macos.platform
2. removed the macOS-specific changes to cmake.package (consistent with docs; prefer Homebrew installation)
3. point users to the dealii wiki page for macOS arm64 install

A couple of additional notes:

Installing CMake with homebrew is more convenient because it will always be in the user's path. This avoids the need to source the dealii dealii/configuration/enable.sh over and over.

The dealii wiki page for macOS arm64 should be updated to be consistent with the ASPECT macOS arm64 wiki page. I can make these changes, but will need privileges to update the dealii wiki

[tjhei]

This is a bit brittle. What if the user has a brew hdf5 and installs hdf using candi?

[buchanankerswell]

Thanks Timo, but I think I am missing something here. Can you explain further please?

candi exits before installing any packages if it finds the incompatible Homebrew version of hdf5. This is implemented in this block:

      # Exit if hdf5 is installed via homebrew because it is not mpi-enabled
      # and will cause compilation errors
      if brew list "hdf5" &>/dev/null; then
          cecho ${BAD} "Error: hdf5 found installed via Homebrew."
          cecho ${BAD} "This conflicts with the deal.II installation!"
          cecho ${BAD} "Please uninstall it using: 'brew uninstall hdf5'"
          cecho ${BAD} "and then run candi again."
          exit 1
      fi

I have tested by installing hdf5 via Homebrew and then running:

curl -O https://raw.githubusercontent.com/geodynamics/aspect/main/contrib/install/local.cfg
./candi.sh -j 8

*******************************************************************************
This is candi (compile and install)

Project: deal.II-toolchain: Found configuration.
using ./deal.II-toolchain/platforms/supported/macos.platform.

Operating System Type detected as: macos
Dynamic shared library file extension detected as: *.dylib
macOS arm64 detected!
Error: hdf5 found installed via Homebrew.
This conflicts with the deal.II installation!
Please uninstall it using: 'brew uninstall hdf5'
and then run candi again.

[tjhei]

The following situation should not produce an error because it will work:

• hdf5 is installed via homebrew
• the user then wants installs hdf using candi

[buchanankerswell]

Ah I understand, thanks for clarifying @tjhei.

I removed the whole block that checks hdf5. Then I tested by:

brew install hdf5
git clone <my branch of candi> && cd candi
curl -O https://raw.githubusercontent.com/geodynamics/aspect/main/contrib/install/local.cfg
./candi.sh -j 8

which breaks:

[ 51%] Building CXX object source/base/CMakeFiles/object_base_release.dir/hdf5.cc.o
In file included from /Users/bck/dealii-candi/tmp/unpack/deal.II-v9.6.1/source/base/hdf5.cc:19:
/Users/bck/dealii-candi/tmp/unpack/deal.II-v9.6.1/include/deal.II/base/hdf5.h:1479:30: error: use of undeclared identifier 'H5Pset_dxpl_mpio'; did you mean 'H5Pset_fapl_onion'?
 1479 |           const herr_t ret = H5Pset_dxpl_mpio(plist, H5FD_MPIO_COLLECTIVE);
      |                              ^~~~~~~~~~~~~~~~
      |                              H5Pset_fapl_onion
/opt/homebrew/include/H5FDonion.h:158:15: note: 'H5Pset_fapl_onion' declared here
  158 | H5_DLL herr_t H5Pset_fapl_onion(hid_t fapl_id, const H5FD_onion_fapl_info_t *fa);
      |               ^
In file included from /Users/bck/dealii-candi/tmp/unpack/deal.II-v9.6.1/source/base/hdf5.cc:19:
/Users/bck/dealii-candi/tmp/unpack/deal.II-v9.6.1/include/deal.II/base/hdf5.h:1479:54: error: cannot initialize a parameter of type 'const H5FD_onion_fapl_info_t *' with an rvalue of type 'H5FD_mpio_xfer_t'
 1479 |           const herr_t ret = H5Pset_dxpl_mpio(plist, H5FD_MPIO_COLLECTIVE);
      |                                                      ^~~~~~~~~~~~~~~~~~~~
/opt/homebrew/include/H5FDonion.h:158:78: note: passing argument to parameter 'fa' here
  158 | H5_DLL herr_t H5Pset_fapl_onion(hid_t fapl_id, const H5FD_onion_fapl_info_t *fa);
      |                                                                              ^
In file included from /Users/bck/dealii-candi/tmp/unpack/deal.II-v9.6.1/source/base/hdf5.cc:19:
/Users/bck/dealii-candi/tmp/unpack/deal.II-v9.6.1/include/deal.II/base/hdf5.h:1511:21: error: use of undeclared identifier 'H5Pget_mpio_actual_io_mode'
 1511 |               ret = H5Pget_mpio_actual_io_mode(plist, &io_mode);
      |                     ^
/Users/bck/dealii-candi/tmp/unpack/deal.II-v9.6.1/include/deal.II/base/hdf5.h:1514:17: error: use of undeclared identifier 'H5Pget_mpio_no_collective_cause'
 1514 |                 H5Pget_mpio_no_collective_cause(plist,
      |                 ^
4 errors generated.
make[2]: *** [source/base/CMakeFiles/object_base_release.dir/hdf5.cc.o] Error 1
make[1]: *** [source/base/CMakeFiles/object_base_release.dir/all] Error 2

The build log shows that candi is pointing to its own version of hdf5:

-- Defining target: interface_hdf5
--     VERSION:             1.12.2
--     LINK_LIBRARIES:      /Users/bck/dealii-candi/hdf5-1.12.2/lib/libhdf5.dylib;/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libz.tbd;/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libdl.tbd;/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib/libm.tbd;/opt/homebrew/Cellar/open-mpi/5.0.8/lib/libmpi.dylib
--     INCLUDE_DIRECTORIES: /Users/bck/dealii-candi/hdf5-1.12.2/include
--
-- DEAL_II_WITH_HDF5 successfully set up with external dependencies.

If you just brew uninstall hdf5 and run candi, everything works fine. Thoughts?

[buchanankerswell]

I tried setting the paths to header files explicitly in hdf5.package:

################################################################################
## HDF5                                                                       ##
################################################################################

MAJORVER=1.12;MINORVER=2;CHECKSUM=30172c75e436d7f2180e274071a4ca97

#MAJORVER=1.10;MINORVER=8;CHECKSUM=61063043a23dd4a32dcd73a704bb47f2
#MAJORVER=1.10;MINORVER=5;CHECKSUM=e115eeb66e944fa7814482415dd21cc4

VERSION=${MAJORVER}.${MINORVER}
NAME=hdf5-${VERSION}
SOURCE=https://www.hdfgroup.org/ftp/HDF5/releases/hdf5-${MAJORVER}/${NAME}/src/
PACKING=.tar.gz
BUILDCHAIN=autotools

CONFOPTS="--enable-shared --enable-parallel"

INSTALL_PATH=${INSTALL_PATH}/${NAME}

package_specific_register () {
    export HDF5_DIR=${INSTALL_PATH}
    export CXXFLAGS="-I${INSTALL_PATH}/include ${CXXFLAGS}"
    export CPPFLAGS="-I${INSTALL_PATH}/include ${CPPFLAGS}"
    export LDFLAGS="-L${INSTALL_PATH}/lib ${LDFLAGS}"
}

package_specific_conf () {
    # Generate configuration file
    CONFIG_FILE=${CONFIGURATION_PATH}/${NAME}
    rm -f ${CONFIG_FILE}
    echo "export HDF5_DIR=${INSTALL_PATH}" >> $CONFIG_FILE
    echo "export CXXFLAGS=-I${INSTALL_PATH}/include ${CXXFLAGS}" >> $CONFIG_FILE
    echo "export CPPFLAGS=-I${INSTALL_PATH}/include ${CPPFLAGS}" >> $CONFIG_FILE
    echo "export LDFLAGS=-L${INSTALL_PATH}/lib ${LDFLAGS}" >> $CONFIG_FILE
}

but for whatever reason the compiler still defaults to the Homebrew header files and the dealii build crashes.

Funny enough, there are no hdf5 header file issues when candi builds netcdf.

I will still try to resolve this issue, but for now the quickest and easiest way to get a clean install on macOS arm64 is to exit candi if a Homebrew version of hdf5 is detected, then ask the user to brew uninstall hdf5 and run candi again.

[buchanankerswell]

Hi @tjhei, this PR is ready for another check.

Here is a summary of the PR changes:

1. removed the if/fi block in macos.platform that does the string substitution on PACKAGES

I believe the block that checks the hdf5 install in macos.platform works as intended, but please give it a look.

Thanks

[buchanankerswell]

Hi @tjhei,

This PR is ready for another review. Summary of changes:

1. Resolve hdf5 conflicts by unlinking/relinking homebrew hdf5 (instead of exiting candi)

Although CMake finds the correct hdf5 header files when configuring dealii, clang tries to compile dealii with the homebrew hdf5 header files. I have tried to point clang towards the right header files without success. There is some default path search in clang that I do not understand.

Instead of exiting candi if it detects an existing homebrew hdf5, I have opted to temporarily hide the homebrew hdf5 source by brew unlink hdf5, which is undone after dealii is installed by brew link hdf5. This effectively addresses your comment:

The following situation should not produce an error because it will work:

• hdf5 is installed via homebrew
• the user then wants installs hdf using candi