YAKL and SAMXX CIME and CMake Integration Notes

The E3SM CMake files are located in E3SM/components and E3SM/components/cmake.

Adding the USE_CUDA CMake variable

This variable needs to be visible at the highest level CMakeLists.txt, so it has to be specified through CIME using the Macros.cmake file in E3SM/cime_config/machines/config_compilers.xml.

To make this variable available to add to that file, however, it had to first be added to the XML schema in cime/config/xml_schemas/config_compilers_v2.xsd. It was added in the <xs:group name="compilerVars"> group as a choice as follows:

      <xs:element name="CUDA_FLAGS" type="flagsVar"/>
      <xs:element name="USE_CUDA" type="upperBoolean"/>

Then they are added to appropriate machine, compiler configurations in config_compilers.xml as:

  <CUDA_FLAGS>
    <append> -O3 -arch sm_70 --use_fast_math </append>
  </CUDA_FLAGS>
  <USE_CUDA>TRUE</USE_CUDA>

Finally, when CMake starts its top-level CMakeLists.txt for E3SM, it sees these things set in the local Macros.cmake files in the case build directory.

Creating USE_SAMXX and USE_YAKL CMake variables

Go to E3SM/components/cmake/common_setup.cmake, and add new variables there. Use existing code to get the feel for where to test for adding a given variable. CAM often hooks into CIME through the ${CAM_CONFIG_OPTS} variable that comes from the CAM configure command. USE_YAKL and USE_SAMXX are created with the code below in cmake/build_model.cmake

# Look for -crm samxx in the CAM_CONFIG_OPTS CIME variable
# If it's found, then enable USE_SAMXX
string(FIND "${CAM_CONFIG_OPTS}" "-crm samxx" HAS_SAMXX)
if (NOT HAS_SAMXX EQUAL -1)
  # The following is for the SAMXX code:
  set(USE_SAMXX TRUE)
endif()

# If samxx is being used, then YAKL must be used as well
set(USE_YAKL ${USE_SAMXX})

# If YAKL is being used, then we need to enable USE_CXX
if (${USE_YAKL})
  set(USE_CXX TRUE)
endif()

We then add the YAKL and SAMXX builds into the CMake file for "eam" with the following code: