Update the documentation

Author: xylar

docs/developers_guide/spack.md

@@ -14,8 +14,11 @@ To enable Spack-based environments for a new machine, you will need to:
    supported compiler and MPI library combination.
 2. Add system-provided (external) packages, saving time and preventing build
    failures if Spack attempts to build them from source.
-3. Optionally, provide shell script templates for module/environment setup if
-   needed.
+3. Prefer automatic shell script generation. Shell snippets used to be
+   maintained as templates in `mache.spack.templates`, but are now derived
+   primarily from the E3SM CIME machine configuration
+   (`mache/cime_machine_config/config_machines.xml`). Only add a minimal
+   template override when strictly necessary (see below).
 
 ## YAML Template Files
 
@@ -82,21 +85,46 @@ all:
     lapack: [[email protected]]
 ```
 
-## Shell Script Templates
+## Automatic shell script generation (preferred)
 
-If the machine requires special module loads or environment variables not handled by Spack, add corresponding shell script templates:
+When downstream packages call
+{py:func}`mache.spack.get_spack_script`, the module loads and
+environment-variable setup are constructed as follows:
 
-- `<machine>.sh` and/or `<machine>_<compiler>_<mpilib>.sh` for bash
-- `<machine>.csh` and/or `<machine>_<compiler>_<mpilib>.csh` for csh/tcsh
+1. Optional: activate Spack and the requested environment.
+2. Auto-generate a shell snippet from the E3SM CIME machine configuration
+   stored in `mache/cime_machine_config/config_machines.xml`, filtered for the
+   requested `(machine, compiler, mpilib)` and rendered for the target shell
+   (`sh` or `csh`).
+3. Append any Jinja2 template override found in
+   `mache/spack/templates/` named either `<machine>.<sh|csh>` or
+   `<machine>_<compiler>_<mpilib>.<sh|csh>`.
 
-These scripts are also Jinja2 templates and can use the same conditional logic as the YAML files.
+This pipeline greatly reduces maintenance and prevents drift between Mache and
+E3SM’s authoritative machine configuration. In most cases, you do not need to
+author or maintain shell script templates in Mache.
+
+### When to add a template override
+
+Only provide a small override in `mache/spack/templates/` if you need to:
+
+- Apply an adjustment that’s not appropriate for the shared E3SM CIME config
+  (machine-local quirk, temporary workaround, etc.).
+- Add conditional behavior toggled by
+  `include_e3sm_lapack` or `include_e3sm_hdf5_netcdf` (both exposed as Jinja
+  booleans in templates) that cannot be expressed in the CIME config.
+
+Templates are Jinja2 files and can use the same conditional logic as YAML
+templates.
 
 ## Testing
 
-After adding or modifying YAML and shell script templates:
+After adding or modifying YAML templates (or an exceptional shell override):
 
-1. Use the `make_spack_env` or `get_spack_script` functions in `mache.spack` to generate and test the environment.
-2. Confirm that all modules load and all external packages are correctly detected by Spack.
+1. Use `make_spack_env` or `get_spack_script` in `mache.spack` to generate and
+  test the environment and load scripts.
+2. Confirm that the generated shell snippet includes the expected module loads
+  from the CIME machine config and that Spack detects all external packages.
 3. Build and run a simple test application to verify the environment.
 
 ## Further Reading

docs/users_guide/spack/build.md

@@ -94,7 +94,8 @@ from mache.spack import get_spack_script
 ```
 
 **Purpose:**
-Generates a shell script snippet to activate a Spack environment and load any required modules or environment variables.
+Generates a shell script snippet to activate a Spack environment and load the
+required modules or environment variables.
 
 **Typical usage in downstream packages:**
 
@@ -118,13 +119,19 @@ spack_script = get_spack_script(
 )
 ```
 
-**Returns:**
-A string containing shell commands to:
+**How the script is composed:**
+
+The returned snippet is assembled in three steps:
 
-- Load required modules (if any).
-- Source the Spack setup script.
-- Activate the specified Spack environment.
-- Set any additional environment variables.
+1. Optionally source Spack and activate the requested environment.
+2. Auto-generate module loads and environment exports from the E3SM CIME
+  machine configuration (`mache/cime_machine_config/config_machines.xml`) for
+  the given `(machine, compiler, mpi)` and target shell (`sh` or `csh`).
+3. Append any Mache template override present in `mache/spack/templates/` named
+  `<machine>.<sh|csh>` or `<machine>_<compiler>_<mpi>.<sh|csh>`.
+
+This design keeps Mache aligned with E3SM’s authoritative machine
+configuration and minimizes maintenance.
 
 **Usage in activation scripts:**
 
@@ -142,7 +149,8 @@ from mache.spack import get_modules_env_vars_and_mpi_compilers
 ```
 
 **Purpose:**
-Returns the MPI compiler wrappers and a shell snippet to load modules and set environment variables for a given machine, compiler, and MPI library.
+Returns the MPI compiler wrappers and a shell snippet to load modules and set
+environment variables for a given machine, compiler, and MPI library.
 
 **Typical usage in downstream packages:**
 
@@ -169,13 +177,18 @@ mpicc, mpicxx, mpifc, mod_env_commands = get_modules_env_vars_and_mpi_compilers(
 - `mpifc`: Name of the MPI Fortran compiler wrapper (e.g., `mpif90` or `ftn`).
 - `mod_env_commands`: Shell commands to load modules and set environment variables.
 
-**Usage in build scripts:**
+**Notes and usage in build scripts:**
 
 ```bash
 {{ mod_env_commands }}
 # Now safe to use $mpicc, $mpicxx, $mpifc for building MPI-dependent software
 ```
 
+- This helper produces a minimal snippet based on Mache’s template overrides
+  (when present). It does not auto-generate content from
+  `config_machines.xml`. If you need the full environment setup used by
+  downstream activation scripts, prefer `get_spack_script`.
+
 ---
 
 ## Example: How Downstream Packages Use These Functions