E3SM-Project · xylar · Jul 18, 2025 · Jul 14, 2025 · Jul 14, 2025 · Jul 14, 2025
diff --git a/README.md b/README.md
@@ -7,6 +7,13 @@ A metapackage for a unified
 for analysis and other pre- and post-processing for the Energy Exascale Earth
 System Model (E3SM).
 
+## Documentation
+
+[Latest documentation](http://docs.e3sm.org/e3sm-unified/main) for users,
+developers and maintainers.
+
+## Getting Started
+
 E3SM-Unified currently supports Linux and OSX, and python >=3.9,<3.11.
 Support for Windows is not planned.
 

diff --git a/dev-speck.txt → dev-spec.txt b/dev-speck.txt → dev-spec.txt
diff --git a/docs/contributing.md b/docs/contributing.md
@@ -0,0 +1,55 @@
+# Contributing & Community
+
+We welcome contributions and feedback from all users and developers of
+E3SM-Unified. Whether you're updating packages, improving documentation, or
+reporting issues, your input helps strengthen the environment and its
+community.
+
+---
+
+## Ways to Contribute
+
+### ✏️ Documentation
+
+* Suggest improvements to the user guide or technical docs.
+* Fix typos or clarify instructions.
+* Add usage examples for tools you use regularly.
+
+### 🚀 Suggest or Update Packages
+
+* Request new tools or features by opening a GitHub Issue.
+* Propose version updates by:
+  * Editing the E3SM Confluence pages defining the next E3SM-Unified version
+    (if you have access)
+  * Or editing the `meta.yaml` (for conda package) or `defaults.cfg` (for spack
+    pacakges) and making a pull request (if you don't have access to E3SM's
+    Confluence pages).
+
+### ⚙️ Development & Testing
+
+* Help test release candidates on supported platforms.
+* Report issues you encounter.
+* Contribute improvements to tools in the E3SM ecosystem (e.g., `mache`,
+  `mpas-analysis`, `zppy`, `e3sm_diags`).
+
+---
+
+## Getting Started
+
+1. Fork the [e3sm-unified GitHub repository](https://github.com/E3SM-Project/e3sm-unified).
+2. Create a new branch for your changes.
+3. Submit a pull request (PR).
+4. Tag reviewers as needed (e.g., `@xylar`).
+
+We recommend following our naming conventions for release branches (e.g.,
+`update-to-1.12.0`).
+
+---
+
+## Communication
+
+* GitHub Issues: [E3SM-Unified GitHub](https://github.com/E3SM-Project/e3sm-unified/issues)
+* Slack: `#e3sm-help-postproc`
+* Email: [[email protected]](mailto:[email protected])
+
+Have questions about where to start? Just ask on Slack or open an issue!
diff --git a/docs/index.md b/docs/index.md
@@ -1,11 +1,49 @@
+# Welcome to E3SM-Unified Documentation
+
 ```{image} logo/e3sm_unified_logo_200.png
 :align: center
 :width: 200 px
 ```
 
-# E3SM-Unfied
+E3SM-Unified is a unified conda + Spack environment developed to support pre-
+and post-processing workflows for the Energy Exascale Earth System Model
+(E3SM). It bundles commonly used analysis, visualization, and workflow tools
+into a single portable environment.
+
+This documentation is for both **users** of the E3SM-Unified environment and
+**developers** who build, test, and deploy it on supported platforms.
+
+---
+
+## 🚀 Start Here
+
+* 📖 [Introduction](introduction.md): What E3SM-Unified is and why you should
+  use it
+* 🧪 [Quickstart Guide](quickstart.md): Load the environment and start using
+  tools
+
+---
+
+## 📚 Contents
+
+* 💠 [Using E3SM-Unified Tools](using-tools.md)
+* 🧪 [Testing Release Candidates](testing-release-candidates.md)
+* 🚚 [The E3SM-Unified Release Workflow](releasing/release-workflow.md)
+* 📦 [Package Catalog](packages.md)
+* ❓ [Troubleshooting & FAQs](troubleshooting.md)
+* 🤝 [Contributing & Community](contributing.md)
+
+---
+
+## 💬 Get Help
+
+```{admonition} Support
+- Slack: #e3sm-help-postproc
+- GitHub Issues: [E3SM-Unified on GitHub](https://github.com/E3SM-Project/e3sm-unified/issues)
+- Maintainer contact: [email protected]
+```
+
+---
 
-A metapackage for a unified
-[conda environment](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html)
-for analysis and other pre- and post-processing for the Energy Exascale Earth
-System Model (E3SM).
+This documentation is maintained by the E3SM Infrastructure Team and
+contributors across the community.
diff --git a/docs/introduction.md b/docs/introduction.md
@@ -0,0 +1,55 @@
+# Introduction to E3SM-Unified
+
+```{image} logo/e3sm_unified_logo_200.png
+:align: center
+:width: 200 px
+```
+
+## What is E3SM-Unified?
+
+E3SM-Unified is a unified conda-based environment that provides pre- and
+post-processing tools for the Energy Exascale Earth System Model (E3SM). It is
+designed to streamline the analysis, visualization, and transformation of model
+output for scientists and developers.
+
+This environment bundles together a curated set of Python and compiled tools
+that work well across supported platforms, particularly on E3SM-managed
+high-performance computing (HPC) systems.
+
+## Key Features
+
+* Combines dozens of packages into one environment, eliminating setup friction.
+* Maintains consistency across HPC platforms (Anvil, Chrysalis, Compy, etc.).
+* Offers both Conda and Spack-installed components for MPI performance.
+* Fully open source and community-maintained via GitHub.
+
+## Common Use Cases
+
+* Diagnostics and evaluation (e.g., `e3sm_diags`, `MPAS-Analysis`)
+* CMIP output conversion (`e3sm_to_cmip`)
+* Time series generation, viewer creation, and archiving (`zppy`, `zstash`)
+* Domain generation and mesh visualization (`cime_gen_domain`, `mosaic`)
+
+## Supported Platforms
+
+E3SM-Unified is available on many E3SM-supported systems:
+
+* Andes
+* Anvil
+* Chrysalis
+* Compy
+* Dane
+* Frontier
+* Perlmutter
+* Polaris (ALCF)
+* Ruby
+
+It can also be installed on Linux or macOS laptops for limited use (see
+[Quickstart Guide](quickstart.md)). Windows is not supported.
+
+## Getting Help
+
+* [Quickstart Guide](quickstart.md)
+* GitHub: [E3SM-Unified repository](https://github.com/E3SM-Project/e3sm-unified)
+* Slack: `#e3sm-help-postproc`
+* Issues/questions: GitHub Issues or contact [[email protected]](mailto:[email protected])
diff --git a/docs/logo/e3sm_unified_logo.png b/docs/logo/e3sm_unified_logo.png
diff --git a/docs/logo/e3sm_unified_logo_200.png b/docs/logo/e3sm_unified_logo_200.png
diff --git a/docs/packages.md b/docs/packages.md
diff --git a/docs/quickstart.md b/docs/quickstart.md
@@ -0,0 +1,98 @@
+# Quickstart Guide
+
+```{note}
+E3SM-Unified is supported only on Linux, OSX and HPC platforms. It is **not**
+supported on Windows.
+```
+
+## Accessing E3SM-Unified on Supported Machines
+
+On most E3SM-supported HPC systems, E3SM-Unified is already installed and
+ready to use via an activation script.
+
+### Example Activation Commands
+
+```bash
+# Andes
+source /ccs/proj/cli115/software/e3sm-unified/load_latest_e3sm_unified_andes.sh
+
+# Anvil
+source /lcrc/soft/climate/e3sm-unified/load_latest_e3sm_unified_anvil.sh
+
+# Chrysalis
+source /lcrc/soft/climate/e3sm-unified/load_latest_e3sm_unified_chrysalis.sh
+
+# Compy
+source /share/apps/E3SM/conda_envs/load_latest_e3sm_unified_compy.sh
+
+# Dane
+source /usr/workspace/e3sm/apps/e3sm-unified/load_latest_e3sm_unified_dane.sh
+
+# Frontier
+source /ccs/proj/cli115/software/e3sm-unified/load_latest_e3sm_unified_frontier.sh
+
+# Perlmutter
+source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh
+
+# Polaris (ALCF)
+source /lus/grand/projects/E3SMinput/soft/e3sm-unified/load_latest_e3sm_unified_polaris.sh
+
+# Ruby
+source /usr/workspace/e3sm/apps/e3sm-unified/load_latest_e3sm_unified_ruby.sh
+```
+
+Once the script is sourced, you'll have access to all the tools provided by
+E3SM-Unified in your environment.
+
+## Verifying Installation
+
+After activation, you can verify that the environment is correctly loaded by
+testing if major packages are importable:
+
+```python
+python -c "import xarray, e3sm_diags, mpas_analysis, zppy"
+```
+
+## Running on Compute Nodes (Optional but Recommended)
+
+Many E3SM-Unified tools (e.g., MOAB, MPAS-Analysis, NCO, TempestRemap) benefit
+from running on compute nodes using MPI-enabled system builds.
+
+Check your system documentation for how to launch interactive compute sessions
+(e.g., `srun`, `salloc`, or `qsub`).
+
+## Installing E3SM-Unified on an Unsupported System
+
+E3SM-Unified is not officially supported on Linux or Mac laptops or
+workstations, but users can install it using `miniforge3`.
+
+### Step-by-Step (Linux/macOS):
+
+```bash
+# Install miniforge3
+wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
+bash "Miniforge3-$(uname)-$(uname -m).sh"
+
+# Create a new environment
+conda create -n esm-unified -c conda-forge e3sm-unified
+
+# Activate it
+conda activate e3sm-unified
+```
+
+Note: On macOS with M1/M2 chips, install the x86\_64 version and use Rosetta 2
+for compatibility.
+
+---
+
+## Related Pages
+
+* [Introduction to E3SM-Unified](introduction.md)
+* [Using E3SM-Unified Tools](using-tools.md)
+* [Troubleshooting](troubleshooting.md)
+
+```{admonition} Need Help?
+- Slack: #e3sm-help-postproc
+- GitHub Issues: https://github.com/E3SM-Project/e3sm-unified/issues
+- Email: [email protected]
+```
diff --git a/docs/releasing/adding-new-machines.md b/docs/releasing/adding-new-machines.md
@@ -0,0 +1,91 @@
+# Adding a New Machine
+
+Support for a new HPC machine in E3SM-Unified requires coordinated updates
+across multiple tools — primarily in
+[`mache`](https://github.com/E3SM-Project/mache), but also in the E3SM Spack
+fork and deployment scripts.
+
+This page provides guidance for E3SM-Unified maintainers and infrastructure
+developers integrating new machines into the release and deployment workflow.
+
+---
+
+## 🔗 Main Mache Documentation
+
+Most of the process is already documented in the official `mache` developer
+guide:
+
+* [Adding a New Machine](https://docs.e3sm.org/mache/main/developers_guide/adding_new_machine.html)
+* [Adding Spack Support](https://docs.e3sm.org/mache/main/developers_guide/spack.html)
+
+Start in `mache` to:
+
+* Add a machine-specific config file (e.g., `pm-cpu.cfg`)
+* Add hostname detection logic in `discover.py`
+* Create Spack templates for supported compiler/MPI stacks
+* Optionally add shell script templates for environment setup
+
+> ⚠️ Machines not listed in the E3SM
+  [`config_machines.xml`](https://github.com/E3SM-Project/E3SM/blob/master/cime_config/machines/config_machines.xml) must first be added upstream before `mache`
+  can support them.
+
+---
+
+## 🧩 Integration with E3SM-Unified Deployment
+
+After updating `mache`, you'll need to:
+
+1. **Reference your `mache` branch in E3SM-Unified Deployment**
+
+   * Use the `--mache_fork` and `--mache_branch` flags to deploy using the
+     updated branch
+   * Confirm the new machine is recognized and templates are applied correctly
+
+2. **Update Spack if needed**
+
+   * If new versions of external tools are required, update the
+     [`spack_for_mache_<version>`](spack-updates.md) branch of the
+     [E3SM Spack fork](https://github.com/E3SM-Project/spack)
+
+---
+
+## ✅ Testing Your Changes
+
+Use the standard test deployment approach from
+[Deploying on HPCs](deploying-on-hpcs.md):
+
+```bash
+cd e3sm_supported_machines
+./deploy_e3sm_unified.py --conda ~/miniforge3 \
+                         --mache_fork <your_fork> \
+                         --mache_branch <your_branch>
+```
+You can also supply these flags:
+```
+                         --machine <new_machine> \
+                         --compiler <compiler> \
+                         --mpi <mpi> \
+```
+but they should not be needed if you have set things up in `mache` correctly.
+
+During testing, focus on:
+
+* Spack external package detection and successful builds
+* Shell script generation and activation behavior
+* Module compatibility and performance of tools like `zppy` and `e3sm_diags`
+
+---
+
+## 💡 Tips and Best Practices
+
+* Reuse YAML templates from similar machines to minimize effort
+* Add common system tools as `buildable: false` in the Spack environment
+* Avoid identifying machines using environment variables unless absolutely
+  necessary.  Instead use the hostnames for login and compute nodes if
+  possible
+* Use `utils/update_cime_machine_config.py` to verify `mache` remains in sync
+  with E3SM
+
+---
+
+➡ Next: [Publishing the Final Release](publishing-final-release.md)