From 785a9448798879dcdb7ec5431437194520207cee Mon Sep 17 00:00:00 2001 From: Xylar Asay-Davis Date: Sun, 20 Jul 2025 17:40:12 +0200 Subject: [PATCH 1/2] Fix various links This merge also makes warnings report as errors when building the docs and removes references to the autogenerated API. Two unused pages have been removed. --- docs/Makefile | 2 +- docs/conf.py | 2 - docs/index.md | 17 ++++- docs/releasing/adding-new-machines.md | 6 +- docs/releasing/maintaining-past-versions.md | 2 +- docs/releasing/planning-updates.md | 2 +- docs/testing-release-candidates.md | 0 docs/using-toos.md | 74 --------------------- 8 files changed, 22 insertions(+), 83 deletions(-) delete mode 100644 docs/testing-release-candidates.md delete mode 100644 docs/using-toos.md diff --git a/docs/Makefile b/docs/Makefile index 38017940..e30ab6f2 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -11,7 +11,7 @@ BUILDDIR = _build # Build into a versioned subdirectory versioned-html: @echo "Building version: $(DOCS_VERSION)" - $(SPHINXBUILD) -b html "$(SOURCEDIR)" "$(BUILDDIR)/html/$(DOCS_VERSION)" + $(SPHINXBUILD) -b html -nW --keep-going "$(SOURCEDIR)" "$(BUILDDIR)/html/$(DOCS_VERSION)" @echo "Build finished. The HTML pages are in $(BUILDDIR)/html/$(DOCS_VERSION)." @echo "Setting up shared version switcher for local preview..." mkdir -p _build/html/shared diff --git a/docs/conf.py b/docs/conf.py index f5d44d70..732f6e27 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -42,8 +42,6 @@ "sphinx.ext.napoleon", ] -autosummary_generate = ['developers_guide/api.md'] - # Otherwise, the Return parameter list looks different from the Parameters list napoleon_use_rtype = False # Otherwise, the Attributes parameter list looks different from the Parameters diff --git a/docs/index.md b/docs/index.md index 5f610635..985ec5f5 100644 --- a/docs/index.md +++ b/docs/index.md @@ -5,8 +5,23 @@ introduction quickstart using-tools -testing-release-candidates + releasing/release-workflow +releasing/conda-vs-spack +releasing/planning-updates +releasing/creating-rcs/overview +releasing/creating-rcs/rc-dependencies +releasing/creating-rcs/rc-e3sm-unified +releasing/creating-rcs/rc-troubleshooting +releasing/testing/overview +releasing/testing/spack-updates +releasing/testing/mache-updates +releasing/testing/deploying-on-hpcs +releasing/testing/troubleshooting-deploy +releasing/adding-new-machines +releasing/finalizing-release +releasing/maintaining-past-versions + packages troubleshooting contributing diff --git a/docs/releasing/adding-new-machines.md b/docs/releasing/adding-new-machines.md index 7fb75105..9eb2c6c8 100644 --- a/docs/releasing/adding-new-machines.md +++ b/docs/releasing/adding-new-machines.md @@ -44,7 +44,7 @@ After updating `mache`, you'll need to: 2. **Update Spack if needed** * If new versions of external tools are required, update the - [`spack_for_mache_`](spack-updates.md) branch of the + [`spack_for_mache_`](testing/spack-updates.md) branch of the [E3SM Spack fork](https://github.com/E3SM-Project/spack) --- @@ -52,7 +52,7 @@ After updating `mache`, you'll need to: ## ✅ Testing Your Changes Use the standard test deployment approach from -[Deploying on HPCs](deploying-on-hpcs.md): +[Deploying on HPCs](testing/deploying-on-hpcs.md): ```bash cd e3sm_supported_machines @@ -88,4 +88,4 @@ During testing, focus on: --- -➡ Next: [Publishing the Final Release](publishing-final-release.md) +➡ Next: [Publishing the Final Release](finalizing-release.md) diff --git a/docs/releasing/maintaining-past-versions.md b/docs/releasing/maintaining-past-versions.md index 2082997b..8ea0dbc0 100644 --- a/docs/releasing/maintaining-past-versions.md +++ b/docs/releasing/maintaining-past-versions.md @@ -95,4 +95,4 @@ because of this. --- -Back to: [Publishing the Final Release](publishing-final-release.md) +Back to: [Publishing the Final Release](finalizing-release.md) diff --git a/docs/releasing/planning-updates.md b/docs/releasing/planning-updates.md index f5dab30e..1bf6a206 100644 --- a/docs/releasing/planning-updates.md +++ b/docs/releasing/planning-updates.md @@ -69,7 +69,7 @@ The final list of packages is curated by the Infrastructure Team based on: * Community need and usage patterns Once the list is mostly settled, the team begins creating release candidates -using the [Creating Release Candidates](creating-rcs.md) workflow. +using the [Creating Release Candidates](creating-rcs/overview.md) workflow. --- diff --git a/docs/testing-release-candidates.md b/docs/testing-release-candidates.md deleted file mode 100644 index e69de29b..00000000 diff --git a/docs/using-toos.md b/docs/using-toos.md deleted file mode 100644 index cd4bde49..00000000 --- a/docs/using-toos.md +++ /dev/null @@ -1,74 +0,0 @@ -# Using E3SM-Unified Tools - -This section provides usage tips and examples for the most common tools -included in the E3SM-Unified environment. It is aimed at both new and -experienced users who want to understand how to apply these tools for analysis, -diagnostics, and processing tasks. - ---- - -## MPAS-Analysis - -> TODO: Add examples for configuring and running MPAS-Analysis, and interpreting output. - -* Description -* Typical use cases -* Running on HPC compute nodes -* Tips for customizing config files - ---- - -## E3SM Diags (`e3sm_diags`) - -> TODO: Add instructions and sample runs for generating diagnostics from E3SM output. - -* Description -* Example input/output -* Viewer and web-ready outputs - ---- - -## e3sm\_to\_cmip - -> TODO: Describe how to use this tool to convert E3SM output to CMIP-compliant NetCDF. - -* Key arguments and configurations -* Output structure -* Known caveats and workarounds - ---- - -## zppy & zppy-interfaces - -> TODO: Guide users through setting up and running zppy workflows. - -* YAML config examples -* Common tasks (`ts`, `e3sm_to_cmip`, `tc_analysis`, etc.) -* New features in zppy-interfaces - ---- - -## zstash - -> TODO: Describe how to archive and retrieve data efficiently using zstash. - -* Parallel archiving best practices -* Example usage -* Non-blocking mode with Globus - ---- - -## Additional Tools - -> TODO: Brief overviews or links to additional packages: - -* `nco`, `cdo` -* `cime_gen_domain`, `mosaic`, `livvkit` -* `tempest-remap`, `moab`, `geometric_features` - ---- - -## Related Pages - -* [Quickstart Guide](quickstart.md) -* [Package Catalog](packages.md) From 54b84f10de1edfa1f5dae10a6f8fc6642edf7eb0 Mon Sep 17 00:00:00 2001 From: Xylar Asay-Davis Date: Sun, 20 Jul 2025 19:08:00 +0200 Subject: [PATCH 2/2] Fix table of contents and simplify package catalog This merge adds `index.md` files in each subdirectory of the docs to help ensure that the table of contents is laid out intuitively. The table providing the package catalog has been updated so that different versions of dependencies for different python versions are split into multiple rows for readability. --- docs/index.md | 20 ++------------------ docs/packages.md | 20 +++++++++++++++----- docs/releasing/creating-rcs/index.md | 19 +++++++++++++++++++ docs/releasing/creating-rcs/overview.md | 13 +------------ docs/releasing/index.md | 22 ++++++++++++++++++++++ docs/releasing/release-workflow.md | 16 +++------------- docs/releasing/testing/index.md | 16 ++++++++++++++++ docs/releasing/testing/overview.md | 7 +------ 8 files changed, 79 insertions(+), 54 deletions(-) create mode 100644 docs/releasing/creating-rcs/index.md create mode 100644 docs/releasing/index.md create mode 100644 docs/releasing/testing/index.md diff --git a/docs/index.md b/docs/index.md index 985ec5f5..4ab4721f 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,27 +1,11 @@ ```{toctree} :hidden: -:maxdepth: 3 +:maxdepth: 1 introduction quickstart using-tools - -releasing/release-workflow -releasing/conda-vs-spack -releasing/planning-updates -releasing/creating-rcs/overview -releasing/creating-rcs/rc-dependencies -releasing/creating-rcs/rc-e3sm-unified -releasing/creating-rcs/rc-troubleshooting -releasing/testing/overview -releasing/testing/spack-updates -releasing/testing/mache-updates -releasing/testing/deploying-on-hpcs -releasing/testing/troubleshooting-deploy -releasing/adding-new-machines -releasing/finalizing-release -releasing/maintaining-past-versions - +releasing/index packages troubleshooting contributing diff --git a/docs/packages.md b/docs/packages.md index 22b5e7c4..5edce96e 100644 --- a/docs/packages.md +++ b/docs/packages.md @@ -1,4 +1,4 @@ -# 📦 E3SM-Unified Package Catalog +# Package Catalog This page provides a complete inventory of packages included in the latest E3SM-Unified release (`1.11.1`), along with their versions. These packages @@ -9,6 +9,12 @@ for what is installed in the environment. ## Main Packages and Dependencies +Empty cells under "Version Constraint" indicate that there is no constraint +on that package. Blank cells under "Package Name" indicate the same package +as above. Some package are forced to have different versions for different +python versions (typically when the dependency has dropped support for a python +version that we wish to support). + | Package Name | Version Constraint | |----------------------|-----------------------------| | blas | | @@ -23,7 +29,8 @@ for what is installed in the environment. | cime_gen_domain | 6.1.59 | | cmocean | | | cython | | -| dask | 2024.8.0 (py<3.10), 2024.11.2 (py>=3.10) | +| dask | 2024.8.0 (py<3.10) | +| | 2024.11.2 (py>=3.10) | | dogpile.cache | | | e3sm_diags | 3.0.0 | | e3sm-tools | 3.0.2 | @@ -74,7 +81,8 @@ for what is installed in the environment. | progessbar2 | | | proj | 9.5.1 | | psutil | | -| pyproj | 3.6.1 (py<3.10), 3.7.0 (py>=3.10) | +| pyproj | 3.6.1 (py<3.10) | +| | 3.7.0 (py>=3.10) | | pyevtk | | | pyremap | | | pytest | | @@ -91,9 +99,11 @@ for what is installed in the environment. | tabulate | | | tempest-extremes | 2.2.3 | | tempest-remap | 2.2.0 | -| uxarray | 2024.11.1 (py<3.10), >=2024.12.0 (py>=3.10) | +| uxarray | 2024.11.1 (py<3.10) | +| | >=2024.12.0 (py>=3.10) | | windspharm | | -| xarray | 2024.7.0 (py<3.10), 2025.1.1 (py>=3.10) | +| xarray | 2024.7.0 (py<3.10) | +| | 2025.1.1 (py>=3.10) | | xcdat | 0.8.0 | | xesmf | 0.8.8 | | zppy | 3.0.0 | diff --git a/docs/releasing/creating-rcs/index.md b/docs/releasing/creating-rcs/index.md new file mode 100644 index 00000000..bee5b451 --- /dev/null +++ b/docs/releasing/creating-rcs/index.md @@ -0,0 +1,19 @@ +# Creating Release Candidates + +E3SM-Unified and its core dependencies follow a structured release process +that relies on **release candidates (RCs)**. These pre-release versions are +used for testing and validation before an official release is finalized and +deployed. + +This section describes how to create RCs for both individual dependencies +(like `e3sm_diags` or `mpas-analysis`) and for the `e3sm-unified` metapackage +itself. It also includes tools and tips for troubleshooting build failures. + +```{toctree} +:maxdepth: 1 + +overview +rc-dependencies +rc-e3sm-unified +rc-troubleshooting +``` diff --git a/docs/releasing/creating-rcs/overview.md b/docs/releasing/creating-rcs/overview.md index e82eb322..52817a33 100644 --- a/docs/releasing/creating-rcs/overview.md +++ b/docs/releasing/creating-rcs/overview.md @@ -1,15 +1,4 @@ -# Creating Release Candidates - -E3SM-Unified and its core dependencies follow a structured release process -that relies on **release candidates (RCs)**. These pre-release versions are -used for testing and validation before an official release is finalized and -deployed. - -This section describes how to create RCs for both individual dependencies -(like `e3sm_diags` or `mpas-analysis`) and for the `e3sm-unified` metapackage -itself. It also includes tools and tips for troubleshooting build failures. - ---- +# Overview of Release Candidate Workflows ## What Is a Release Candidate? diff --git a/docs/releasing/index.md b/docs/releasing/index.md new file mode 100644 index 00000000..36c0e123 --- /dev/null +++ b/docs/releasing/index.md @@ -0,0 +1,22 @@ +# Release Workflow + +Releasing a new version of E3SM-Unified is an iterative, collaborative process +involving Conda, Spack, and coordinated deployment across HPC systems. This +guide serves as a roadmap for each stage in the workflow. + +Whether you're updating packages, building release candidates, or testing on +HPC platforms, this section documents the steps needed to bring a new version +of E3SM-Unified from planning to full deployment. + +```{toctree} +:maxdepth: 1 + +release-workflow +conda-vs-spack +planning-updates +creating-rcs/index +testing/index +adding-new-machines +finalizing-release +maintaining-past-versions +``` \ No newline at end of file diff --git a/docs/releasing/release-workflow.md b/docs/releasing/release-workflow.md index d659d9c3..ec4f37fc 100644 --- a/docs/releasing/release-workflow.md +++ b/docs/releasing/release-workflow.md @@ -1,16 +1,4 @@ -# The E3SM-Unified Release Workflow - -Releasing a new version of E3SM-Unified is an iterative, collaborative process -involving Conda, Spack, and coordinated deployment across HPC systems. This -guide serves as a roadmap for each stage in the workflow. - -Whether you're updating packages, building release candidates, or testing on -HPC platforms, this section documents the steps needed to bring a new version -of E3SM-Unified from planning to full deployment. - ---- - -## Overview of the Workflow +# Overview of the Workflow The release process typically follows this progression: @@ -18,7 +6,9 @@ The release process typically follows this progression: 2. **[Planning Package Updates](planning-updates.md)** 3. **[Creating Release Candidates](creating-rcs/overview.md)** 4. **[Deployment and Testing](testing/overview.md)** +5. **[Adding a New Machine](adding-new-machines.md)** 6. **[Finalizing the Release](finalizing-release.md)** +7. **[Maintaining Past Versionse](maintaining-past-versions.md)** Each of these steps is detailed in its own page. See below for a high-level summary. diff --git a/docs/releasing/testing/index.md b/docs/releasing/testing/index.md new file mode 100644 index 00000000..618ba12f --- /dev/null +++ b/docs/releasing/testing/index.md @@ -0,0 +1,16 @@ +# Deployment and Testing + +Once a release candidate (RC) of E3SM-Unified has been successfully built, it +must be thoroughly tested across supported HPC systems before a full release +can occur. This phase ensures compatibility with system modules, +performance-critical tools, and real-world analysis workflows. + +```{toctree} +:maxdepth: 1 + +overview +spack-updates +mache-updates +deploying-on-hpcs +troubleshooting-deploy +``` diff --git a/docs/releasing/testing/overview.md b/docs/releasing/testing/overview.md index ae5bdf49..705362b4 100644 --- a/docs/releasing/testing/overview.md +++ b/docs/releasing/testing/overview.md @@ -1,9 +1,4 @@ -# Deployment and Testing Overview - -Once a release candidate (RC) of E3SM-Unified has been successfully built, it -must be thoroughly tested across supported HPC systems before a full release -can occur. This phase ensures compatibility with system modules, -performance-critical tools, and real-world analysis workflows. +# Overview of Deployment and Testing This section documents the full testing and deployment process, including how to: