Documentation update Pt. 1 (#290)

* switch hera to ursa

* update Introduction

* change LANDDAROOT to BASEDIR

* update stack/jedi dependencies in tech overview

* general build/run updates

* add draft JCB info

* add glossary terms

* Tech Overview edits

* update build/run sections

* update background info

* testing ch updates

* reorganize JCB info

* add curly braces around BASEDIR

* doc bugfix

* Rm JCB from chapter list

* Rm comment re: failing WE2Es

* fix typo

Author: gspetro-NOAA

.readthedocs.yaml

@@ -7,13 +7,16 @@ version: 2
 
 # Set the version of Python and other tools you might need
 build:
-  os: ubuntu-20.04
+  os: ubuntu-22.04
   tools:
-    python: "3.9"
+    python: "3.12"
     # You can also specify other tool versions:
     # nodejs: "16"
     # rust: "1.55"
     # golang: "1.17"
+  jobs:
+    post_create_environment:
+      - python -m pip install --upgrade "pip" "setuptools<81"
 
 # Build documentation in the docs/ directory with Sphinx
 sphinx:

doc/requirements.in

@@ -1,8 +1,9 @@
-sphinx>=7.4.7
-sphinx_rtd_theme
-sphinxcontrib-bibtex
+sphinx>=8.0.0
+sphinx_rtd_theme>=3.0.0
+sphinxcontrib-bibtex>=2.6.4
 urllib3>=1.26.19
 Jinja2>=3.1.6
-requests>=2.32.0
+requests>=2.32.2
 idna>=3.7
 certifi>=2024.7.4
+setuptools<81.0.0

doc/requirements.txt

@@ -2,7 +2,7 @@
 # This file is autogenerated by pip-compile with Python 3.13
 # by the following command:
 #
-#    pip-compile requirements.in
+#    pip-compile --allow-unsafe --strip-extras requirements.in
 #
 alabaster==0.7.16
     # via sphinx
@@ -56,17 +56,17 @@ six==1.16.0
     #   pybtex
 snowballstemmer==2.2.0
     # via sphinx
-sphinx==7.4.7
+sphinx==8.0.2
     # via
     #   -r requirements.in
     #   sphinx-rtd-theme
     #   sphinxcontrib-bibtex
     #   sphinxcontrib-jquery
-sphinx-rtd-theme==2.0.0
+sphinx-rtd-theme==3.1.0
     # via -r requirements.in
 sphinxcontrib-applehelp==1.0.8
     # via sphinx
-sphinxcontrib-bibtex==2.6.2
+sphinxcontrib-bibtex==2.6.4
     # via -r requirements.in
 sphinxcontrib-devhelp==1.0.6
     # via sphinx
@@ -84,3 +84,9 @@ urllib3==2.4.0
     # via
     #   -r requirements.in
     #   requests
+
+# The following packages are considered to be unsafe in a requirements file:
+setuptools==80.10.2
+    # via
+    #   -r requirements.in
+    #   sphinxcontrib-bibtex

doc/source/BackgroundInfo/Components.rst

@@ -66,20 +66,22 @@ The UFS Weather Model now contains a Noah-MP land component, which is used as th
 Data Assimilation
 ===================
 
-The Joint Effort for Data assimilation Integration (:term:`JEDI`) is a unified and versatile :term:`data assimilation` (DA) system for Earth system prediction that can be run on a variety of platforms. In the Land DA System, JEDI software can be used to perform snow data assimilation using :term:`GHCN`, :term:`IMS`, or :term:`SFCSNO` data with :term:`LETKF-OI` or :term:`3D-Var` algorithms. JEDI is developed by the Joint Center for Satellite Data Assimilation (`JCSDA <https://www.jcsda.org/>`_) and partner agencies, including NOAA. 
+The Joint Effort for Data assimilation Integration (:term:`JEDI`) is a unified and versatile :term:`data assimilation` (DA) system for Earth system prediction that can be run on a variety of platforms. JEDI is developed by the Joint Center for Satellite Data Assimilation (`JCSDA <https://www.jcsda.org/>`_) and partner agencies, including NOAA. In the Land DA System, JEDI software can be used to perform snow data assimilation using :term:`GHCN`, :term:`IMS`, or :term:`SFCSNO` data with :term:`LETKF-OI` or :term:`3D-Var` algorithms. Recently, the Land DA System added the capability to perform soil moisture data assimilation via JEDI with :term:`SMAP` or :term:`SMOPS` data. 
 
-The Land DA System uses three main JEDI components: 
+The Land DA System uses several JEDI components, including: 
 
    * The Object-Oriented Prediction System (:jedi:`OOPS <inside/jedi-components/oops/index.html>`) for the data assimilation algorithm 
    * The Interface for Observation Data Access (:jedi:`IODA <inside/jedi-components/ioda/index.html>`) for the observation formatting and processing
    * The Unified Forward Operator (:jedi:`UFO <inside/jedi-components/ufo/index.html>`) for comparing model forecasts and observations 
+   * The System Agnostic Background Error Representation (:jedi:`SABER <inside/jedi-components/saber/index.html>`) for computing and manipulating the background error covariance matrix
+   * The VAriable DErivation Repository (:jedi:`VADER <inside/jedi-components/vader/index.html>`) for producing new variables from known variables
 
 These three components (and others) are conveniently packaged and provided via JCSDA's :term:`jedi-bundle`. Users must build/install ``jedi-bundle`` prior to using the Land DA System if they are *not* working on a :ref:`Level 1 <level1>` platform; ``jedi-bundle`` does *not* come packaged in the Land DA System. Users are encouraged to visit the :jedi:`JEDI Documentation <inside/jedi-components/index.html>` for more information. 
 
 apply_incr.fd
 ---------------
 
-The Land DA System's ``apply_incr`` submodule points to NOAA PSL's `land-apply_jedi_incr <https://github.com/NOAA-PSL/land-apply_jedi_incr>`_ repository. This repository contains code to add the DA increment generated by JEDI to the ``sfc_data`` restart file. A `DA increment <https://www.meted.ucar.edu/bom/mdata_assim/navmenu.php?tab=1&page=3-7-0&type=flash>`_, or analysis increment, is the difference between what the model predicted and what the DA objective analysis computes as the best fit between the model state and the observations. Replacing the model state directly with the objective analysis can cause artificial shocks that will propagate through the system, so DA systems require code that slowly nudges the model state towards the analysis by applying part of the increment at a time as the new forecast steps forward in time. 
+The Land DA System's ``apply_incr`` submodule points to NOAA PSL's `land-apply_jedi_incr <https://github.com/NOAA-PSL/land-apply_jedi_incr>`_ repository. This repository contains code to add the DA increment generated by JEDI to the ``sfc_data`` restart file. A `DA increment <https://www.meted.ucar.edu/bom/mdata_assim/navmenu.php?tab=1&page=3-7-0&type=flash>`_, or analysis increment, is the difference between what the model predicted and what the DA objective analysis computes as the best fit between the model state and the observations. Replacing the model state directly with the objective analysis can cause artificial shocks that will propagate through the system, so DA systems require code that slowly moves the model state towards the analysis by applying part of the increment at a time as the new forecast steps forward in time. 
 
 .. _preprocessing:
 
@@ -107,7 +109,7 @@ The Land DA System includes the ``calcfIMS.fd`` submodule, which points to the l
 IODA Converters
 -----------------
 
-The Land DA System accepts :term:`GHCN`, :term:`IMS`, or :term:`SFCSNO` data as input. The ``prep_data`` task then converts these data from their original format into the format needed by JEDI's :term:`UFO` and :term:`OOPS` components for data assimilation (see :ref:`Workflow Tasks <wflow-overview>`). The Interface for Observation Data Access (:term:`IODA`) is the component of :term:`JEDI` that handles data processing for the data assimilation system (see :ref:`DA Components <da-components>` for more). The ``land-DA_workfow/ush`` directory contains scripts (e.g., ``ghcn_snod2ioda.py`` and ``imsfv3_scf2ioda.py``) that convert :term:`GHCN` and :term:`IMS` data to a JEDI-formatted NetCDF file using IODA. The :ref:`calcfIMS <calcfims>` executable mentioned above is an intermediate converter that converts the raw ASCII files to NetCDF format before performing additional JEDI formatting. 
+The Land DA System accepts :term:`GHCN`, :term:`IMS`, or :term:`SFCSNO` data as input for snow DA and :term:`SMAP` or :term:`SMOPS` data as input for soil moisture DA. The ``prep_data`` task then converts these data from their original format into the format needed by JEDI's :term:`UFO` and :term:`OOPS` components for data assimilation (see :ref:`Workflow Tasks <wflow-overview>`). The Interface for Observation Data Access (:term:`IODA`) is the component of :term:`JEDI` that handles data processing for the data assimilation system (see :ref:`DA Components <da-components>` for more). The ``land-DA_workfow/ush`` directory contains scripts (e.g., ``ghcn_snod2ioda.py`` and ``imsfv3_scf2ioda.py``) that convert input data to a JEDI-formatted NetCDF file using IODA. The :ref:`calcfIMS <calcfims>` executable mentioned above is an intermediate converter that converts the raw ASCII files to NetCDF format before performing additional JEDI formatting. 
 
 .. _t2tc:
 
@@ -153,17 +155,17 @@ The Land DA System has a portable, CMake-based build system that packages togeth
 
 Additional libraries necessary for the Land DA System must be installed separately via :term:`spack-stack` and :term:`jedi-bundle` unless users are working on a :ref:`supported platform <prerequisites>` or using a container. Once built, users can generate a Rocoto-based workflow that will run each task in the proper sequence (see :numref:`Chapter %s <RocotoInfo>` or the `Rocoto documentation <https://github.com/christopherwharrop/rocoto/wiki/Documentation>`_ for more information on Rocoto and workflow management). The workflow makes use of several configuration tools: 
 
-* JEDI Configuration Builder
+* :ref:`JEDI Configuration Builder <JCBInfo>` (JCB)
 * Unified Workflow (UW) Tools
 
-The Land DA System allows users to configure various elements of the workflow. For example, users can modify the start and end cycles for the experiment, the cycling frequency, and the duration of each forecast. It also allows for configuration of other elements of the workflow, such as data assimilation algorithm. More information on configurable variables is available in :numref:`Section %s <ConfigWorkflow>`.
+The Land DA System allows users to configure various elements of the workflow. For example, users can modify the start and end cycles for the experiment, the cycling frequency, and the duration of each forecast. It also allows for configuration of other elements of the workflow, such as the data assimilation algorithm. More information on configurable variables is available in :numref:`Section %s <ConfigWorkflow>`.
 
 .. _jcb-component:
 
-JEDI Configuration Builder
-----------------------------
+JEDI Configuration Builder (JCB)
+---------------------------------
 
-The JEDI Configuration Builder (JCB) is a python package used to assemble information on :term:`JEDI` algorithms (e.g., letkf-oi, 3dvar) and data assimilation types (e.g., snow, marine, atmosphere) into one convenient YAML file for use in data assimilation applications. The `jcb-algorithms <https://github.com/NOAA-EPIC/jcb-algorithms>`_ repository contains YAML algorithm files (e.g., LETKF-OI, 3DVar) for JCB; these files contain the high-level configuration structure that is prescribed by the JEDI data assimilation system. The `jcb-gdas <https://github.com/NOAA-EPIC/jcb-gdas>`_ repository contains information for different types of analysis (e.g., snow, marine, atmosphere).  
+The JEDI Configuration Builder (JCB) is a python package used to assemble information on :term:`JEDI` algorithms (e.g., ``letkf-oi``, ``3dvar``) and data assimilation types (e.g., snow, marine, atmosphere) into one convenient YAML file for use in data assimilation applications. The `jcb-algorithms <https://github.com/NOAA-EPIC/jcb-algorithms>`_ repository contains YAML algorithm files (e.g., LETKF-OI, 3DVar) for JCB; these files contain the high-level configuration structure that is prescribed by the JEDI data assimilation system. The `jcb-gdas <https://github.com/NOAA-EPIC/jcb-gdas>`_ repository contains information for different types of analysis (e.g., snow, marine, atmosphere). For more information, see :numref:`Section %s: JCB <JCBInfo>`.
 
 
 Unified Workflow (UW) Tools (``uwtools``)