Merge pull request #224 from EC-USGS/v0.2.1

Release 0.2.1

Author: jmccreight

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,6 +1,8 @@
 <!-- Feel free to remove check-list items aren't relevant to your change -->
 
 - [ ] Closes #xxxx
-- [ ] Tests and/or performance benchmarks added
+- [ ] Tests added
+- [ ] Performance benchmarks added
+- [ ] Performance regression benchmarks run
 - [ ] User visible changes (including notable bug fixes) are documented in `whats-new.rst`
 - [ ] New functions/methods are listed in `api.rst` or it's sub rsts?

.github/RELEASE.md

@@ -47,19 +47,21 @@ To release a new version:
    remote: `git push -u upstream vx.y.z`. This starts a job to:
 
     - Check out the release branch Update version number in `version.txt` and
-    - `pywatershed/version.py` to match the version in the branch name Build and
-    - check the Python package Generate a changelog since the last release
-    - Prepend the changelog to the cumulative `HISTORY.md` Upload the package
-    - and changelog as artifacts Draft a PR against `main` with the updated
-    - version files and cumulative changelog. The cumulative `HISTORY.md` is
-    - version-controlled, release changelogs are not.
+      `pywatershed/version.py` to match the version in the branch name
+    - Build and check the Python package Generate a changelog since the last
+	  release
+    - Prepend the changelog to the cumulative `HISTORY.md`. The cumulative
+	  `HISTORY.md` is version-controlled, release changelogs are not.
+	- Upload the package and changelog as artifacts Draft a PR against `main`
+	  with the updated version files and cumulative changelog.
 
 1. On all platforms, pull the release from upstream and perform ASV performance
-	benchmarks against previous release , e.g., ``` asv continuous --verbose
-	--show-stderr --factor 1.3 previous_release this_release ``` Collect
-	performance reports from various machines into a single report and use `asv
-	publish` to generate the static webpages to be included with the release as
-	artifacts in that step below.
+	benchmarks against previous release , e.g., ```asv run --verbose
+	--show-stderr HASHFILE:pws_refs_for_asv.txt``` after editing the file to
+	contain the previous and current release. Collect performance results from
+	various machines into a single report and use `asv publish` to generate
+	the static webpages to be included with the release as artifacts in that
+	step below.
 
 1. Inspect the package and changelog. If they look good, merge the PR to `main`.
 

.github/workflows/release.yaml

@@ -193,7 +193,14 @@ jobs:
         run: |
           version=$(cat version.txt | xargs)
           title="pywatershed $version"
-          notes=$(cat "changelog/CHANGELOG_$version.md" | grep -v "### Version $version")
+
+          # don't fail if the changelog is empty
+          if [[ -s "changelog/CHANGELOG_$version.md" ]]; then
+            notes=$(cat "changelog/CHANGELOG_$version.md" | grep -v "### Version $version")
+          else
+            notes="No changes found, are recent commits conventional?"
+          fi
+          
           gh release create "$version" \
             --target main \
             --title "$title" \

README.md

@@ -38,56 +38,80 @@ The following motivations are taken from our [AGU poster from December
 which provides additional details on motivations, project status, and current
 directions of this project as of approximately January 2023.
 
-Goals of the USGS Enterprise Capacity (EC) project include: * A sustainable
-integrated, hydrologic modeling framework for the U.S. Geological Survey (USGS)
-* Interoperable modeling across the USGS, partner agencies, and academia
-
-Goals for EC Watershed Modeling: * Couple the Precipitation-Runoff Modeling
-System (PRMS, e.g. Regan et al, 2018)  with MODFLOW 6 (MF6, e.g. Langevin et al,
-2017) in a sustainable way * Redesign PRMS to be more modern and flexible *
-Prioritize process representations in the current National Hydrological Model
-(NHM) based on PRMS 5.2.1
-
-Prototype an EC watershed model: "pywatershed" * Redesign PRMS quickly in python
-* Couple to MF6 via BMI/XMI interface (Hughes et al, 2021; Hutton et al, 2020) *
-Establish a prototyping ground for EC codes that couples to the compiled
-framework: low cost proof of concepts (at the price of potentially less
-computational performance) * Enable process representation hypothesis testing *
-Use cutting-edge techniques and technologies to improve models * Machine
-learning, automatic differentiation * Address challenges of modeling across
-space and time scales * Transition prototype watershed model to compiled EC code
+Goals of the USGS Enterprise Capacity (EC) project include:
 
-## Installation
+  * A sustainable integrated, hydrologic modeling framework for the U.S.
+    Geological Survey (USGS)
+  * Interoperable modeling across the USGS, partner agencies, and academia
+
+Goals for EC Watershed Modeling:
 
-To install the software you will need Python 3.9 or 3.10.
+  * Couple the Precipitation-Runoff Modeling System (PRMS, e.g. Regan et al,
+	2018)  with MODFLOW 6 (MF6, e.g. Langevin et al, 2017) in a sustainable
+	way
+  * Redesign PRMS to be more modern and flexible
+  * Prioritize process representations in the current National Hydrological
+    Model (NHM) based on PRMS 5.2.1
 
-We currently recommend dependencies be installed with
-[Mamba](https://mamba.readthedocs.io/en/latest/) which will be much faster than
-Ananconda (but the conda command can also be used). An environment containing
-all core and optional dependencies can be created from the project root with:
+Prototype an EC watershed model: "pywatershed"
 
-```mamba env create -f environment_w_jupyter.yml```
+  * Redesign PRMS quickly in python
+  * Couple to MF6 via BMI/XMI interface (Hughes et al, 2021; Hutton et al, 2020)
+  * Establish a prototyping ground for EC codes that couples to the compiled
+	framework: low cost proof of concepts (at the price of potentially less
+    computational performance) * Enable process representation hypothesis testing
+  * Use cutting-edge techniques and technologies to improve models 
+  * Machine learning, automatic differentiation 
+  * Address challenges of modeling across space and time scales 
+  * Transition prototype watershed model to compiled EC code
 
-(The environment `environment.yml` does not contain jupyter or jupyterlab
-in order to not interfere with installation in WholeTale, see Example
-Notebooks seection below.)
+
+## Installation
+
+`pywatershed` uses Python 3.9 or 3.10.
 
 The `pywatershed` package is [available on
-PyPI](https://pypi.org/project/pywatershed/). At the moment, the installation
-may not be reliable on all platforms and we are working to fix this.
+PyPI](https://pypi.org/project/pywatershed/) but installation of dependencies
+may not be reliable on all platforms. 
 
-Using PyPI (with the above caveat), `pywatershed` can be installed with:
+We recommend dependencies be first installed with
+[Mamba](https://mamba.readthedocs.io/en/latest/). This will be much faster than
+Ananconda (but the conda command could also be used). We hope to provide
+`pywatershed` installation with all its dependencies on conda-forge in the 
+near future.
 
-``` pip install pywatershed ```
+If you wish to use the stable release, you will use `main` in place of 
+`<branch>` in the following commands. If you want to follow developemnt, you'll
+use `develop` instead.
 
-A number of extra dependencies are needed to run the example notebooks. These
-can be installed with pip with
+Without using `git` (directly), you may:
+```
+curl -L -O https://raw.githubusercontent.com/EC-USGS/pywatershed/<branch>/environment_w_jupyter.yml
+mamba env create -f environment_w_jupyter.yml
+conda activate pws
+pip install git+https://github.com/EC-USGS/pywatershed.git@<branch>
+```
 
-``` pip install "pywatershed[optional]" ```
+Or to use `git` and to be able to develop:
 
-These installation steps are suitable for `pywatershed` end users. See the
-[developer documentation](./DEVELOPER.md) for detailed instructions on
-configuring a development environment.
+```
+git clone https://github.com/EC-USGS/pywatershed.git
+cd pywatershed
+mamba env create -f environment_w_jupyter.yml
+activate pws
+pip install -e .
+```
+
+(If you want to name the environment other than the default `pws`, use the
+command 
+`mamba env update --name your_env_name --file environment_w_jupyter.yml --prune`
+you will also need to activate this environment by name.)
+
+
+We install the `environment_w_jupyter.yml` to provide all known dependencies 
+including those for running the eample notebooks. (The `environment.yml` 
+does not contain jupyter or jupyterlab because this interferes with installation
+on WholeTale, see Example Notebooks seection below.)
 
 ## Contributing
 
@@ -116,10 +140,11 @@ to register.
 There are containers for both the `main` and `develop` branches.
 
 [![WholeTale](https://raw.githubusercontent.com/whole-tale/wt-design-docs/master/badges/wholetale-explore.svg)](https://dashboard.wholetale.org)
-* [WholeTale container for latest release (main
-  branch)](https://dashboard.wholetale.org/run/64ae29e8a887f48b9f173678?tab=metadata)
-* [WholeTale container for develop
-  branch](https://dashboard.wholetale.org/run/64ae25c3a887f48b9f1735c8?tab=metadata)
+
+  * [WholeTale container for latest release (main
+	branch)](https://dashboard.wholetale.org/run/64ae29e8a887f48b9f173678?tab=metadata)
+  * [WholeTale container for develop
+	branch](https://dashboard.wholetale.org/run/64ae25c3a887f48b9f1735c8?tab=metadata)
 
 WholeTale will give you a jupyter-lab running in the root of this
 repository. You can navigate to `examples/` and then open and run the notebooks

asv_benchmarks/benchmarks/prms.py

@@ -85,17 +85,17 @@ def setup(self, *args):
         else:
             self.params = pws.PrmsParameters.load(self.parameter_file)
 
-        # backwards compatability
+        # backwards compatability pre 0.2.0
         try:
+            self.ge_v0_2_0 = False
             self.control = pws.Control.load(
                 self.control_file, params=self.params
             )
-            self.ge_v0_2_0 = False
         except:
-            self.control = pws.Control.load(self.control_file)
             self.ge_v0_2_0 = True
 
-        self.control.edit_n_time_steps(n_time_steps)
+        if hasattr(self, "control"):
+            del self.control
 
         # setup input_dir with symlinked prms inputs and outputs
         self.domain_dir = pl.Path(f"PRMSModels_{self.domain}")
@@ -119,17 +119,28 @@ def model_setup_run(
         processes: tuple = None,
         write_output: Union[bool, Literal["separate", "together"]] = None,
     ):
+        # seem to need to load control inside the model setup run bc
+        # results are strange/inconsistent
+
         if self.ge_v0_2_0:
+            self.control = pws.Control.load(self.control_file)
             self.control.options["input_dir"] = self.tag_input_dir
             self.control.options["budget_type"] = "warn"
             self.control.options["calc_method"] = "numba"
+            self.control.edit_n_time_steps(n_time_steps)
 
             model = pws.Model(
                 self.processes,
                 control=self.control,
                 parameters=self.params,
             )
+
         else:
+            self.control = pws.Control.load(
+                self.control_file, params=self.params
+            )
+            self.control.edit_n_time_steps(n_time_steps)
+
             model = pws.Model(
                 *self.processes,
                 control=self.control,
@@ -143,6 +154,8 @@ def model_setup_run(
                 self.tag_dir, separate_files=(write_output == "separate")
             )
         model.run(finalize=True)
+        del model
+        del self.control
 
     @parameterized(
         ["domain", "processes", "output"],

asv_benchmarks/pws_refs_for_asv.txt

@@ -0,0 +1,3 @@
+0.1.1
+0.2.0
+

doc/whats-new.rst

@@ -12,9 +12,22 @@ What's New
     np.random.seed(123456)
 
 
+.. _whats-new.0.2.1:
+
+v0.2.1 (19 July 2023)
+---------------------
+
+Bug fixes
+~~~~~~~~~
+- Package data was not properly installed.
+  (:pull:`219`) By `James McCreight <https://github.com/jmccreight>`_.
+- Small addition to notebook 02
+  (:pull:`219`) By `James McCreight <https://github.com/jmccreight>`_.
+
+
 .. _whats-new.0.2.0:
 
-v0.2.0 (12 July 2023)
+v0.2.0 (18 July 2023)
 ---------------------
 
 New features

environment.yml

@@ -1,4 +1,4 @@
-name: pywatershed
+name: pws
 channels:
   - conda-forge
   - nodefaults
@@ -11,12 +11,11 @@ dependencies:
   - filelock
   - geopandas
   - geoviews
+  - git
   - holoviews
   - hvplot
-  - git
   - ipython
   - ipywidgets
-  - jupyter_contrib_nbextensions
   - nb_conda_kernels
   - nbconvert
   - netCDF4
@@ -43,11 +42,12 @@ dependencies:
   - tqdm
   - xarray>=2023.05.0
   - pip:
-      - click != 8.1.0
+      - asv
       - black
+      - click != 8.1.0
       - isort
       - flake8
-      - jupyter_black
-      - pylint
       - git+https://github.com/modflowpy/flopy.git
+      - jupyter_black
       - modflow-devtools
+      - pylint

environment_w_jupyter.yml

@@ -1,4 +1,4 @@
-name: pywatershed
+name: pws
 channels:
   - conda-forge
   - nodefaults
@@ -11,13 +11,12 @@ dependencies:
   - filelock
   - geopandas
   - geoviews
+  - git
   - holoviews
   - hvplot
-  - git
   - ipython
   - ipywidgets
   - jupyter
-  - jupyter_contrib_nbextensions
   - jupyterlab
   - nb_conda_kernels
   - nbconvert
@@ -45,11 +44,12 @@ dependencies:
   - tqdm
   - xarray>=2023.05.0
   - pip:
-      - click != 8.1.0
+      - asv
       - black
+      - click != 8.1.0
       - isort
       - flake8
-      - jupyter_black
-      - pylint
       - git+https://github.com/modflowpy/flopy.git
+      - jupyter_black
       - modflow-devtools
+      - pylint

examples/02_prms_legacy_models.ipynb

@@ -410,8 +410,10 @@
     "    \"input_dir\": run_dir,\n",
     "    \"budget_type\": \"warn\",\n",
     "    \"calc_method\": \"numba\",\n",
-    "    # \"netcdf_output_dir\": nb_output_dir / \"nhm\",\n",
-    "}"
+    "    \"netcdf_output_dir\": nb_output_dir / \"nhm_submodel\",\n",
+    "}\n",
+    "\n",
+    "control.options[\"netcdf_output_var_names\"] = pws.PRMSChannel.get_variables()"
    ]
   },
   {

pyproject.toml

@@ -110,7 +110,7 @@ include = ["pywatershed", "pywatershed.*"]
 
 [tool.setuptools.package-data]
 "pywatershed.static.metadata" = ["*.yaml"]
-"pywatershed.data" = ["*.nc"]
+"pywatershed.data.drb_2yr" = ["*"]
 
 [tool.black]
 line-length = 79

pywatershed/version.py

@@ -1,6 +1,6 @@
-# pywatershed version file automatically created using update_version.py on July 18, 2023 16:50:45
+# pywatershed version file automatically created using update_version.py on July 21, 2023 02:12:07
 
-__version__ = "0.2.0"
+__version__ = "0.2.1"
 __pakname__ = "pywatershed"
 
 author_dict = {

version.txt

@@ -1 +1 @@
-0.2.0
+0.2.1