Add EPANETMSX support to WNTR (#462)

* initial creation of MSX reaction object

The class will be similar to the WaterNetworkModel in terms of its
structure and options.

* update requirements so that reactions test with sympy

* Importing module before it existed, oops

* Documentation updates

* Code updates for MSX utilities

* Forgot some files

* Updated code structure checkin

* Update to core reactions model codes

* Updates to read things in - there are bugs

* Updates to documentation and completing the I/O

* Completed MSX-input file IO

* Citations update

* Adding a (very basic) citations class

* Update documentation

* Updates after internal discussion

* Update to read/write citations on the WaterNetworkModel

* Documentation updates

* Update to add basic citation class

* Getting rid of merge conflict

* Exceptions (#133)

* Update to exception documentation

* Update documentation for EN errors

* Update to sphinx config (#134)

* Updates to toolkit

* Rebase library

* Updates to documentation, organization, and utilities

* Rename

* Typo

* Updates to tests and organization

* Updates for testing and documentation

* Update to documentation and library functionality

* Updates pulling MSX into the WaterNetworkModel

* Added as a part of the WaterNetworkModel named "msx"
* Made names consistent with wntr.network namings

* Stashing

* Update to package structure

* Tests still broken, but now fully running MSX

* Renaming to conform to other wntr styles

* Updates to naming, documentation

* Delete test file

* Add test file

* Updates to documentation for MSX

* Update to example files and tests

* Test updates

* Fix tolerances

* MSX documentation update

* minor updates

* MSX docs update (#135)

* Add scheduled testing to workflow

* MSX documentation update

* minor updates

---------

Co-authored-by: Kirk Bonney <[email protected]>

* API docs update, removed excess imports, removed cite_msx

* API docs update, removed excess imports, removed cite_msx (#136)

* Add scheduled testing to workflow

* MSX documentation update

* minor updates

* API docs update, removed excess imports, removed cite_msx

---------

Co-authored-by: Kirk Bonney <[email protected]>

* Docs update

* Fix changed name

* Update to correct the exceptions listings

* fix exceptions

* Updates to address comments in PR. Tests forthcoming

* Added the tests for exceptions

* Get rid of apidoc to match docs-theme

* Exceptions/docs-theme merges

* Final conflict

* Updates to msx docs

* Fixing typos

* Adding MSX binary files and updating .gitignore

* Update dll paths

* Updating tests...ish

* MSX binaries update

* Updates to documentation configuration

* Fix toolkit problems.

* Updated MSX binaries, PR issued to EPANETMSX main
* Updated toolkit to match modified code in MSX DLL
* Fixed an error in the filename encoding

* Updates to fix the wn.msx variable assignment

* JSON schema commit

* Update to MSX examples and bug fix

* Update to MSX demo

* Update to the MSX demo jupyter notebook.

* Update to demo

* MSX updates

* Move library to new directory
* Updates to documentation
* New ipynb demo
* various bug fixes

* Reverting

* add msx simulation test

* Revert documentation config changes

* Revert conf.py

* Finish revert erroneous commits

* CrowdStrike mess backup

* This addresses comments made regarding the msx model files. Documentation is not pulled in this commit

* Update the epanetmsx binary builds from the USEPA official branch

* tests: correct filename in demo file, clean binary objects out of ipynb

* Updates to MSX documentation

* build: Update binaries with structure condusive to multi-platform Darwin builds

* chore: Load libraries in a different manner

* fix: misspelling in library name

* fix: misspelling in library name in regular wntr too

* fix: Update DLLs for EPANET

* debug: mark threaded test as skip temporarily.

All os/version combos are failing in the threading test, including using the old binaries. Trying to turn off the test first to see if it can get past this test.

* fix: found bug with v2.2 threading

* fix: updated the initial_quality to valid gis name

* Test build script update

* build: add libomp brew formula

* fix: missing comma breaking demos.

* docs: Update msx-library and string outputs in documentation

* fix: skip doctest ordered list

* test: Skip v2.0 on ARM processor

* test: missed one test that needs to be skipped on arm processor

* ci: Remove doctests for arm processor

* ci: update release to proper macos names

* test: remove spurrious print statement

* Rename/move the brew formula for EPANETMSX compatibility.

Also adds the ruby brew formula to the manifest for source distributions.

* Remove duplicated MSX example file.

---------

Co-authored-by: Katherine Klise <[email protected]>
Co-authored-by: Kirk Bonney <[email protected]>
Co-authored-by: kbonney <[email protected]>

Author: 3 people

.github/workflows/build_tests.yml

@@ -18,7 +18,7 @@ jobs:
     strategy:
       matrix:
         python-version: ['3.9', '3.10', '3.11', '3.12']
-        os: [windows-latest, macOS-13, ubuntu-latest]
+        os: [windows-latest, macOS-13, macos-latest, ubuntu-latest]
       fail-fast: false
     steps:
     - uses: actions/checkout@v4
@@ -48,7 +48,7 @@ jobs:
     strategy:
       matrix:
         python-version: ['3.9', '3.10', '3.11', '3.12']
-        os: [windows-latest, macOS-13, ubuntu-latest]
+        os: [windows-latest, macOS-13, macos-latest, ubuntu-latest]
       fail-fast: false
     steps:
     - name: Set up Python 
@@ -73,27 +73,39 @@ jobs:
     strategy:
       matrix:
         python-version: ['3.9', '3.10', '3.11', '3.12']
-        os: [windows-latest, macOS-13, ubuntu-latest]
+        os: [windows-latest, macOS-13, ubuntu-latest, macos-latest]
       fail-fast: false
     steps:
     - uses: actions/checkout@v4
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
+    - if: ${{ matrix.os == 'macos-latest' || matrix.os == 'macOS-13'}}
+      run: |
+        brew reinstall --build-from-source --formula wntr/epanet/libepanet/darwin-formula/libomp.rb
     - name: Install dependencies
       run: |
         python --version
         python -m pip install --upgrade pip
         pip install -r requirements.txt
         python -m pip install -e .
     - name: Run Tests
+      if: ${{ matrix.os != 'macos-latest' }}
       run: | 
         coverage erase
         coverage run --context=${{ matrix.os }}.py${{ matrix.python-version }} --source=wntr --omit="*/tests/*","*/sim/network_isolation/network_isolation.py","*/sim/aml/evaluator.py" -m pytest  --doctest-modules --doctest-glob="*.rst" wntr
         coverage run --context=${{ matrix.os }}.py${{ matrix.python-version }} --source=wntr --omit="*/tests/*","*/sim/network_isolation/network_isolation.py","*/sim/aml/evaluator.py" --append -m pytest --doctest-glob="*.rst" documentation
       env:
         COVERAGE_FILE: .coverage.${{ matrix.python-version }}.${{ matrix.os }}
+    - name: Run Tests (ARM-processor)
+      if: ${{ matrix.os == 'macos-latest'}}
+      # doctests are not flexible enough to skip EPANET=v2.0 errors on ARM processor, so do not run doctests on ARM system
+      run: | 
+        coverage erase
+        coverage run --context=${{ matrix.os }}.py${{ matrix.python-version }} --source=wntr --omit="*/tests/*","*/sim/network_isolation/network_isolation.py","*/sim/aml/evaluator.py" -m pytest  --doctest-modules --doctest-glob="*.rst" wntr
+      env:
+        COVERAGE_FILE: .coverage.${{ matrix.python-version }}.${{ matrix.os }}
     - name: Save coverage
       uses: actions/upload-artifact@v4
       with:

.github/workflows/release.yml

@@ -14,7 +14,7 @@ jobs:
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        os: [windows-latest, macOS-13, macos-13, ubuntu-latest] 
+        os: [windows-latest, macos-latest, macos-13, ubuntu-latest] 
     steps:
     - uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
     - name: Build wheels

.gitignore

@@ -1,26 +1,98 @@
+# Developer IDE directories
 /.project
-/.spyproject
-_build/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# VS Code project settings
+*.code-workspace
+/.vscode
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Distribution / packaging
+.Python
 build/
-wntr.egg-info/
+develop-eggs/
 dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
 docker/
 /.vscode
 
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# Mac cleanup
+**/.DS_Store
+
+# Other items / test output
+temp*
 *.pyd
 *.pyc
 *.egg
-*.coverage
 *.ipynb
 *.html
 *.pickle
 *.xlsx
 
-temp*
-
 examples/*.inp
 wntr/tests/*.png
 wntr/tests/*.tif
 
+# Documentation build files
+documentation/_build
 documentation/_local
 documentation/apidoc
+
+# WNTR specific
+wntr/sim/aml/*.so
+wntr/sim/aml/*.dll
+wntr/sim/aml/*.dylib
+wntr/sim/network_isolation/*.so
+wntr/sim/network_isolation/*.dll
+wntr/sim/network_isolation/*.dylib

MANIFEST.in

@@ -7,4 +7,7 @@ include wntr/sim/aml/evaluator*
 include wntr/sim/aml/numpy.i
 include wntr/sim/network_isolation/network_isolation*
 include wntr/sim/network_isolation/numpy.i
-include wntr/tests/networks_for_testing/*.inp
+include wntr/tests/networks_for_testing/*.inp
+include wntr/library/msx/*.json
+include wntr/library/msx/*.msx
+include wntr/epanet/libepanet/darwin-formula/libomp.rb