Skip to content

Commit df82852

Browse files
SheargrubCopilot
andauthored
Documentation revamp (#330)
* documentation revamp README rewritten Added Sections 1, 2, 3.1, 3.3 Added stub for Section 3.2 Moved images to `docs/img/` Moved many old files into `archive/` * Formatting and clarification * Move and update contribution guide Filename is non-final; it's just set relatively high to keep it at the end of the docs. * Add info on models Also adds a stub for building the container, since I've now learned that this is the only way to add BMI modules. * Add contact page, move "contribute" I think this is more or less a finalized table of contents? * Model attribution changes LSTM: retargeted to CIROH fork, credited Jonathan Frame T-Route: credited Josh Cunningham * Add realizations documentation Reordered other pages to better align with realizations info Added paths/class names to models where I could find them * Merge attribution changes * Pre-PR cleanup Slight de-stubbing of build page Replace leftover carpentries formatting from Installation Add placeholder notices until SMP/SFT issue is resolved * Typo fixed Co-authored-by: Copilot <[email protected]> * Apply Copilot nitpicks where appropriate Perhaps unsurprisingly, LLMs are far better at proofreading than code review. (Still produced a few bizarre misfires though, of course) * Add performance advice to installation guide Need to remember to add this info to NGIAB 101, too... --------- Co-authored-by: Copilot <[email protected]>
1 parent 13c2d92 commit df82852

36 files changed

+650
-167
lines changed

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
# User directory for custom scripts
2+
.localignore/*
3+
14
# Local .terraform directories
25
**/.terraform/*
36

README.md

Lines changed: 40 additions & 158 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,15 @@
22

33
> Run the NextGen National Water Resources Modeling Framework locally with ease.
44
5-
NGIAB provides a containerized and user-friendly solution for running the NextGen framework, allowing you to control inputs, configurations, and execution on your local machine.
5+
[NGIAB](https://ngiab.ciroh.org) provides a containerized and user-friendly solution for running the NextGen framework, allowing you to control inputs, configurations, and execution on your local machine.
66

77
<p align="center">
8-
<img src="https://github.com/CIROH-UA/NGIAB-CloudInfra/blob/main/image/README/ngiab.png" width="300">
8+
<img src="./docs/img/ngiab.png" width="300">
99
</p>
1010

1111
| | |
1212
| --- | --- |
13-
| ![CIROH Logo](https://ciroh.ua.edu/wp-content/uploads/2022/08/CIROHLogo_200x200.png) | Funding for this project was provided by the National Oceanic & Atmospheric Administration (NOAA), awarded to the Cooperative Institute for Research to Operations in Hydrology (CIROH) through the NOAA Cooperative Agreement with The University of Alabama (NA22NWS4320003). |
13+
| ![CIROH Logo](./docs/img/ciroh-bgsafe.png) | Funding for this project was provided by the National Oceanic & Atmospheric Administration (NOAA), awarded to the Cooperative Institute for Research to Operations in Hydrology (CIROH) through the NOAA Cooperative Agreement with The University of Alabama (NA22NWS4320003). |
1414

1515
[![ARM Build and push final image](https://github.com/CIROH-UA/NGIAB-CloudInfra/actions/workflows/docker_image_main_branch.yml/badge.svg)](https://github.com/CIROH-UA/NGIAB-CloudInfra/actions/workflows/docker_image_main_branch.yml)
1616

@@ -19,157 +19,33 @@ NGIAB provides a containerized and user-friendly solution for running the NextGe
1919
- **Run NextGen Locally**: Experiment with the framework on your machine
2020
- **Control Over Inputs**: Choose specific regions/basins and modify input data
2121
- **Simplified Setup**: Easy deployment using Docker containers
22-
- **Open Research**: Promote transparency through open-source tools
23-
- **Visualization**: Built-in support for output visualization
22+
- **Open Research**: Promotes transparency through open-source tooling
2423
- **Evaluation Tools**: Integrated TEEHR evaluation capabilities
24+
- **Visualization**: Built-in support for output visualization via Tethys Platform
2525

26-
## Prerequisites
27-
28-
### Windows
29-
1. **Install WSL**
30-
```bash
31-
wsl --install
32-
# If the above doesn't work, try:
33-
sudo apt install wsl
34-
```
35-
36-
2. **Install Docker Desktop**
37-
- Download from [Docker's official website](https://docs.docker.com/desktop/install/windows-install/#install-docker-desktop-on-windows)
38-
- Launch Docker Desktop
39-
- Open WSL as Administrator
40-
- Verify installation: `docker ps -a`
41-
42-
### Mac
43-
1. **Install Docker Desktop**
44-
- Download from [Docker's Mac installer page](https://docs.docker.com/desktop/install/mac-install/)
45-
- Launch Docker Desktop
46-
- Verify installation: `docker ps -a`
47-
48-
### Linux
49-
1. **Install Docker**
50-
- Follow [Linux installation guide](https://docs.docker.com/desktop/install/linux-install/)
51-
- Start Docker service
52-
- Verify installation: `docker ps -a`
53-
54-
## Quick Start Guide
55-
56-
### 1. Set Up Project Directory
57-
```bash
58-
mkdir -p NextGen/ngen-data
59-
cd NextGen/ngen-data
60-
```
61-
62-
### 2. Download Sample Data
63-
#### Option 1: AWI-009 input data (realization file includes - SLOTH, NoahOWP, CFE) - calibrated realization file for Provo River near Woodland, UT
64-
```bash
65-
wget https://ciroh-ua-ngen-data.s3.us-east-2.amazonaws.com/AWI-009/AWI_16_10154200_009.tar.gz
66-
tar -xf AWI_16_10154200_009.tar.gz
67-
```
68-
#### Option 2: AWI-007 input data (realization file includes - SLOTH, NoahOWP, CFE)
69-
```bash
70-
wget https://ciroh-ua-ngen-data.s3.us-east-2.amazonaws.com/AWI-007/AWI_16_2863657_007.tar.gz
71-
tar -xf AWI_16_2863657_007.tar.gz
72-
```
73-
#### Option 3: AWI-008 input data (realization file includes - SLOTH, Demostration LSTM)
74-
```bash
75-
wget --no-parent https://ciroh-ua-ngen-data.s3.us-east-2.amazonaws.com/AWI-008/AWI_16_2863806_008.tar.gz
76-
tar -xf AWI_16_2863806_008.tar.gz
77-
```
78-
79-
### 3. Clone and Run
80-
```bash
81-
cd NextGen
82-
git clone https://github.com/CIROH-UA/NGIAB-CloudInfra.git
83-
cd NGIAB-CloudInfra
84-
./guide.sh
85-
```
86-
87-
### 4. NextGen Run Directory Structure (`ngen-run/`)
88-
89-
Running NextGen requires building a standard run directory complete with only the necessary files. Below is an explanation of the standard. Reference for discussion of the standard [here](https://github.com/CIROH-UA/NGIAB-CloudInfra/pull/17).
90-
91-
A NextGen run directory `ngen-run` is composed of three necessary subfolders `config, forcings, outputs` and an optional fourth subfolder `metadata`.
92-
93-
```
94-
ngen-run/
95-
96-
├── config/
97-
98-
├── forcings/
99-
100-
├── lakeout/
101-
|
102-
├── metadata/
103-
104-
├── outputs/
105-
106-
├── restart/
107-
```
108-
109-
The `ngen-run` directory contains the following subfolders:
110-
111-
- `config`: model configuration files and hydrofabric configuration files. A deeper explanation [here](#configuration-directory-ngen-runconfig)
112-
- `forcings`: catchment-level forcing timeseries files. These can be generated with the [forcingprocessor](https://github.com/CIROH-UA/ngen-datastream/tree/main/forcingprocessor). Forcing files contain variables like wind speed, temperature, precipitation, and solar radiation.
113-
- `lakeout`: for t-route
114-
- `metadata` is an optional subfolder. This is programmatically generated and it used within to ngen. Do not edit this folder.
115-
- `outputs`: This is where ngen will place the output files.
116-
- `restart`: For restart files
117-
118-
#### Configuration directory `ngen-run/config/`
119-
This folder contains the NextGen realization file, which serves as the primary model configuration for the ngen framework. This file specifies which models to run and with which parameters, run parameters like date and time, and hydrofabric specifications.
120-
121-
Based on the models defined in the realization file, BMI configuration files may be required. For those models that require per-catchment configuration files, a folder will hold these files for each model in `ngen-run/config/cat-config`. See [here](https://github.com/CIROH-UA/ngen-datastream/blob/main/docs/NGEN_MODULES.md) for which models ngen-datastream supports automated BMI configuration file generation. See the directory structure convention below.
122-
123-
```
124-
ngen-run/
125-
|
126-
├── config/
127-
| │
128-
| ├── nextgen_09.gpkg
129-
| |
130-
| ├── realization.json
131-
| |
132-
| ├── ngen.yaml
133-
| |
134-
| ├── cat-config/
135-
| │ |
136-
| | ├──PET/
137-
| │ |
138-
| | ├──CFE/
139-
| │ |
140-
| | ├──NOAH-OWP-M/
141-
...
142-
```
143-
144-
Hydrofabric Example files: `conus_nextgen.gpkg`
145-
NextGen requires a single geopackage file. This file is the [hydrofabric](https://mikejohnson51.github.io/hyAggregate/) (spatial data). An example geopackage can be found on Lynker-Spatial [here](https://www.lynker-spatial.com/data?path=hydrofabric%2Fv2.2%2F). Tools to subset a geopackage into a smaller domain can be found at [Lynker's hfsubset](https://github.com/LynkerIntel/hfsubset).
146-
147-
## Case Study: Provo River Basin, UT
148-
149-
![Provo River Basin Map](https://github.com/CIROH-UA/NGIAB-CloudInfra/blob/main/image/README/VPU16_007.png)
150-
151-
This repository includes a complete case study of the Provo River Basin, demonstrating NGIAB's capabilities in a real-world scenario.
152-
1. **Geospatial Visualization**
153-
![Nexus Output](https://github.com/CIROH-UA/NGIAB-CloudInfra/blob/main/image/README/Provo_GeoSpatial.png)
154-
155-
2. **Time Series Analysis**
156-
- Catchments
157-
![Catchment Time Series](https://github.com/CIROH-UA/NGIAB-CloudInfra/blob/main/image/README/Provo_catchments.png)
158-
- Nexus Points
159-
![Nexus Time Series](https://github.com/CIROH-UA/NGIAB-CloudInfra/blob/main/image/README/Provo_nexus_point.png)
160-
161-
## Advanced Usage
162-
163-
### Running the Visualizer
164-
```bash
165-
./viewOnTethys.sh
166-
```
167-
168-
### Building NGIAB Locally
169-
```bash
170-
cd docker
171-
docker build -f Dockerfile -t awiciroh/ciroh-ngen-image:latest . --no-cache
172-
```
26+
| | | |
27+
| --- | --- | --- |
28+
| ![Nexus Output](./docs/img/Provo_GeoSpatial.png) | ![Catchment Time Series](./docs/img/Provo_catchments.png) | ![Nexus Time Series](./docs/img/Provo_nexus_point.png) |
29+
30+
## Navigating this repository
31+
32+
### For general use
33+
34+
- **NGIAB Guide Scripts**: This repository holds several guide scripts: `guide.sh`, `runTeehr.sh`, and `viewOnTethys.sh`. These scripts are the recommended way to run NGIAB.
35+
36+
- **Documentation**: The [`docs/` folder](./docs/00_CONTENTS.md) contains information on all of the finer details that can help you get the most out of the contents of this repository.
37+
- For broader ecosystem-wide documentation, please visit DocuHub at [docs.ciroh.org/products/ngiab](https://docs.ciroh.org/products/ngiab), where all of the information from this and other NGIAB repositories is mirrored.
38+
39+
### For development
40+
41+
- `docker/`: This folder contains the Dockerfile and entrypoint for the NGIAB container. See [Section 3.1](./docs/03_01_CONTAINERS.md) of the documentation for more information.
42+
- Releases built from this folder are available at [https://hub.docker.com/r/awiciroh/ciroh-ngen-image](https://hub.docker.com/r/awiciroh/ciroh-ngen-image).
43+
- `.github/`: Workflows, issue templates, and other GitHub-focused configuration files.
44+
- `archive/`: Older files that are no longer maintained.
45+
46+
## Contributing
47+
48+
Interested in contributing? Please see our [contribution guide](05_CONTRIBUTE.md) for more information.
17349

17450
## Contributors
17551
- Arpita Patel, Alabama Water Institute, CIROH ([email protected])
@@ -187,14 +63,20 @@ docker build -f Dockerfile -t awiciroh/ciroh-ngen-image:latest . --no-cache
18763
Project: CIROH: Community Water Model Infrastructure, Stewardship, and Integration (PI - Steven Burian)
18864
Project: [Advancing Community NextGen and NextGen In A Box (NGIAB) – Paving the Pathway to Operations](https://ciroh.ua.edu/research-projects/advancing-community-nextgen-and-nextgen-in-a-box-ngiab-paving-the-pathway-to-operations/) (PI - Arpita Patel)
18965

190-
## Additional Resources
66+
## Additional resources
19167

192-
- [End-to-End Setup Guide](https://docs.ciroh.org/docs/products/Community%20Hydrologic%20Modeling%20Framework/nextgeninaboxDocker/workflow)
68+
- [NGIAB Website](https://ngiab.ciroh.org)
69+
- [NGIAB 101 Training Module](https://docs.ciroh.org/training-NGIAB-101/)
70+
- [NGIAB on DocuHub](https://docs.ciroh.org/)
71+
72+
### Upstream repositories
19373
- [NextGen Framework Prototype](https://github.com/NOAA-OWP/ngen)
19474
- [Community ngen Repository](https://github.com/CIROH-UA/ngen)
19575
- [Community troute Repository](https://github.com/CIROH-UA/t-route)
196-
- [NGIAB Data Preprocessor](https://github.com/AlabamaWaterInstitute/NGIAB_data_preprocess)
197-
- [ngen-datastream Repository](https://github.com/CIROH-UA/ngen-datastream/tree/main)
76+
77+
### NGIAB ecosystem
78+
- [NGIAB Data Preprocess](https://github.com/CIROH-UA/NGIAB_data_preprocess)
19879
- [NGIAB TEEHR Integration](https://github.com/CIROH-UA/ngiab-teehr)
199-
- [Data Visualizer](https://github.com/CIROH-UA/ngiab-client)
200-
80+
- [NGIAB Data Visualizer](https://github.com/CIROH-UA/ngiab-client)
81+
- [DataStreamCLI and Research Datastream](https://github.com/CIROH-UA/ngen-datastream/tree/main)
82+
- [NGIAB Calibration](https://github.com/CIROH-UA/ngiab-cal)
File renamed without changes.
File renamed without changes.

archive/README_ARCHIVE.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Archive
2+
3+
This folder contains deprecated files that were formerly used alongside this repository. While they may still be of use, the contents of this folder are not actively maintained.
4+
5+
- `config/`: Contains a realization file that calibrated for the [AWI-009 sample input dataset](https://ciroh-ua-ngen-data.s3.us-east-2.amazonaws.com/AWI-009/AWI_16_10154200_009.tar.gz). (*Warning: link leads to large file download*)
6+
- `docs/`: Some stray planning documents. Likely outdated.
7+
- `terraform/`: Older configurations for running NGIAB on CIROH's AWS servers. Superceded by [DataStreamCLI](https://github.com/CIROH-UA/ngen-datastream).
8+
- `AWI_007_windows_test.ps1`: A PowerShell script that invokes NGIAB on Windows. Somewhat redundant, since the container runs on WSL anyway.
9+
- `Example.json`: An example realization file.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.

0 commit comments

Comments
 (0)