Skip to content

Paper: Eyes in the Sky: Estimating Inland Water Quality Using Landsat Data - #1111

Merged
fwkoch merged 6 commits into
scipy-conference:2025from
dkedar7:watersat
Oct 14, 2025
Merged

Paper: Eyes in the Sky: Estimating Inland Water Quality Using Landsat Data#1111
fwkoch merged 6 commits into
scipy-conference:2025from
dkedar7:watersat

Conversation

@dkedar7

@dkedar7 dkedar7 commented Jun 14, 2025

Copy link
Copy Markdown

Algal blooms threaten human health and aquatic ecosystems, making monitoring essential. While Chlorophyll-A (Chl-a) effectively indicates algal presence, laboratory analysis is complex. This study utilizes satellite imagery as an alternative, addressing previous research limitations caused by scarce lab data. Moreover, it also demonstrates how openly available Chl-a measurements obtained from the Water Quality Portal (WQP) can enable communities and organizations of all sizes to measure Chl-a in their waters without access to specialized in-situ water sampling skills or laboratory analysis equipment.

By combining the extensive WQP dataset with Landsat satellite imagery, these models estimate Chl-a levels in New York's inland waters. Training with eight years of data demonstrated a strong correlation between satellite-derived and actual measurements (MAPE: 0.96%; RMSE: 3.2 μg/L), enabling improved spatial and temporal monitoring capabilities.

@github-actions

github-actions Bot commented Jun 14, 2025

Copy link
Copy Markdown

Curvenote Preview

Directory Preview Checks Updated (UTC)
papers/kedar_dabhadkar_watersat 🔍 Inspect 28 checks passed (1 optional) Oct 14, 2025, 5:50 PM

@dkedar7 dkedar7 changed the title Paper: Eyes in the Sky Estimating Inland Water Quality Using Landsat Data Paper: Eyes in the Sky: Estimating Inland Water Quality Using Landsat Data Jun 14, 2025
@scipy-conference scipy-conference deleted a comment from github-actions Bot Jun 14, 2025
@rowanc1 rowanc1 added paper This indicates that the PR in question is a paper draft This triggers Curvenote Preview actions labels Jun 14, 2025
@ameyxd

ameyxd commented Jun 18, 2025

Copy link
Copy Markdown
Contributor

Inviting reviewers: @fr1ll and @madhavmk

@ameyxd

ameyxd commented Jul 2, 2025

Copy link
Copy Markdown
Contributor

@cdlindsey will serve as editor for this paper.

@fr1ll fr1ll left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well-written, clear work. The one place I got lost is how the WQA data is spatially indexed - by the name of the body of water, by lat/long, or other. If you clear this up, it will help to explain how the Landsat data ties. I also became curious how the data moves over time for any bodies with many measurements. The fact that these two questions came up indicates it's an interesting and accessible topic, clearly presented.

Comment thread papers/kedar_dabhadkar_watersat/main.md Outdated

Harmful algal blooms (HABs) can severely impact both the environment and human health. Environmentally, HABs deplete oxygen in water bodies, leading to fish kills and loss of aquatic biodiversity. They can block sunlight, disrupting aquatic plant growth and altering food webs. Many HABs produce toxins that accumulate in the ecosystem, affecting wildlife and domestic animals. For humans, exposure to these toxins—through drinking water, recreation, or consumption of contaminated fish and shellfish—can cause a range of health issues, including skin irritation, respiratory problems, gastrointestinal illness, and, in severe cases, liver or neurological damage. HABs also threaten water supplies and recreational activities, resulting in economic losses for affected communities. Therefore, detecting and mitigating HABs is an area of continued attention.

A key indicator of HABs is the concentration of Chlorophyll a (Chl-a) in water. Chl-a is a pigment found in all photosynthetic algae, and its presence is directly correlated with the abundance of algal biomass. Elevated Chl-a levels typically signal increased algal growth, which can indicate the onset or presence of a bloom. Therefore, monitoring Chl-a concentrations provides a reliable proxy for detecting and quantifying HABs in aquatic environments.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding a reference tying Chl-a to algal blooms. This could help an interested reader to understand more of the scientific underpinnings of your work.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. References added.

Comment thread papers/kedar_dabhadkar_watersat/main.md Outdated
Values of Chl-a predicted by the best performance model plotted against actual values.
:::

Furthermore, using Landsat-based satellite remote sensing insights is scalable and yields more than 30 times as many Chl-a readings as in-situ readings obtained from WQP (@fig:nycomparison).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By my understanding, the WQA dataset gives Chl-a "readings", while the LANDSAT data lets you predict Chl-a. I would suggest to re-phrase this sentence to indicate that these are predictions or inferences.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rephrased in the text as well as figure caption.

or if you prefer LaTeX you can try `tabular` or `longtable` environments:
Querying the WQP for Chl-a measurements within the bounds of New York State between January 2015 and October 2023 yields a dataset of 68,075 measurements. WQP is not consistent in its nomenclature and data quality checks, so it was essential to specify different ways of referring to Chl-a. Some of those ways are _Chlorophyll_, _Chlorophyll A_, _Chlorophyll a_, _Chlorophyll a (probe relative fluorescence)_, _Chlorophyll a (probe)_, _Chlorophyll a - Periphyton (attached)_, _Chlorophyll a - Phytoplankton (suspended)_, _Chlorophyll a, corrected for pheophytin_, and so on. HyRiver, a Python library published by [](https://doi.org/10.21105/joss.03175), was used to connect to the WQP and export data.

To reduce the effect of repeated measurements from the same water body and around the same time, we deduplicate data by removing consecutive values with the same date and time if the relative standard deviation (RSD) of those values was less than 10. Passive remote sensing measurements like Landsat are not sensitive to measuring water quality beyond 100 meters depth. So, we removed all measurements that were reported to be measured beyond 100m in depth. After applying these filters and removing unclean values (where measurement values were not numerical), we are left with 13,717 measurements.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you will be joining SQA data and LANDSAT data, it would be helpful to specify what location information is present in each SQA measurement. Do the measurements feature a lat/long value, a named body of water, or something else?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few lines explaining how the spatial joining happens with stress on indexing fields: location (latitude, longitude) and dates.

Comment thread papers/kedar_dabhadkar_watersat/main.md Outdated

:::{figure} nycomparison.png
:label: fig:nycomparison
The number of remotely sensed Chl-a estimations (B) is more than 30x the number of in-situ measurements obtained from the Water Quality Portal (A) across NY’s 200 biggest lakes and in the same 8-year period (2015-2023).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image would benefit from a key for the bubble size. Does each bubble represent the center of a body of water, and is the size indicating different measurements e.g. from different times? For large bodies of water, can there sometimes be more than one measurement from the same day, but from different locations?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a bubble size legend and explanation about what bubble centers and sizes mean.

@fr1ll

fr1ll commented Jul 16, 2025

Copy link
Copy Markdown

By the way, @dkedar7 - this is my first time doing one of these reviews, so please let me know if my review has any issues I can fix!

@madhavmk

Copy link
Copy Markdown

Interesting work, and well-written paper @dkedar7 🙂 !

I had a few points for discussion and potential improvement.

  1. Curious if you have any hypotheses why the standard deviation of error seems to increase with higher concentrations of Chl-a (refer to 'predvsactual.png')
  2. I believe listing the hyperparameter values for the best set of ML model evaluations is beneficial for reproducibility. It may also help analyze in which situations the second-best model, XGBoost, can be used instead.
  3. The model's training data size after pre-processing is quite small - just 428 samples. Why are we limiting our data to only New York State's water sources? Since the water-quality data website gives info on a large range of countries and states, can this experiment and results be generalized to a wider region? Doing so should increase the training data size and make the model more robust. I understand you mentioned it in the future work section, but curious why it wasn't done here.

@dkedar7

dkedar7 commented Jul 26, 2025

Copy link
Copy Markdown
Author

Thank you for the comments. Based on the feedback, I'm listing the following action items to help us track.

  • Add a reference tying Chl-a to algal blooms
  • Add more details explaining spatially joining Landsat data with data from the Water Quality Portal (WQP) (common data columns available in both data sources, method, and assumptions)
  • Attempt to find and show examples of temporal trend(s) for water bodies with sufficient data
  • Rephrase for clarity how this approach (Landsat + WQP -> ML predictions) allows "30 times as many readings as WQP-only".
  • Rephrase the caption of the bubble chart for clarity
  • Add details around model training and interpretation: why the standard deviation goes up with increased Chl-a values, hyperparameters, and model choices.
  • Explain: why this work doesn't consider a larger geography.

@dkedar7

dkedar7 commented Aug 23, 2025

Copy link
Copy Markdown
Author

@madhavmk My thoughts about your discussion points:

  • Most of the chl-a values in the training dataset are biased towards lower values, and higher values are rare. This is the nature of the inland water bodies (a good sign for water quality). Unfortunately, due to the underrepresentation of extreme values (mainly higher in this case), trained models don't perform well due to mean bias. So the standard increases with actual chl-a values.
  • I've listed hyperparameters in the revision. This was a good point; thanks for bringing it up.
  • The most significant reason behind limiting to a single geography at the moment is that restricting the geography allows us to build models that are more specific to the region and its water bodies, since other studies have shown that water quality models are not easily transferable across geographies. I also added this explanation with references in the paper text for readers.

@dkedar7

dkedar7 commented Aug 23, 2025

Copy link
Copy Markdown
Author

For the third point from the checklist, I attempted to find cases where there is sufficient data across WQP and Landsat-inferred values, but unfortunately, there aren't any. This is due to strict data cleaning steps. There are three cases (distinct lakes) where there are some overlapping points, but not enough to show a comparison trend plot. These are Greenwood Lake, Skaneateles Lake, and Owasco Lake. The overlaps can be seen using the accompanying web app: https://watersat-hpn4y2dvda-ue.a.run.app/

@madhavmk

Copy link
Copy Markdown

The updates look good to me !
Also, could someone add me as a reviewer to this PR? I'm unable to do it myself.

@dkedar7

dkedar7 commented Sep 7, 2025

Copy link
Copy Markdown
Author

@cdlindsey What should be our next steps here?

@dkedar7
dkedar7 requested a review from fr1ll September 8, 2025 11:19
@fwkoch fwkoch added approved This triggers Curvenote Submission action and removed draft This triggers Curvenote Preview actions labels Oct 14, 2025
@fwkoch
fwkoch merged commit 1de40ab into scipy-conference:2025 Oct 14, 2025
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved This triggers Curvenote Submission action assigned-editor assigned-reviewer paper This indicates that the PR in question is a paper

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants