Skip to content

Commit 158e4c7

Browse files
robmarkcolerobmarkcole
authored
Merge pull request #623 from allenai/add-nasa-hls
Add NASA HLS v2.0 data sources and integration tests
2 parents 7038748 + 719ea1f commit 158e4c7

9 files changed

Lines changed: 1708 additions & 15 deletions

File tree

‎docs/DataSources.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,9 @@ detailed page with configuration options and available bands. See
4646
|---|---|---|
4747
| [planetary_computer.Hls2S30](data_sources/planetary_computer_Hls2.md#rslearndatasourcesplanetary_computerhls2s30) | Microsoft Planetary Computer | HLS v2 Sentinel-2 (30m) |
4848
| [planetary_computer.Hls2L30](data_sources/planetary_computer_Hls2.md#rslearndatasourcesplanetary_computerhls2l30) | Microsoft Planetary Computer | HLS v2 Landsat (30m) |
49+
| [nasa_hls.Hls2](data_sources/nasa_hls_Hls2.md#rslearndatasourcesnasa_hlshls2) | NASA LP DAAC | Combined semantic-band HLS v2.0 time-series across Sentinel and Landsat |
50+
| [nasa_hls.Hls2S30](data_sources/nasa_hls_Hls2.md#rslearndatasourcesnasa_hlshls2s30) | NASA LP DAAC | HLSS30 v2.0, direct materialization via protected S3 |
51+
| [nasa_hls.Hls2L30](data_sources/nasa_hls_Hls2.md#rslearndatasourcesnasa_hlshls2l30) | NASA LP DAAC | HLSL30 v2.0, direct materialization via protected S3 |
4952

5053
#### NAIP
5154

‎docs/data_sources/nasa_hls_Hls2.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
## rslearn.data_sources.nasa_hls.Hls2S30
2+
3+
NASA LP DAAC HLS v2.0 Sentinel-2 ([HLSS30](https://hls.gsfc.nasa.gov/)) data on
4+
CMR STAC / LP DAAC cloud storage. Direct materialization is supported.
5+
6+
This data source prefers LP DAAC `s3_*` assets and automatically fetches temporary
7+
AWS credentials from the LP DAAC `s3credentials` endpoint. Set:
8+
9+
- `EARTHDATA_TOKEN`
10+
11+
### Region Behavior
12+
13+
LP DAAC's temporary AWS credentials are intended for same-region direct S3 access,
14+
with best support/performance in `us-west-2`.
15+
16+
- If your job is running in or near `us-west-2`, rslearn will try the LP DAAC
17+
`s3://` asset first.
18+
- If direct S3 access is denied or unavailable, rslearn falls back to the
19+
authenticated HTTPS asset URL from STAC.
20+
- The HTTPS fallback is intended to make local development and non-`us-west-2`
21+
runs work, but it will usually be slower than direct in-region S3 access.
22+
23+
### Configuration
24+
25+
```jsonc
26+
{
27+
"class_path": "rslearn.data_sources.nasa_hls.Hls2S30",
28+
"init_args": {
29+
// Optional list of bands to expose.
30+
"band_names": null,
31+
// Optional STAC query filter.
32+
"query": null,
33+
// Optional STAC sort property, e.g. "eo:cloud_cover".
34+
"sort_by": null,
35+
"sort_ascending": true,
36+
// Optional Earthdata bearer token override (otherwise EARTHDATA_TOKEN is used).
37+
"earthdata_token": null,
38+
"timeout": "30s"
39+
}
40+
}
41+
```
42+
43+
### Available Bands
44+
45+
The default band set includes:
46+
- B01
47+
- B02
48+
- B03
49+
- B04
50+
- B05
51+
- B06
52+
- B07
53+
- B08
54+
- B8A
55+
- B11
56+
- B12
57+
58+
Additional supported raster assets:
59+
- B09
60+
- B10
61+
- Fmask
62+
- SAA
63+
- SZA
64+
- VAA
65+
- VZA
66+
67+
Band names are the HLS asset keys listed above.
68+
69+
## rslearn.data_sources.nasa_hls.Hls2
70+
71+
Combined NASA LP DAAC HLS v2.0 time-series datasource with a shared semantic band
72+
schema across Sentinel-2 and Landsat observations.
73+
74+
This datasource is intended for single chronological time-series use. It does **not**
75+
expose raw `Bxx` asset keys, because those keys do not mean the same thing across
76+
HLSS30 and HLSL30. Instead it exposes semantic band names shared by both sensors.
77+
78+
Authentication, direct S3 preference, and HTTPS fallback behavior are the same as
79+
for `rslearn.data_sources.nasa_hls.Hls2S30`.
80+
81+
For multi-item chronological series, configure the layer query with `max_matches > 1`.
82+
Like other rslearn datasources, the default query behavior only returns a single match.
83+
84+
### Configuration
85+
86+
```jsonc
87+
{
88+
"class_path": "rslearn.data_sources.nasa_hls.Hls2",
89+
"init_args": {
90+
// Optional semantic band list. Defaults to the common reflectance bands below.
91+
"band_names": null,
92+
// Optional source subset. Valid values are "sentinel" and "landsat".
93+
"sources": ["sentinel", "landsat"],
94+
"query": null,
95+
// Defaults to datetime so the merged series is chronological.
96+
"sort_by": "datetime",
97+
"sort_ascending": true,
98+
"earthdata_token": null,
99+
"timeout": "30s"
100+
}
101+
}
102+
```
103+
104+
### Available Bands
105+
106+
Default bands:
107+
- coastal
108+
- blue
109+
- green
110+
- red
111+
- nir
112+
- swir16
113+
- swir22
114+
115+
Optional additional shared bands:
116+
- cirrus
117+
- fmask
118+
- solar_azimuth
119+
- solar_zenith
120+
- view_azimuth
121+
- view_zenith
122+
123+
Use `Hls2S30` or `Hls2L30` instead if you need source-specific raw bands like
124+
Sentinel red-edge bands or Landsat thermal bands.
125+
126+
## rslearn.data_sources.nasa_hls.Hls2L30
127+
128+
NASA LP DAAC HLS v2.0 Landsat ([HLSL30](https://hls.gsfc.nasa.gov/)) data on
129+
CMR STAC / LP DAAC cloud storage. Direct materialization is supported.
130+
131+
Authentication and temporary S3 credential handling are the same as for
132+
`rslearn.data_sources.nasa_hls.Hls2S30`.
133+
134+
The same region behavior applies here: direct `s3://` reads are preferred when
135+
same-region LP DAAC access is available, and rslearn falls back to authenticated
136+
HTTPS otherwise.
137+
138+
### Configuration
139+
140+
```jsonc
141+
{
142+
"class_path": "rslearn.data_sources.nasa_hls.Hls2L30",
143+
"init_args": {
144+
"band_names": null,
145+
"query": null,
146+
"sort_by": null,
147+
"sort_ascending": true,
148+
"earthdata_token": null,
149+
"timeout": "30s"
150+
}
151+
}
152+
```
153+
154+
### Available Bands
155+
156+
The default band set includes:
157+
- B01
158+
- B02
159+
- B03
160+
- B04
161+
- B05
162+
- B06
163+
- B07
164+
- B09
165+
- B10
166+
- B11
167+
168+
Additional supported raster assets:
169+
- Fmask
170+
- SAA
171+
- SZA
172+
- VAA
173+
- VZA
174+
175+
Band names are the HLS asset keys listed above.

0 commit comments

Comments
 (0)