Skip to content

Commit ef74edd

Browse files
Merge pull request #24 from manaakiwhenua/s2
Adds S2 support
2 parents 90e35cd + 30383bc commit ef74edd

File tree

12 files changed

+1708
-1247
lines changed

12 files changed

+1708
-1247
lines changed

README.md

Lines changed: 108 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,18 @@
44

55
Python-based CLI tool to index raster files to DGGS in parallel, writing out to Parquet.
66

7-
Currently this supports the H3 and rHEALPix DGGSs. Contributions (particularly for additional DGGSs), suggestions, bug reports and strongly worded letters are all welcome.
7+
Currently this supports the following DGGSs:
8+
9+
- [H3](https://h3geo.org/)
10+
- [rHEALPix](https://datastore.landcareresearch.co.nz/dataset/rhealpix-discrete-global-grid-system)
11+
- [S2](http://s2geometry.io/)
12+
13+
And these geocode systems:
14+
15+
- [Geohash](https://en.wikipedia.org/wiki/Geohash)
16+
- [Maidenhead Locator System](https://en.wikipedia.org/wiki/Maidenhead_Locator_System)
17+
18+
Contributions (particularly for additional DGGSs), suggestions, bug reports and strongly worded letters are all welcome.
819

920
![Example use case for raster2dggs, showing how an input raster can be indexed at different DGGS resolutions, while retaining information in separate, named bands](docs/imgs/raster2dggs-example.png "Example use case for raster2dggs, showing how an input raster can be indexed at different H3 resolutions, while retaining information in separate, named bands")
1021

@@ -23,8 +34,11 @@ Options:
2334
--help Show this message and exit.
2435
2536
Commands:
26-
h3 Ingest a raster image and index it to the H3 DGGS.
27-
rhp Ingest a raster image and index it to the rHEALPix DGGS.
37+
geohash Ingest a raster image and index it using the Geohash...
38+
h3 Ingest a raster image and index it to the H3 DGGS.
39+
maidenhead Ingest a raster image and index it using the Maidenhead...
40+
rhp Ingest a raster image and index it to the rHEALPix DGGS.
41+
s2 Ingest a raster image and index it to the S2 DGGS.
2842
```
2943

3044
```
@@ -96,6 +110,9 @@ Output is in the Apache Parquet format, a directory with one file per partition.
96110

97111
For a quick view of your output, you can read Apache Parquet with pandas, and then use h3-pandas and geopandas to convert this into a GeoPackage for visualisation in a desktop GIS, such as QGIS. The Apache Parquet output is indexed by the DGGS column, so it should be ready for association with other data prepared in the same DGGS.
98112

113+
<details>
114+
<summary>For H3 output...</summary>
115+
99116
```python
100117
>>> import pandas as pd
101118
>>> import h3pandas
@@ -118,6 +135,10 @@ h3_09
118135
[5656 rows x 10 columns]
119136
>>> o.h3.h3_to_geo_boundary().to_file('~/Downloads/Sen2_Test_h3-9.gpkg', driver='GPKG')
120137
```
138+
</details>
139+
140+
<details>
141+
<summary>For rHEALPix output...</summary>
121142

122143
For rHEALPix DGGS output, you can use [`rHP-Pandas`](https://github.com/manaakiwhenua/rHP-Pandas):
123144

@@ -143,6 +164,89 @@ R88727068808 22 39 43 80 146 163 177 198 165 83
143164
[223104 rows x 10 columns]
144165
>>> o.rhp.rhp_to_geo_boundary().to_file('~/Downloads/Sen2_Test_rhp-11.gpkg', driver='GPKG')
145166
```
167+
</details>
168+
169+
<details>
170+
<summary>For S2 output...</summary>
171+
172+
For S2 output, use [`s2sphere`](https://pypi.org/project/s2sphere/):
173+
174+
```python
175+
import pandas as pd
176+
import geopandas as gpd
177+
import s2sphere
178+
from shapely.geometry import Polygon
179+
180+
df = pd.read_parquet('./tests/data/output/7/sample_tif_s2')
181+
df = df.reset_index()
182+
183+
def s2id_to_polygon(s2_id_hex):
184+
# Parse the S2CellId
185+
cell_id = s2sphere.CellId.from_token(s2_id_hex)
186+
cell = s2sphere.Cell(cell_id)
187+
188+
# Get the 4 vertices of the S2 cell
189+
vertices = []
190+
for i in range(4):
191+
vertex = cell.get_vertex(i)
192+
# Convert to lat/lon degrees
193+
lat_lng = s2sphere.LatLng.from_point(vertex)
194+
vertices.append((lat_lng.lng().degrees, lat_lng.lat().degrees)) # (lon, lat)
195+
196+
return Polygon(vertices)
197+
198+
df['geometry'] = df['s2_15'].apply(s2id_to_polygon)
199+
gdf = gpd.GeoDataFrame(df, geometry='geometry', crs='EPSG:4326') # WGS84
200+
gdf.to_parquet('sample_tif_s2_geoparquet.parquet')
201+
```
202+
</details>
203+
204+
<details>
205+
<summary>For Geohash output...</summary>
206+
207+
For Geohash output, you can use [`python-geohash`] or other similar Geohash library. Example:
208+
209+
```python
210+
import pandas as pd
211+
import geohash
212+
from shapely.geometry import Point, box
213+
import geopandas as gpd
214+
o = pd.read_parquet('./tests/data/output/8/sample_geohash')
215+
216+
217+
def geohash_to_geometry(gh, mode="polygon"):
218+
lat, lon, lat_err, lon_err = geohash.decode_exactly(gh)
219+
220+
if mode == "point":
221+
return Point(lon, lat)
222+
elif mode == "polygon":
223+
return box(lon - lon_err, lat - lat_err, lon + lon_err, lat + lat_err)
224+
else:
225+
raise ValueError("mode must be 'point' or 'polygon'")
226+
227+
o["geometry"] = o["geohash_08"].apply(lambda gh: geohash_to_geometry(gh, mode="polygon"))
228+
229+
'''
230+
band geohash_08 1 2 3 geometry
231+
0 u170f2sq 0 0 0 POLYGON ((4.3238067626953125 52.16686248779297...
232+
1 u170f2sr 0 0 0 POLYGON ((4.3238067626953125 52.16703414916992...
233+
2 u170f2sw 0 0 0 POLYGON ((4.324150085449219 52.16686248779297,...
234+
3 u170f2sx 0 0 0 POLYGON ((4.324150085449219 52.16703414916992,...
235+
4 u170f2sy 0 0 0 POLYGON ((4.324493408203125 52.16686248779297,...
236+
... ... .. .. .. ...
237+
232720 u171mc2g 0 0 0 POLYGON ((4.472808837890625 52.258358001708984...
238+
232721 u171mc2h 0 0 0 POLYGON ((4.471778869628906 52.25852966308594,...
239+
232722 u171mc2k 0 0 0 POLYGON ((4.4721221923828125 52.25852966308594...
240+
232723 u171mc2s 0 0 0 POLYGON ((4.472465515136719 52.25852966308594,...
241+
232724 u171mc2u 0 0 0 POLYGON ((4.472808837890625 52.25852966308594,...
242+
243+
[232725 rows x 5 columns]
244+
'''
245+
246+
gdf = gpd.GeoDataFrame(o, geometry="geometry", crs="EPSG:4326")
247+
gdf.to_file('sample.gpkg')
248+
```
249+
</details>
146250

147251
## Installation
148252

@@ -196,7 +300,7 @@ Two sample files have been uploaded to an S3 bucket with `s3:GetObject` public p
196300
- `s3://raster2dggs-test-data/Sen2_Test.tif` (sample Sentinel 2 imagery, 10 bands, rectangular, Int16, LZW compression, ~10x10m pixels, 68.6 MB)
197301
- `s3://raster2dggs-test-data/TestDEM.tif` (sample LiDAR-derived DEM, 1 band, irregular shape with null data, Float32, uncompressed, 10x10m pixels, 183.5 MB)
198302

199-
You may use these for testing. However you can also test with local files too, which will be faster.
303+
You may use these for testing. However you can also test with local files too, which will be faster. A good, small (5 MB) sample image is available [here](https://github.com/mommermi/geotiff_sample).
200304

201305
## Example commands
202306

conda_dev.yml

Lines changed: 0 additions & 25 deletions
This file was deleted.

0 commit comments

Comments
 (0)