Skip to content

Commit 637c027

Browse files
jenningsandersonjenningsanderson
committed
uv-ification
1 parent 0a77bbc commit 637c027

12 files changed

Lines changed: 754 additions & 13 deletions

‎.gitignore‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Python
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
*.so
6+
.Python
7+
build/
8+
develop-eggs/
9+
dist/
10+
downloads/
11+
eggs/
12+
.eggs/
13+
lib/
14+
lib64/
15+
parts/
16+
sdist/
17+
var/
18+
wheels/
19+
share/python-wheels/
20+
*.egg-info/
21+
.installed.cfg
22+
*.egg
23+
MANIFEST
24+
25+
# Virtual environments
26+
.env
27+
.venv
28+
env/
29+
venv/
30+
ENV/
31+
env.bak/
32+
venv.bak/
33+
34+
# PyCharm
35+
.idea/
36+
37+
# VS Code
38+
.vscode/
39+
*.code-workspace
40+
41+
# Testing
42+
.pytest_cache/
43+
.coverage
44+
htmlcov/
45+
.tox/
46+
.nox/
47+
coverage.xml
48+
*.cover
49+
*.log
50+
51+
# MyPy
52+
.mypy_cache/
53+
.dmypy.json
54+
dmypy.json
55+
56+
# Ruff
57+
.ruff_cache/
58+
59+
# Jupyter Notebook
60+
.ipynb_checkpoints
61+
62+
# pyenv
63+
.python-version
64+
65+
# UV
66+
.uv/
67+
68+
# Distribution / Packaging
69+
*.whl
70+
71+
# Output directories
72+
public_releases/
73+
output/
74+
75+
# Misc
76+
.DS_Store
77+
*.swp
78+
*.swo
79+
*~

‎README.md‎

Lines changed: 115 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,30 @@
11
# Overture STAC
2-
Generate STACs for all public Overture releases
32

4-
See it in action here:
5-
https://radiantearth.github.io/stac-browser/#/external/labs.overturemaps.org/stac/catalog.json?.language=en
3+
Generate STAC (SpatioTemporal Asset Catalog) catalogs for all public Overture Maps releases.
64

7-
The structure looks like this:
5+
See it in action here:
6+
<https://radiantearth.github.io/stac-browser/#/external/labs.overturemaps.org/stac/catalog.json?.language=en>
7+
8+
## Installation
9+
10+
This project uses [uv](https://github.com/astral-sh/uv) for dependency management. If you don't have it installed:
11+
12+
```bash
13+
# Install uv
14+
curl -LsSf https://astral.sh/uv/install.sh | sh
15+
```
16+
17+
### Install the package
18+
19+
```bash
20+
# Install in development mode with all dev dependencies
21+
uv pip install -e ".[dev]"
22+
23+
# Or install just the package
24+
uv pip install -e .
25+
```
26+
27+
The structure looks like this:
828

929
```
1030
- catalog.json
@@ -20,7 +40,8 @@ The structure looks like this:
2040
| - 00002.json
2141
```
2242

23-
The top-level `catalog.json` intends to be a catalog of all publicly available Overture releases. Briefly, it looks like this:
43+
The top-level `catalog.json` intends to be a catalog of all publicly available Overture releases. Briefly, it looks like this:
44+
2445
```json
2546
{
2647
"type": "Catalog",
@@ -40,31 +61,113 @@ The top-level `catalog.json` intends to be a catalog of all publicly available O
4061
"href": "./2025-06-25.0/catalog.json",
4162
"type": "application/json",
4263
"title": "2025-06-25.0 Overture Release"
43-
},
64+
}
4465
],
4566
"latest": "2025-07-23.0"
4667
}
4768
```
48-
The top level catalog points to the `latest` Overture release, and this release also has the tag `latest:true`.
69+
70+
The top level catalog points to the `latest` Overture release, and this release also has the tag `latest:true`.
4971

5072
## Additional Files
73+
5174
At the root of each release, there are two additional files: `manifest.geojson` and `collections.parquet`
75+
5276
```
5377
- <RELEASE>/
5478
| - manifest.geojson
5579
| - collections.parquet
5680
```
5781

58-
#### `manifest.geojson`: Basic GeoJSON Manifest
82+
#### `manifest.geojson`: Basic GeoJSON Manifest
83+
84+
To support the download functionality of `explore.overturemaps.org`, a basic `manifest.geojson` summary of the distribution is available at the root of a release.
85+
86+
#### `collections.parquet`: STAC GeoParquet
87+
88+
An Overture release is composed of nearly 500 individual parquet files, and therefore the STAC index is composed of nearly 500 individual `json` files. This single `collections.parquet` is created by the [`stac-geoparquet`](https://github.com/stac-utils/stac-geoparquet) utility.
5989
To support the download functionality of `explore.overturemaps.org`, a basic `manifest.geojson` summary of the distribution is available at the root of a release.
6090

61-
#### `collections.parquet`: STAC GeoParquet
91+
#### `collections.parquet`: STAC GeoParquet
92+
6293
An Overture release is composed of nearly 500 individual parquet files, and therefore the STAC index is composed of nearly 500 individual `json` files. This single `collections.parquet` is created by the [`stac-geoparquet`](https://github.com/stac-utils/stac-geoparquet) utility.
6394

95+
## Usage
96+
97+
### Command Line
6498

65-
## Running
99+
After installation, you can use the `gen-stac` command:
66100

67101
```bash
68-
pip install -r requirements
69-
python3 gen-all-release-stac.py`
102+
# Generate STAC catalogs for all releases
103+
gen-stac --output ./public_releases
104+
105+
# Run in debug mode (generates only 1 item per collection)
106+
gen-stac --output ./public_releases --debug
107+
```
108+
109+
### Python API
110+
111+
You can also use the package programmatically:
112+
113+
```python
114+
from pathlib import Path
115+
from overture_stac import OvertureRelease, RegistryManifest
116+
117+
# Generate STAC catalog for a specific release
118+
release = OvertureRelease(
119+
release="2025-07-23.0",
120+
schema="1.11.0",
121+
output=Path("./output"),
122+
)
123+
release.build_release_catalog(title="My Release")
124+
125+
# Create registry manifest
126+
registry = RegistryManifest()
127+
manifest_data = registry.create_manifest()
128+
print(f"Found {len(manifest_data)} files in registry")
129+
```
130+
131+
## Development
132+
133+
### Running Tests
134+
135+
```bash
136+
# Run all tests
137+
pytest
138+
139+
# Run with coverage
140+
pytest --cov=overture_stac --cov-report=html
141+
142+
# Run specific test file
143+
pytest tests/test_registry_manifest.py
144+
```
145+
146+
### Code Quality
147+
148+
```bash
149+
# Format code with ruff
150+
ruff format .
151+
152+
# Lint code
153+
ruff check .
154+
155+
# Fix linting issues automatically
156+
ruff check --fix .
157+
```
158+
159+
## Project Structure
160+
161+
```
162+
stac/
163+
├── src/overture_stac/ # Main package
164+
│ ├── __init__.py # Package initialization
165+
│ ├── cli.py # Command-line interface
166+
│ ├── overture_stac.py # Main STAC generation logic
167+
│ └── registry_manifest.py # Registry manifest generation
168+
├── tests/ # Test suite
169+
│ ├── test_overture_stac.py
170+
│ └── test_registry_manifest.py
171+
├── pyproject.toml # Project configuration
172+
└── README.md
70173
```

‎gen-all-release-stac.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
import pystac
66
import yaml
77
from src.overture_stac import OvertureRelease
8+
from src.registry_manifest import RegistryManifest
89

910
if __name__ == "__main__":
1011

@@ -109,7 +110,9 @@
109110
this_release.release_catalog.extra_fields["latest"] = True
110111
overture_releases_catalog.extra_fields = {"latest": release}
111112

112-
overture_releases_catalog.extra_fields["registry"] = {}
113+
registry_manifest = RegistryManifest()
114+
registry_data = registry_manifest.create_manifest()
115+
overture_releases_catalog.extra_fields["registry"] = registry_data
113116

114117
overture_releases_catalog.normalize_and_save(
115118
root_href=str(output), catalog_type=pystac.CatalogType.SELF_CONTAINED

‎pyproject.toml‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
[project]
2+
name = "overture-stac"
3+
version = "0.1.0"
4+
description = "Generate STAC catalogs for Overture Maps public releases"
5+
authors = [
6+
{name = "Overture Maps Foundation"}
7+
]
8+
readme = "README.md"
9+
license = {text = "MIT"}
10+
requires-python = ">=3.9"
11+
dependencies = [
12+
"pystac>=1.8.4",
13+
"pyarrow>=14.0.1",
14+
"stac-geoparquet>=0.7.0",
15+
"pyyaml>=6.0.2",
16+
]
17+
18+
[project.optional-dependencies]
19+
dev = [
20+
"pytest>=7.4.0",
21+
"pytest-cov>=4.1.0",
22+
"ruff>=0.1.0",
23+
]
24+
25+
[project.scripts]
26+
gen-stac = "overture_stac.cli:main"
27+
28+
[build-system]
29+
requires = ["hatchling"]
30+
build-backend = "hatchling.build"
31+
32+
[tool.hatch.build.targets.wheel]
33+
packages = ["src/overture_stac"]
34+
35+
[tool.pytest.ini_options]
36+
testpaths = ["tests"]
37+
python_files = ["test_*.py"]
38+
python_classes = ["Test*"]
39+
python_functions = ["test_*"]
40+
addopts = [
41+
"--verbose",
42+
"--strict-markers",
43+
"--strict-config",
44+
"--cov=overture_stac",
45+
"--cov-report=term-missing",
46+
"--cov-report=html",
47+
]
48+
49+
[tool.ruff]
50+
line-length = 88
51+
target-version = "py39"
52+
53+
[tool.ruff.lint]
54+
select = [
55+
"E", # pycodestyle errors
56+
"W", # pycodestyle warnings
57+
"F", # pyflakes
58+
"I", # isort
59+
"B", # flake8-bugbear
60+
"C4", # flake8-comprehensions
61+
]
62+
ignore = [
63+
"E501", # line too long (handled by formatter)
64+
]
65+
66+
[tool.ruff.lint.per-file-ignores]
67+
"__init__.py" = ["F401"]
68+
"tests/*" = ["B008"]
69+
70+
[tool.coverage.run]
71+
source = ["src/overture_stac"]
72+
omit = [
73+
"*/tests/*",
74+
"*/__init__.py",
75+
]
76+
77+
[tool.coverage.report]
78+
exclude_lines = [
79+
"pragma: no cover",
80+
"def __repr__",
81+
"raise AssertionError",
82+
"raise NotImplementedError",
83+
"if __name__ == .__main__.:",
84+
"if TYPE_CHECKING:",
85+
]

‎src/__init__.py‎

Whitespace-only changes.

‎src/overture_stac/__init__.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
"""Overture STAC - Generate STAC catalogs for Overture Maps public releases."""
2+
3+
from overture_stac.overture_stac import OvertureRelease
4+
from overture_stac.registry_manifest import RegistryManifest
5+
6+
__version__ = "0.1.0"
7+
8+
__all__ = ["OvertureRelease", "RegistryManifest"]

0 commit comments

Comments
 (0)