Skip to content

Commit e233ba1

Browse files
authored
Move scripts/iris/cross_region_ops and scripts/storage into scripts/ops/ (#4965)
Consolidate operational scripts under a new scripts/ops/ directory. Moves scripts/iris/cross_region_ops.py to scripts/ops/cross_region.py and scripts/storage/ to scripts/ops/storage/. Updates all internal path references in docstrings, help text, and README.
1 parent c8e51f9 commit e233ba1

15 files changed

Lines changed: 30 additions & 30 deletions
Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ prep ──> server (human triage) ──> compute (manifest) ──> clea
2424

2525
## Architecture
2626

27-
All state lives in `scripts/storage/purge/`:
27+
All state lives in `scripts/ops/storage/purge/`:
2828

2929
```
3030
purge/
@@ -57,16 +57,16 @@ are persisted as JSON files and loaded into DuckDB tables on startup.
5757

5858
```bash
5959
# See step status
60-
uv run scripts/storage/prep.py plan
60+
uv run scripts/ops/storage/prep.py plan
6161

6262
# Run all prep steps
63-
uv run scripts/storage/prep.py run
63+
uv run scripts/ops/storage/prep.py run
6464

6565
# Run a single step
66-
uv run scripts/storage/prep.py prep scan-objects --scan-workers 64
66+
uv run scripts/ops/storage/prep.py prep scan-objects --scan-workers 64
6767

6868
# Force re-scan (ignore cached markers)
69-
uv run scripts/storage/prep.py run --force
69+
uv run scripts/ops/storage/prep.py run --force
7070
```
7171

7272
### Steps
@@ -92,14 +92,14 @@ pricing and a 30% CUD discount.
9292

9393
```bash
9494
# Local development
95-
uv run scripts/storage/server.py --dev
95+
uv run scripts/ops/storage/server.py --dev
9696

9797
# Production (deployed via Docker)
98-
uv run scripts/storage/deploy.py deploy
98+
uv run scripts/ops/storage/deploy.py deploy
9999
```
100100

101101
The dashboard is a FastAPI app serving a Vue frontend from
102-
`scripts/storage/dashboard/`. It provides:
102+
`scripts/ops/storage/dashboard/`. It provides:
103103

104104
- **Overview** (`/api/overview`) — Per-region storage breakdown by class with
105105
monthly cost estimates.
@@ -133,13 +133,13 @@ staging prefix that the container downloads on cold start.
133133

134134
```bash
135135
# See step status
136-
uv run scripts/storage/compute.py plan
136+
uv run scripts/ops/storage/compute.py plan
137137

138138
# Compute the deletion manifest
139-
uv run scripts/storage/compute.py run
139+
uv run scripts/ops/storage/compute.py run
140140

141141
# Force recompute (e.g. after changing rules)
142-
uv run scripts/storage/compute.py run --force
142+
uv run scripts/ops/storage/compute.py run --force
143143
```
144144

145145
Evaluates delete rules against protect rules and the `dir_summary` table,
@@ -166,13 +166,13 @@ Inspect with standard tools:
166166

167167
```bash
168168
# Quick look
169-
column -t -s, scripts/storage/purge/deletion_manifest.csv | head -20
169+
column -t -s, scripts/ops/storage/purge/deletion_manifest.csv | head -20
170170

171171
# Sort by size
172-
sort -t, -k4 -rn scripts/storage/purge/deletion_manifest.csv | head
172+
sort -t, -k4 -rn scripts/ops/storage/purge/deletion_manifest.csv | head
173173

174174
# Filter to one bucket
175-
grep marin-eu-west4 scripts/storage/purge/deletion_manifest.csv
175+
grep marin-eu-west4 scripts/ops/storage/purge/deletion_manifest.csv
176176
```
177177

178178
### Steps
@@ -187,16 +187,16 @@ validation.
187187

188188
```bash
189189
# See step status
190-
uv run scripts/storage/cleanup.py plan
190+
uv run scripts/ops/storage/cleanup.py plan
191191

192192
# Dry run (no mutations)
193-
uv run scripts/storage/cleanup.py run --dry-run
193+
uv run scripts/ops/storage/cleanup.py run --dry-run
194194

195195
# Execute cleanup
196-
uv run scripts/storage/cleanup.py run
196+
uv run scripts/ops/storage/cleanup.py run
197197

198198
# Run a single step
199-
uv run scripts/storage/cleanup.py run --only cleanup.delete_cold_objects
199+
uv run scripts/ops/storage/cleanup.py run --only cleanup.delete_cold_objects
200200
```
201201

202202
### Steps
Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
populated by the purge workflow.
99
1010
Usage:
11-
uv run scripts/storage/server.py [--port 8000] [--dev]
11+
uv run scripts/ops/storage/server.py [--port 8000] [--dev]
1212
"""
1313

1414
from __future__ import annotations
@@ -1418,7 +1418,7 @@ def main(port: int, host: str, dev: bool) -> None:
14181418
host=host,
14191419
port=port,
14201420
reload=True,
1421-
reload_dirs=["scripts/storage"],
1421+
reload_dirs=["scripts/ops/storage"],
14221422
log_level="debug",
14231423
access_log=True,
14241424
)
Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,10 @@
1919
to head/wc/awk without buffering the full delete set.
2020
2121
Usage:
22-
uv run scripts/storage/delete_files.py \\
22+
uv run scripts/ops/storage/delete_files.py \\
2323
--scan-parquet gs://marin-us-central2/storage-scan/YYYY-MM-DD/ \\
24-
--protect-rules scripts/storage/purge/protect_rules.json \\
25-
--delete-rules scripts/storage/purge/delete_rules.json \\
24+
--protect-rules scripts/ops/storage/purge/protect_rules.json \\
25+
--delete-rules scripts/ops/storage/purge/delete_rules.json \\
2626
[--dry-run]
2727
"""
2828

0 commit comments

Comments
 (0)