Skip to content

Commit 4da9225

Browse files
ryan-williamsclaude
andcommitted
daily: pre-pull downstream cascade targets (now safe — dvx fixes landed)
Re-applies [3ead452] (reverted in [1c31c9d] when S3 was missing dir-output inner blobs). Two upstream dvx fixes make this safe: - [099855640] (dvx) `cache_blob` handles directory outputs — manifests now make it to local cache during `dvx run`, so subsequent pushes have something to upload. - [9edbef726] (dvx) `dvx push` (and `dvx run --push each|end`) walks dir manifests and backfills inner blobs missing from remote. Heals existing broken S3 state on the next run. Locally verified after manually backfilling `ymccmc`'s 17 missing inner blobs (`aws s3 cp` from `.dvc/cache/files/md5/…` per the spec workaround) — fresh-clone `dvx pull` now fetches all 4 targets (46 files, 44 added). Bump dvx pin a180cda1e → 9edbef726. [3ead452]: 3ead452e550 [1c31c9d]: 1c31c9d73b4 [099855640]: runsascoded/dvx@099855640 [9edbef726]: runsascoded/dvx@9edbef726 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 1c31c9d commit 4da9225

3 files changed

Lines changed: 10 additions & 4 deletions

File tree

.github/workflows/daily.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,12 @@ jobs:
8686
dvx pull www/public/njsp/crashes.db.dvc
8787
# Side-effect output of `njsp match_njdot`; needed by supplement_with_sp
8888
dvx pull njsp/data/njsp_njdot_residuals.parquet.dvc
89+
# Downstream parquets/dbs — pre-stage so `is_output_fresh` sees them on
90+
# disk and reports `up-to-date` instead of "output missing" → re-execute.
91+
dvx pull njdot/data/aashto_supplemented_crashes.parquet.dvc
92+
dvx pull www/public/njdot/cmymc.db.dvc
93+
dvx pull www/public/data/njdot/ymccmc.dvc
94+
dvx pull www/public/data/njdot/ymccmcs.dvc
8995
dvx pull www/public/Municipal_Boundaries_of_NJ.geojson
9096
- name: DVX status
9197
run: dvx status

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ dependencies = [
1616
"dask>=2024",
1717
"dotenv>=0.9.9",
1818
"dvc[s3]>=3.59",
19-
"dvx @ git+https://github.com/runsascoded/dvx@a180cda1e",
19+
"dvx @ git+https://github.com/runsascoded/dvx@9edbef726",
2020
"geopandas>=1.0",
2121
"geopy>=2.4",
2222
"GitPython>=3.1",

uv.lock

Lines changed: 3 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)