Skip to content

Parallelize fits and predictions#231

Merged
swo merged 5 commits into
mainfrom
swo_parallel
Jan 6, 2026
Merged

Parallelize fits and predictions#231
swo merged 5 commits into
mainfrom
swo_parallel

Conversation

@swo

@swo swo commented Jan 5, 2026

Copy link
Copy Markdown
Collaborator
  • Parallelize the fits and predictions over forecast dates
    • Make a output/fits/fit_DATE.pkl for each forecast date (including all models)
    • Make a output/pred/forecast_start=DATE/part-0.parquet
    • Create a completion flag so that the parquets can be accessed Hive-style
  • Silence the jax warnings about GPUs-- we're using CPUs
  • Make diagnostics optional, since they are only workable with a small number of geographies

and refactor the diagnostics script, to account for the new per-forecast
date system
@swo swo marked this pull request as ready for review January 6, 2026 18:10
@swo swo merged commit b1e387d into main Jan 6, 2026
4 checks passed
@swo swo deleted the swo_parallel branch January 6, 2026 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants