Herbie v2 API (Major refactor/rewrite)

[blaylockbk]

I've learned a thing or two about Python since I started Herbie. I am also learning how to use Claude, which has helped me experiment with and implement some of my ideas a lot faster than I can code by hand.

In this branch I'm trying out some new ideas to better organize Herbie, make it easier to maintain, perform better, and be more ergonomic to the user. It's still a work in progress. Let's consider this PR a preview of things to come.

I'm building the v2 API as an opt-in feature. You can keep using the classic Herbie API the same you did before, but you can also use the new v2 API. I expect that over time the v2 API will become the preferred API as more models are added and refined, while continued development for the classic API will reduce until it's not used anymore.

Install

uv add herbie-data[v2]

Usage

from herbie.v2 import Herbie

H = Herbie.HRRR('2026-01-01', product='sfc', step=6)

# Same as before
H.inventory(search)
H.download(search)
H.xarray(search)

# New!
# Herbie instantiation is lazy (only knows where to look for files and doesn't know what exists)
# until you invoke inventory, download, or xarray, or the new resolve method.
H.resolve()         # New! Find a file that exists.
H.resolve('all')    # New! Check all sources for existing files.
H.status()          # New! Show which source was check and which has an existing file

You may also import model classes directly

from herbie.v2 import HRRR, GFS, IFS

HRRR('2025-01-01')

• Each model is now its own class, accessed through the Herbie namespace or imported directly.
• Use Polars to load and parse inventory files instead of Pandas.
• fxx is renamed step
• A Herbie object is now lazy at instantiation. Herbie doesn't check if a file exist until .resolve(), .download(), .inventory(), or .xarray() is used.
• subsetted downloads happen in parallel, and done better.
• FastHerbie downloads all desired fields into a single grib2 file instead of downloading many subsets. This is very different behavior from before, but aligns with my original intent of the function to "make your own GRIB file."

  from herbie.v2 import FastHerbie, HRRR
  FastHerbie('2025-01-01', step=range(7), model=HRRR)

• Have .latest(valid_time=None, step=None, source='aws', **kwargs, max_tdelta=timedelta(days=1) as a HerbieModel class method which locates the latest available dataset from the specified list of sources. Loops through datetimes backwards from requested valid_time until at known initialization increment until one is found.

Zarr datasets

While Herbie has been mainly focused on getting data from GRIB sources, I'd like to extend this to access other datasets, like Zarr. I'm thinking of using a class method to target Zarr archives...

from herbie.v2 import HRRR

HRRR.from_zarr('dynamical', product='forecast') # https://dynamical.org/catalog/noaa-hrrr-forecast-48-hour/
HRRR.from_zarr('dynamical', product='analysis') # https://dynamical.org/catalog/noaa-hrrr-analysis/

[blaylockbk]

Some very basic downloa benchmarks

herbie.v2

$ uv run download_timer_v2.py 
                                        
Download Summary:
model='hrrr', date=datetime.date(2026, 5, 4), search='TMP:'

google     █                               1.781 s  (7 fields, 5.27 MB)
nomads     █                               1.782 s  (7 fields, 5.27 MB)
pando2     ██▌                             1.965 s  (7 fields, 5.27 MB)
pando      ████████████████                3.287 s  (7 fields, 5.27 MB)
azure      ██████████████████              3.458 s  (7 fields, 5.27 MB)
aws        ██████████████████████████████  4.639 s  (7 fields, 5.27 MB)

herbie (legacy)

$ uv run download_timer.py 
                                        
Download Summary:
model='hrrr', date=datetime.date(2026, 5, 4), search=':TMP:'

nomads     █                               1.769 s  (7 fields, 5.27 MB)
google     ████▌                           2.671 s  (7 fields, 5.27 MB)
pando2     ███████                         3.168 s  (7 fields, 5.27 MB)
pando      ██████████                      3.919 s  (7 fields, 5.27 MB)
aws        ████████████████████████▌       7.242 s  (7 fields, 5.27 MB)
azure      ██████████████████████████████  8.413 s  (7 fields, 5.27 MB)

The Version 2 parallel downloads are a bit faster.