pyIEM 1.26.0 Release

API Changes

• Enforce a SHEF data model limit of 8 characters for a station identifier.
• Ween off domain= nomenclature for the default IEMRE domain.

New Features

• Add an IEMRE CONUS alias that points back to the default domain="".
• Add IEM autoplot alias option to allow for legacy variable migration.

Bug Fixes

• Account for a corrupted HRRR bufkit file found in MTArchive.
• Correct LSR location parsing for Guam and Pago Pago.

pyIEM 1.25.0 Release

API Changes

• Change NoDataFound exception in iemapp to HTTP status 422.
• Mark pyiem.util.{get_dbconn*,get_sqlalchemy_conn} for removal in 1.26.0
• Remove pyiem.plot.use_agg.plt (pyplot) as it is a walking memory leak.
• Remove pyiem.prism per previous deprecation.
• Rename pyiem.nws.products.vtec.skip_con to is_skip_con (meh, internal).
• Support TAF Collectives by changing prod.data: TAFReport to
  prod.data: list[TAFReport] (#913).

New Features

• Add method=head option to archive_fetch so to do quick checks.
• Drop nh3 requirement.
• Support BECMG, PROB30, and PROB40 TAF forecast elements (#1104).
• Support more TAF quirks and poor formatting.

Bug Fixes

• Account for python-metar edge case with peak wind time being in the future.
• Correct TAF parsing edge case when segment has no equals.
• Demote a SHEF warning for a poorly encoded paired depth value.
• Fix edge case with empty scripts in autoplot vtecps type.
• Prevent unknown UGC from causing a full database write failure (#1097).
• Rework XSS detection within webutil.iemapp.
• Update bundled NWS CWAs (18 Mar 2025) (#1100).
• Workaround a spacing issue within LSRs that may not be fixed upstream.

What's Changed

• Demote SHEF warning about negative depth by @akrherz in #1093
• Forgive LSR spacing issue by @akrherz in #1094
• 🐛 Account for python-metar edge case by @akrherz in #1096
• 🐛 Prevent unknown UGC to prevent db write failure by @akrherz in #1098
• 💥 Remove pyiem.plot.use_agg.plt by @akrherz in #1099
• 📝 Update bundled CWA geodf by @akrherz in #1101
• 🎨 Use default value for empty vtecps string by @akrherz in #1102
• ✅ Add test for old-bug not currently present by @akrherz in #1103
• ✨ Support PROB30,PROB40,BECMG TAF Fields by @akrherz in #1105
• 🐛 Address TAF edge parsing case by @akrherz in #1106
• ✅ Add test to ensure TAF processing by @akrherz in #1107
• 🎨 Account for additional TAF edge case by @akrherz in #1108
• 🎨 Address some typing lint by @akrherz in #1109
• 🔥 Drop is_tempo database storage by @akrherz in #1110
• TAF QoL by @akrherz in #1111
• 💥 Support TAF Collectives by @akrherz in #1112
• 🎨 Allow for TAF AMD by @akrherz in #1113
• 🎨 Allow for more TAF one-offs by @akrherz in #1114
• 🐛 Address RWIS iemid change by @akrherz in #1115
• Bump actions/checkout from 4 to 5 in the github-actions group by @dependabot[bot] in #1117
• PRISM cleanups by @akrherz in #1118
• 🚧 Wordsmithing VTEC Jabber Messages for #1119 by @akrherz in #1120
• ✨ Add method=head to archive_fetch by @akrherz in #1121
• Stop needless smtp connect by @akrherz in #1122
• 🐛 Rework XSS detection in iemapp by @akrherz in #1123
• 🐛 Account for XSS edge case by @akrherz in #1124
• 📌 Debug transient CI failure by @akrherz in #1125
• 🗑️ Mark pyiem.util database stuff for removal by @akrherz in #1126
• 🐛 [taf] Fix parse logic error when no = by @akrherz in #1127

Full Changelog: v1.24.0...v1.25.0

pyiem 1.24.0 Release

The package name is clarified as lowercase pyiem, will see how much trouble that causes :/

API Changes

• Bundled IEM website template migrated to Bootstrap 5.
• Define PRISM800 grid navigation for updated PRISM 800m grid.
• Mark pyiem.prism for removal in next release.
• Move get_product_id() up one class level to WMOProduct.
• Reclassify LSR LANDSLIDE typetext to code x (debris flow).
• Reclassify LSR SNOW/ICE DMG typetext to code 5 (ice related).
• Update LSR web links within Jabber messages per new
  IEM LSR App.

New Features

• Add iailmo (Iowa + Illinois + Missouri) MapPlot sector.
• Add sa IEMRE domain per DEP needs akrherz/iem#1173.
• Bundle a docutils based CSS to improve the iemapp help rendered HTML.
• Bundle GeoDataframes of discontinued NWS UGC Counties/Zones. Add
  discontinued=bool option to MapPlot.fill_ugcs() to enable.
• Forgive encoded space as + in datetime to autoplot context.
• Introduce appmode flag on IEM template to control default CSS/JS inclusion.
• Introduce pyiem.ncei.ghcnh to process NCEI GHCNh data.
• Introduce pyiem.ncei.igra to process NCEI IGRA2.2 Data (#1036).
• Introduce pyiem.nws.products.metar_util with metar_from_dict helper to
  create METAR strings from dicts of data.
• Remove null byte 0x00 from WMO Products.
• Support new MCD most probabe tags (#1042).
• VTEC events are now created in the unfortunate case of a missed expansion
  or creation product in the time series. This is a life choice made in the
  face of increasing NWS dissemination problems.

Bug Fixes

• Account for empty strings in vtec_ps autoplot form type.
• Correct fontscale calculation.
• Correct offset calculation in SIGMETs.
• Correct timezone offsets for some Atlantic timezones.
• Correct web links for SPC day 2,3 convective outlook.
• Correct metadata for a number of NCEI IGRA2 sites.
• Correct SIGMET offset computation to use nautical miles.
• Fix appending product_id to IEM Access summary table for DSM parsing.
• Fix CWA parsing issue when lat/lon box is found.
• Fix SPC Day2 outlook link.
• Implement better conditioning on inbound text for WMOProduct#unixtext
• Improve multi-line PIREP report join when there is no front spaces.
• Update NCEP MRMS base URL for real-time downloads.
• Use text product issuance time, in leiu of ambiguity with invalid VTEC
  series (ie issuance was missing).

pyIEM 1.23.0

API Changes

• Drop summary database table processing of max_tmpf_qc, min_tmpf_qc,
  pday_qc, and snow_qc. These are ill-designed and unused.
• Drop poorly designed iemdb support within webutil.iemapp.
• Internal refactor of WMOProduct timestamp processing in parent class.

New Features

• Add text pattern support within get_autoplot_context.
• Bundle NWS AWIPS geodatabase valid 18 March 2025 (#1017).
• Introduce database.sql_helper as a hacky attempt to ease my ignorance
  with psycopg + sqlalchemy + pandas usage.
• Introduce database.with_sqlalchemy_conn decorator helper.
• Introduce util.ddhhmm2datetime helper to convert a WMO header timestamp
  to a UTC timestamp.
• Support decorated webutil.iemapp functions that return generators.
• Write DSM product_id to IEMAccess summary table.

Bug Fixes

• Add color for Cold Weather Advisory (AFEEEE).
• Constrain FEMA Region 9 to CA, NV, AZ (#1007).
• Improve solar radiation summary table update within IEMAccess.
• Prevent UGCProvider from constantly reloading from database (#1010).
• Skip PDS cross-check for watches from Hawaii.
• Wordsmith message for Snow Squall Warnings (#1013).

pyIEM 1.22.1

API Changes

• Map runner user to mesonet for CI database support.
• Refactored PIREP data model to pyiem.models.pirep.

New Features

• Add nwsice color ramp per suggestion.
• Retain reference to CGIModel schema as environ["_cgimodel_schema"].
• Parse flight level within PIREPs (#1003).
• Use akrherz/iem_database:test_data container for integration tests.

Bug Fixes

• Attempt more robust NWS product headline gleaning.

pyIEM 1.22.0 Release

API Changes

• Add pyiem.reference.StationAttributes class to hold database attribute
  keys used.
• Discontinue raw SPS product text within the database.
• Ensure raw text products into WMOProduct end with a line feed.
• Move VTEC SE phenomena out of the default WFO jabber channels.
• Replace requests usage with httpx.
• Stop postprocessing the MapPlot figure into 8bit colorspace.

New Features

• Add allowed_as_list option to iemapp() helper to stop lists.
• Add pyiem.grid.nav with IEM grid information in a fancy form.
• Add MapPlot.imshow with some optimized panel plotting.
• Add maximum risk threshold within SPC outlook message (#969).
• Add pyiem.era5land with IEM grid reference information.
• Add pyiem.stage4 with grid reference information.
• Add support for plotting by FEMA Regions.
• Assign base WFO jabber channel to Tsunami Warnings (#978).
• Improvements for IEM netcdf grid navigation and handling.
• Include simple table of un-plotted states for MapPlot(sector="nws") #967.
• Introduce radar_ptype color ramp and draw_radar_ptype_legend for
  generating plots of HRRR ptype.

Bug Fixes

• Accomodate ancient LSRs using TRACE as the magnitude field.
• Add nounce to LSR autoplot link to prevent some ugliness around a NWS issue.
• Correct MapPanel GIS worldfile logic as upper left is center of grid cell.
• Ensure geometries going into masking helper are CCW, to mask outside of.
• Improve dev experience using ugc.UGCProvider (#980).
• Fix grid and affine definitions for IEMRE and MRMS.
• Properly check USDM service response prior to parsing it.
• Refine which MWW products are not defaulted into the main WFO Channels,
  those being VTEC codes SC, MF, and GL.
• Require valid wind speed and direction values going into windrose_utils.
• Support parsing CLI products circa 2007 with a bad space.
• Use more exact grid naviation for pyiem.prism, fixes off-by-one.

pyIEM 1.21.0 Release

API Changes

• Discontinue persisting text product into {mcd,mpd} table storage.
• Discontinue storage of text product into sigmets, use product_id instead.
• Raise IncompleteWebRequest exception when autoplot datetime parsing fails.

New Features

• Add _check_dueling_tropics VTEC check looking for TR+HU overlap (#930).
• Add IEMRE DOMAINS to support upcoming expansion.
• Add pyiem.grid.util.grid_smear to shift masked data to fill in missing.
• Allow iemapp(memcachexpire) to be a callable, called back with environ.
• Cross check WCNs against watch database storage for PDS status (#925).
• For iemapp if the memcachekey= callable returns None, the check of
  memcache is short circuited.
• Store product_id for PIREPs and do some faked AFOS assignments to help.
• Support SPC afternoon Day 3 outlook (cycle assigned as 20) (#936).
• Support VTEC significance addition of Extreme Heat (XH) (#953).

Bug Fixes

• Fix and refactor SPC saw parsing of watch replacement number.
• Pop kwargs fig on MapPlot.
• Use less confusing landing page at WPC for ERO.
• Use round instead of int to compute zonal stats grid navigation.

pyIEM 1.20.0 release

API Changes

• Change storage logic of SBW to not include CAN polygons in the case of
  a CAN/CON combination update (#888).
• Depend on docutils for reStructedText to HTML conversion.
• Don't generate jabber messages for LSR products with more than 4 LSRs (#901).
• util.get_autoplot_context can now raise IncompleteWebRequest.
• The VTEC/SBW database storage of raw NWS product text was discontinued.
  Instead, raw product_id values are stored, which can be used against IEM
  web services to get the raw text. (#857)

New Features

• Account for common AM/PM typos and off-by-one year timestamps in NWS Prods.
• Add backgroundcolor option to MapPlot.plot_values.
• Add comprehensive_climate_index and temperature_humidity_index.
• Add imgsrc_from_row helper for SPC outlook link generation.
• Add parse_commas option to webutil.ensure_list, so to allow comma
  delimited CGI params by default.
• For pydantic schema based iemapp, keys like wfo[] go to wfo.
• Introduce some pydantic based validators for web requests, experimental...
• Store product_signature in sbw table (requires iem-database schema update).
• Support SPC corrections for MCDs so to prevent database dups.

Bug Fixes

• Allow for pydantic field validation to work when there's legacy year vs
  year1 present.
• Correct generated LSR summary link to have timestamps in UTC.
• Correct handling of is_emergency when done within VTEC correction (#899).
• Correct SKC parsing within TAFs, level is now None in this case (#453).
• Don't allow CF6 data from "today" or the "future" to be parsed, if missing.
• Fix numeric instability with pyiem.plot.centered_bins (#871).
• Forward propagate sbw database storage of VTEC issuace (#862).
• Handle edge case with NWS text product having a polygon at the end without
  a trailing newline (ancient text).
• Improve NWS Text Product signature logic so to more generally match what
  looks like a signature (#865).
• Improved NWS MND header logic for timestamp retrieval.
• Make better life choices attempting to glean a NWS Text Product signature.
• Prevent situation in sbw database having polygon_begin > polygon_end (#862).
• Refactor VTEC/SBW storage logic to use vtec_year column and operate on the
  parent table (#863).
• Update link to new NWPS website.

pyIEM 1.19.0 Release

API Changes

• Drop pyiem.nwnformat as it is no longer used.
• Introduce pyiem.wmo.WMOProduct as a lightweight class for products that
  need little default processing. pyiem.nws.product.TextProduct inherits.
• Remove import * usage from pyiem.plot. This was a bad daryl mistake.
• Require pygrib as we add some grib processing.
• Require pyogrio for faster geopandas shapefile reading.

New Features

• Add pyiem.iemre.grb2iemre helper to reproject a grib message onto IEMRE.
• Add pyiem.util.archive_fetch helper to get IEM archived resources.
• Add VTEC storage of product_ids (#857).
• Sync NWS VTEC colors per IEM database cross-check review.
• Update bundled NWS zones database to 5 March 2024 release (#821).

Bug Fixes

• Account for a common autoplot context timestamp typo.
• Add a hacky gc.collect() within MapPlot#close attempting to workaround
  some matplotlib memory leaking.
• Constrain a VTEC database search for expiration times not infinitely into
  the future.
• Fix iemre.reproject2iemre to return a masked_array and handle an input
  masked array.
• Fix invalid rtrim() usage with removesuffix().
• Fix off-by-one when grid.zs hits a mask right at the border.
• Fix placement of GU, AK, HI, PR inset axes for sector="nws" plot.
• Improve setuptools_scm plumbing to avoid runtime import.
• Remove hackish website telemetry writing from a thread.
• Support a variant Trace value specified in LSRs.

pyIEM 1.18.0 Release

API Changes

• Add dependencies on nh3 and paste.
• Raise NewDatabaseConnectionFailure from pyiem.database.get_dbconn for
  all failures and stop emitting UserWarning.
• Raise UnknownStationException from pyiem.util.get_autoplot_context when
  an unknown station is provided.
• Repurpose LSR type internal code q for SNOW SQUALL.

New Features

• Add #{state}wx hash tags to NWS Local Storm Report social media posts.
• Add cursor_name option to database.get_dbconnc.
• Add pyiem.iemre.reproject2iemre helper to bring arbitrary grids onto IEMRE.
• Add pyiem.util.{delete,set}_property helpers.
• Introduce pyiem.webutil.iemapp decorator to do fancy things for IEM
  mod_wsgi apps and remove downstream boilerplate.

Bug Fixes

• [SHEF] Lines startig with ":" are double checked for slashes, before use.
• [SHEF] Ensure E message elements have a physical_element set.
• [SHEF] Fix DD modifier when we are crossing a month.
• [SHEF] Fix significant bug with Paired Values (ref SHEF Manual 7.4.6) not
  properly computing magnitude for encoded float values.
• [SHEF] Forgive a random nan that keeps appearing in RRSRAH.
• [SHEF] Prevent extraneous "/" added to contination .A and .E messages.
• [SHEF] Support 0.001 encoding of Trace values.
• Allow for calm percentage plotting space in windrose with rmax set (#798).
• Fix compute_wfos method for SAW processor.
• Make presence of tabs in CLI products more forgiving.
• Refactor SHEF parsing such to better handle situations with time modifiers
  being missing and enforce SHEFElement pydantic validation.