- Adds the functionality to kill flows stuck in a running state. (#250)
- Add import_export.py and upgrade_L0_to_v0e.py scripts. (#252)
- Makes flow scheduling faster by cutting the number of DB round-trips, particularly with calibration file selection in level 1. (#259)
- Add a 'no-earlier-than' option for QuickPUNCH scheduling (#264)
- Updates Level 3 processing for preliminary products. (#266)
- Supports batching of mulitple core flow runs in one Prefect flow (#267)
- sync_db_to_files now retrieves a smaller mother lode of data from the DB (#268)
- Production of "recent" files can now be directly prioritized over back-processing. (#279)
- BADPKTS is set in the L0 flow and tracked in the database (#280)
- Support scheduling the new polarized stray light model generation. (#281)
- When an L1 scheduler defers more than 1000 files for the same reason, only a summary of those files is logged. (#282)
- Adds v0h upgrade script for L0 files (#283)
- Supports interpolating between NFI vignetting maps (#284)
- Support configuring the L2 mosaic rolloff (#288)
- Enabled QuickPUNCH F-corona modeling. (#296)
- Adds speedster, a launcher alternative that can achieve ~5x throughput for short-running flows (#304)
- Set processing_flow for L0 files. (#252)
- Fixes a few crashes (#252)
- Fix a crash in stray light selection (#264)
- Adds DB index, which seems to avoid some locking timeouts (#268)
- Misc bugfixes for L3 flows (#269)
- The database date_obs and date_created are updated to match what's written in the file at the end of each flow. (#270)
- Handling of outlier limits files in L0 is fixed when there are multiple applicable limits files (with different dates or versions). (#280)
- The cleaner flow can successfully stop flows that are still running and holding DB locks, and will avoid deleting flows it can successfully cancel. (#282)
- Avoid a scheduling crash when there are no stray light models in the DB (#282)
- The LQ CNN scheduler properly stops if a CNN flow is already pending (#294)
- Darks are no longer checked for outlier images. (#299)
- Update pre-commit. (#274)
- Make TLMLoader only expect one returned object in #233
- Temporarily turn off NFI PSF correction in #234
- Tweaks to stray light models scheduling in #237
- Tagging of flows with output file types in #237 and #238
- Set L0 polarization state in database in #239
- Fix LQ CNN scheduling, set stray light polarization in DB, and cap stray light generation thread count in #240
- Redo stray light model scheduling and selection in #244
- Dash board updates, including Files page, in #242
- Misc pipeline optimizations and fixes in #243
- Dashboard code moved out of cli.py in #245
- Support setting alpha file path and edge trimming in L2/LQ flows in #248
- Mark outliers in L0 stage, and propagate state forward in #247
- Broaden stray light search window to 31 in #217
- Fixes to run the stray light model generation flow in #216
- Fix to selection of "after" stray light model in #218
- Set stray light models' date_obs to the reference time; only select "created" models for L1 inputs; use DataLoader API for all cache types; dashboard fix in #219
- Absolute file paths are no longer stored in the database in #222
- Fixes for PTM scheduling and input file selection in #224
- Split L1 flow, implemented regular stray light models, and improvements to flow scheduling and running in #225
- Dont make mosiacs out of very recently-written files in #226
- For now, skips NFI PSF correction in #227
- Limits recency of file upload for NOAA QuickPUNCH in #230
- Updates replay request cleaning script in #190
- Don't schedule for disabled flows in #203
- Dashboard fix for file cards with multiple file types in #202
- Cleans flows stuck in a 'launched' state in #209
- Set FILEVRSN for files before writing in #204
- Adds docs button to view source in #208
- Implements rolling stray light models in #212
- Support batched LQ CNN and improved outlier limits in #210
- Fixes metadata usage for quicklook animations in #185
- Relabels CCD halves in #174
- Don't remake existing files in #188
- Save an L1-with-stray-light intermediate file in #193
- Support separated LQ flows and exclude NFI from LQ CTM in #192
- Clear .jp2, .sha, and parent directories in cleaner flow in #191
- L2s and LQ CTMs with missing input files can be made anyway after a certain number of days in #194
- Reduces L2 code duplication in #195
- Set date_obs correctly in DB for L2s, and update cleaner flow for L2 in #196
- Temporarily disabled NFI in L2s in #197
- Relabels the polarizers for WFI for the flipped orientation in #179
- Group L2 and LQ inputs with time flexiblity, check there are enough files to fit for LQ, and fix L2 input file queries in #170
- Many changes for QuickPUNCH compatibility in #175
- Disables logging in cache manager if not in a flow context in #159
- Weighted launching and NUMA configuration in #160
- Save floating-point values for COMPBITS when appropriate in L0 in #164
- Updates precision of square root decoding table bitrate in #167
- LQ speedup, PCA support, and fixes to DB file state in #168
- Adds script to add calibration files to database and adds distortion to Level 1 processing in #169
- Prepares punchpipe for SOC2NOAA Interface by @jmbhughes in #94
- Specify path for codecov by @jmbhughes in #95
- Update issue templates by @jmbhughes in #97
- Update README.md by @jmbhughes in #98
- Adds vignetting to level 1 processing by @lowderchris in #103
- Makes AWS profile configurable by @jmbhughes in #112
- Added notes in README about testing in #114
- Creates VAM/VAN flow automation, corrects flash block length, fixes attitude quaternions in #102
- Checked that all times were UTC in #119
- Many automation improvements in #115
- Uses central dask cluster in #129
- Improves level 0 metadata population in #128
- Splits CCD parameters per chip half in #131
- Iterates over sequence count instead of packet index in #132
- Varied improvements to the pipeline, including launching and scheduling in #134
- Fixed database entries for simpunch and launching improvements in #135
- Added a shared memory cache, streamlined the launcher, improved robustness, and changed logging to local time in #136
- Added a "cleaner" flow, staggered flow launching, and L2 fixes in #145
- Add flow throughput and duration stats to the dashboard in #144
- Expands ffmpeg movie creation options in #147
- Appropriately uses TAI time in #146
- Stores quicklook movies in date-based file structure in #150
- Checks that input files for quicklook movies are sorted and only schedules if files are found in #151
- Updates L0 header generation in #156
- Uses the shared memory cache for simpunch and sets date_created for files in the database in #154
- Improvements to dashboard status cards and flow table in #155
- if sequence counters don't increase properly, call it a bad image by @jmbhughes in #90
- Updates for V4 RFR2
- Fix l0 image form by @jmbhughes in #87
- Improve l0 by @jmbhughes in #88
- Improve monitoring utility by @jmbhughes in #78
- Prepare for End2End Test by @jmbhughes in #81
- Fix reference times for f corona models by @jmbhughes in #83
- fully tested level 0 by @jmbhughes in #84
- save only the TLM filename instead of the whole path by @jmbhughes in #85
- make sure the path is extracted for tlm by @jmbhughes in #86
Initial Release