Repository navigation
Open
Description
Data ingested into the MinIO incoming bucket occasionally fails to publish to WIS2. While the pipeline operates correctly for most files, seemingly correct files fail during conversion, blocking publication. Failures originate at both the CSV-to-BUFR4 (csv2bufr) conversion stage and the subsequent BUFR4-to-GeoJSON generation stage.
We have verified that the .CSV data in MinIO is correct by running the wis2box-csv2bufr API process.
Observed Error Categories
- .csv failed to convert to BUFR4
- .bufr4 failed to convert to GeoJSON
- No features generated
- Incorrect padding
Relevant Logs
[2026-10-19T08:02:51Z] {/app/wis2box/data/csv2bufr.py:85} WARNING - input=WIGOS_0-231-0-63478_20261019T070000.csv warning=#1#airTemperature: Value (-725.85) out of valid range (193.15 - 333.15).; Element set to missing
[2026-10-19T08:02:51Z] {/app/wis2box/data/csv2bufr.py:85} WARNING - input=WIGOS_0-231-0-63478_20261019T070000.csv warning=#1#relativeHumidity: Value (-999.0) out of valid range (0 - 100).; Element set to missing
[2026-10-19T08:02:52Z] {/app/wis2box/data/csv2bufr.py:87} ERROR - file=WIGOS_0-231-0-63453_20261019T070000.csv failed to convert to BUFR4, result={'error': 'Incorrect paddingNo features generated', 'items': []}
[2026-10-19T08:02:52Z] {/app/wis2box/data/csv2bufr.py:87} ERROR - file=WIGOS_0-231-0-63471_20261019T070000.csv failed to convert to BUFR4, result={'error': 'No features generated', 'items': []}
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
[2026-10-19T09:02:51Z] {/app/wis2box/data/csv2bufr.py:85} WARNING - input=WIGOS_0-231-0-63478_20261019T080000.csv warning=#1#airTemperature: Value (-725.85) out of valid range (193.15 - 333.15).; Element set to missing
[2026-10-19T09:02:51Z] {/app/wis2box/data/csv2bufr.py:85} WARNING - input=WIGOS_0-231-0-63478_20261019T080000.csv warning=#1#relativeHumidity: Value (-999.0) out of valid range (0 - 100).; Element set to missing
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
[2026-10-19T10:02:52Z] {/app/wis2box/data/csv2bufr.py:85} WARNING - input=WIGOS_0-231-0-63478_20261019T090000.csv warning=#1#airTemperature: Value (-725.85) out of valid range (193.15 - 333.15).; Element set to missing
[2026-10-19T10:02:52Z] {/app/wis2box/data/csv2bufr.py:85} WARNING - input=WIGOS_0-231-0-63478_20261019T090000.csv warning=#1#relativeHumidity: Value (-999.0) out of valid range (0 - 100).; Element set to missing
[2026-10-19T10:02:52Z] {/app/wis2box/data/bufr2geojson.py:75} ERROR - file=WIGOS_0-231-0-63478_20261019T090000.bufr4 failed to convert to GeoJSON
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
wis2box.cron is alive
[2026-10-19T11:02:52Z] {/app/wis2box/data/csv2bufr.py:85} WARNING - input=WIGOS_0-231-0-63478_20261019T100000.csv warning=#1#airTemperature: Value (-725.85) out of valid range (193.15 - 333.15).; Element set to missing
[2026-10-19T11:02:52Z] {/app/wis2box/data/csv2bufr.py:85} WARNING - input=WIGOS_0-231-0-63478_20261019T100000.csv warning=#1#relativeHumidity: Value (-999.0) out of valid range (0 - 100).; Element set to missing
[2026-10-19T11:02:53Z] {/app/wis2box/data/csv2bufr.py:87} ERROR - file=WIGOS_0-231-0-63475_20261019T100000.csv failed to convert to BUFR4, result={'error': 'No features generated', 'items': []}
[2026-10-19T11:02:56Z] {/app/wis2box/data/csv2bufr.py:87} ERROR - file=WIGOS_0-231-0-63474_20261019T100000.csv failed to convert to BUFR4, result={'error': 'No features generated', 'items': []}
[2026-10-19T11:02:57Z] {/app/wis2box/data/bufr2geojson.py:75} ERROR - file=WIGOS_0-231-0-63338_20261019T100000.bufr4 failed to convert to GeoJSON
Metadata
Metadata
Assignees
Labels
data transformationData transformationData transformation
Activity
maaikelimper commentedon Oct 19, 2026
@alhelguera can you have a look a this ?
alhelguera commentedon Oct 19, 2026
@robeleq can you provide some sample files that produce the issue?
robeleq commentedon Oct 22, 2026
@alhelguera Here are some of the .CSV files. I have also tried to reproduce the issue with the files, but all tests passed (CSV->BUFF and BUFF -> GEOJSON) using the
wis2box-csv2bufrandbufr2geojsonAPI processesWIGOS_0-231-0-63471_20261019T070000.csv
WIGOS_0-231-0-63502_20261012T110000.csv
WIGOS_0-231-0-63502_20261012T070000.csv
WIGOS_0-231-0-63502_20261012T060000.csv
alhelguera commentedon Oct 23, 2026
@tomkralidis found that the requests from csv2bufr are showing results from bufr2geojson and viceversa.
This led me to think there could be a concurrency issue somewhere in the stack with a race condition enabling these exchanges. Traced to a plausible race condition which was introduced in pygeoapi commit geopython/pygeoapi@3a63f5b0cc
In short, changing the processor to be a class level variable enabled the sharing, and subsequent exchanging of it between different requests
robeleq commentedon Oct 23, 2026
@tomkralidis I agree with your analysis, and it's likely that concurrent requests can overwrite the selected processor.
self.processor = self.get_processor(process_id)Later, after adding the job, it passes
self.processorto the execution handler. Another request could change that value in between. If request A and request B overlap, request B can replaceself.processorwhile request A is still executing.The appropriate code correction should be to use a local variable throughout
execute_process:and replace the remainingself.processorreferences inside that method with processor.tomkralidis commentedon Nov 6, 2026
FYI fixed in pygeoapi (geopython/pygeoapi#2479)
robeleq commentedon Nov 11, 2026
@tomkralidis Thanks! When can we expect this pygeoapi fix to be included in an official wis2box API release/image so we can upgrade our production node?