bufr_dupupr.f is a code developed by Dennis Keiser and updated by Chris Hill, and it was added to the obsproc v1.2.0 release
It is the slowest step in the generation of uprairbufr_d files and leads to long gdas and gfs dump run times, unacceptable by NCO in their current ecf configuration (kick off times).
Two changes were tested and made to speed up the code:
- latitude presence check - reduced the number of pressure levels in a profile checked for missing latitudes, from 25 to 5, saving ~150s of processing time
- sorting the array with all profiles by receipt time stamp is skipped (sorting by obs time stamp and station id remains)
One more change was considered, but not implemented - turn off the CORN mnemonic check (was a profile correction done by data provider). While this mnemonic is good to have in the bufr file, and it's in the prepbufr layout, it's not actually written in the prepbufr file , and it's not read by GSI. It would have ~150s . This could be turned off when the next amount of TAC profiles is replaced by BUFR high res profiles, as an easy "speed up" solution.
What else to explore? (scope of this task)
- Investigate the use of UFBTAB in bufr_dupupr and consider replacing it with faster (according to J.Woolen) UFBTAM or UFBMEM which may speed up the reading/writing process, but may require opening the file for read with a different bufrlib command.
- Evaluate ratio of TAConly/TAC&BUFR/BUFRonly profiles in the tanks and consider turning off the check for TAC&BUFR stations (if that saves us time)
Study the code and test replacing the UFBTAB calls, if help is needed, turn to Jack Woollen and Ron McLaren.
At the current time, we are working with SPAs to start the global dump steps earlier and are evaluating the impact on data loss, as a temporary solution in starting to use bufr profiles asap.
bufr_dupupr.f is a code developed by Dennis Keiser and updated by Chris Hill, and it was added to the obsproc v1.2.0 release
It is the slowest step in the generation of uprairbufr_d files and leads to long gdas and gfs dump run times, unacceptable by NCO in their current ecf configuration (kick off times).
Two changes were tested and made to speed up the code:
One more change was considered, but not implemented - turn off the CORN mnemonic check (was a profile correction done by data provider). While this mnemonic is good to have in the bufr file, and it's in the prepbufr layout, it's not actually written in the prepbufr file , and it's not read by GSI. It would have ~150s . This could be turned off when the next amount of TAC profiles is replaced by BUFR high res profiles, as an easy "speed up" solution.
What else to explore? (scope of this task)
Study the code and test replacing the UFBTAB calls, if help is needed, turn to Jack Woollen and Ron McLaren.
At the current time, we are working with SPAs to start the global dump steps earlier and are evaluating the impact on data loss, as a temporary solution in starting to use bufr profiles asap.