Remove all L3 survival corrected map dependencies

[pleasant-menlo]

Change Summary

L3 survival corrected map processing will be triggered using a daily job. This change removes all the lines in the dependency_config.csv to prevent these jobs being triggered by the batch starter.

Updated Files

• dependency_config.csv
  • removed lines for the Hi L3 survival corrected map
  • removed lines for the Lo L3 survival corrected map
  • removed lines for the Ultra L3 survival corrected map

[pleasant-menlo]

Hi @lacoak21, deleting these dependencies caused some batch starter tests to fail around special case jobs. We've updated the L3 maps to use scheduled jobs that handle the processing and reprocessing, so we no longer need this special case handling. Is there still a use for this logic for reprocessing L2 maps?

[lacoak21]

Hi @lacoak21, deleting these dependencies caused some batch starter tests to fail around special case jobs. We've updated the L3 maps to use scheduled jobs that handle the processing and reprocessing, so we no longer need this special case handling. Is there still a use for this logic for reprocessing L2 maps?

I think I misunderstood this. I had thought these jobs should still be triggered by batch starter? If not I think there is a lot of code that can be removed surrounding "special case jobs" unless, you do still need the reprocessing code. Do you expect to reprocess these maps using batch starter logic?

[pleasant-menlo]

I think I misunderstood this. I had thought these jobs should still be triggered by batch starter? If not I think there is a lot of code that can be removed surrounding "special case jobs" unless, you do still need the reprocessing code. Do you expect to reprocess these maps using batch starter logic?

Hi Luisa,
We initially thought that we would be able to reprocess using the batch starter but due to changes that have been made upstream with GLOWS, it makes more sense to trigger processing through a scheduled job.

The scheduled job automatically reprocesses when new input data becomes available, so we don't believe this logic will be needed in the batch starter.

We did our best to remove the special cases, let us know if anything looks wrong. Thanks for working with us on this.

[lacoak21]

We did our best to remove the special cases, let us know if anything looks wrong. Thanks for working with us on this.

Got it! Ok thanks for removing that logic. I will take a look and see if anything else needs to be removed.

[lacoak21]

I think this makes these functions outdated:

get_special_case_date_range
handle_special_case_reprocessing_jobs

[pleasant-menlo]

Hi @lacoak21, we took out those 2 functions, let us know if there is anything else we should do. Thanks!