fix: warehouse jobs skip processing destinations

[achettyiitr]

Description

• Additional filters will be added to skip destinations during the pickup of sync jobs. Later, this can be exposed in the UI for customers to pause syncs during scenarios like migrations.
• Corresoponding Slack Thread: https://rudderlabs.slack.com/archives/C02AALVFHJN/p1744730044640719

Security

• The code changed/added as part of this pull request won't create any security issues with how the software is being used.

[RanjeetMishra]

@achettyiitr
Can you explain state of system the moment we update config for skipDestination? Any tasks till upload state will keep doing its work, only at last stage of upload we will keep skipping?

And once this skipDestination list is cleared, it will resume?

[achettyiitr]

@achettyiitr Can you explain state of system the moment we update config for skipDestination? Any tasks till upload state will keep doing its work, only at last stage of upload we will keep skipping?

And once this skipDestination list is cleared, it will resume?

Once the config is set, we will not pick any jobs for those destinations. If there are some jobs already being processed, it will continue processing.
Upon removal, jobs from those destinations will be picked up.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.01%. Comparing base (b5af4c2) to head (376e332).

Additional details and impacted files

@@                Coverage Diff                 @@
##           release/1.47.x    #5747      +/-   ##
==================================================
+ Coverage           76.98%   77.01%   +0.02%     
==================================================
  Files                 487      487              
  Lines               66244    66257      +13     
==================================================
+ Hits                50997    51025      +28     
+ Misses              12475    12463      -12     
+ Partials             2772     2769       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.