-
Notifications
You must be signed in to change notification settings - Fork 2
ZFIN-10059: Instead of running "Load-GPAD-Noctua_w" every day, run "L… #1665
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…oad-GPAD-Noctua-Daily-Trigger_d" daily and it will trigger the former if necessary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR implements an optimization for the GPAD Noctua data load process by introducing a conditional trigger mechanism. Instead of running the "Load-GPAD-Noctua_w" job daily, it now runs a "Load-GPAD-Noctua-Daily-Trigger_d" job that first checks if new data is available before triggering the actual load job.
Key changes:
- Created CheckNewLoadFileAvailableTask to check for new files before triggering loads
- Refactored isDownloadAlreadyProcessed method from GafLoadJob to DownloadService for reuse
- Updated Jenkins scheduling to use the new conditional trigger approach
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| source/org/zfin/task/CheckNewLoadFileAvailableTask.java | New task class that checks if a new load file is available and exits with appropriate codes (0=new file, 1=no new file, 2=error, 3=invalid args) |
| source/org/zfin/datatransfer/service/DownloadService.java | Added isDownloadAlreadyProcessed method moved from GafLoadJob to enable reuse |
| source/org/zfin/datatransfer/go/service/GafLoadJob.java | Refactored to call isDownloadAlreadyProcessed from DownloadService instead of local private method |
| server_apps/jenkins/trigger.production.properties | Updated schedule to run the new daily trigger job instead of the weekly load job |
| server_apps/jenkins/jobs/Load-GPAD-Noctua-Daily-Trigger_d/config.xml | New Jenkins job configuration that checks for new files and triggers Load-GPAD-Noctua_w only on success |
| server_apps/DB_maintenance/build.xml | Added check-noctua-gpad-available Ant target to execute the new check task |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <[email protected]>
485ab09 to
2d58f0a
Compare
…oad-GPAD-Noctua-Daily-Trigger_d" daily and it will trigger the former if necessary.