-
Notifications
You must be signed in to change notification settings - Fork 14
Support job_nbr in ts task #741
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Sure, I will work on that today.
Sorry I meant to say rc2.. |
|
Sure, happy to help. LMK if you have any specific questions. |
Fixed with e27e797. GitHub Actions pass too for 3.11, 3.12, 3.13. |
czender
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes look appropriate to allow optional specification of --job_nbr in zppy.
|
Thank you, @czender ! |
Summary
This feature is useful in
glbtime-series task for land, as 300+ variables from land are being processed in parallel, and potentially is useful for ne120 HR processing. According to NCO documentation:For Background parallelism, job_nbr defaults to the number of variables to be split, so that not specifying job_nbr results in launching var_nbr simultaneous splitter tasks. This scales well to over a hundred variables in our tests [76](https://nco.sourceforge.net/nco.html#FOOT76). In practice, splitting timeseries consumes minimal memory, since ncrcat (which underlies the splitter) only holds one record (timestep) of a variable in memory [Memory Requirements](https://nco.sourceforge.net/nco.html#Memory-Requirements).With added job_nbr flag for ts task, user can manage RAM use by reducing the number for parallel process.
Issue resolution:
Select one: This pull request is...
Small Change