Conversation
Merge back of 1.1.10 release
"sproviding" typo fix -> "providing"
* Drop python 3.7, unpin mkdocs * Update poetry to 1.8.x, and update dependencies to enable 10.x mermaid versions in the docs * Patch landing page mermaid diagram code to work with 10.x syntax * Remove python 3.7 from ci, add 3.12, 3.13 * Tick poetry version in ci * Update second version matrix * Bump poetry and python versions for docs build * Fix workflow definitions, pin poetry version * Update mkdocs emoji config * Fix typo in emoji extension config * Test snippet from readthedocs' docs for poetry dependency installs * Fix another broken mermaid diagram. * Tick dev version
Add the ability to change some of a study's configuration during execution, including: * rlimit (restart limit) * throttle * sleep Implementation works like cancel, taking paths to executing study workspaces. Configuration parameters can be updated either via an interactive prompt for each study workspace given, or using explicit/repeated keyword arguments for each configuration parameter. Current Limitations: * rlimit is a per step configuration, but it is only set globally in current implementation * If a study is no longer running, updating rlimit will not yet resume the study to account for any increases Adds termynal to the docs to improve readability of interactive cli commands and their outputs
Update script adapters to use same path sanitizer as for workspace creation for script serialization. Resolves #450 Add hypothesis to enable property testing to stress the path sanitizer. Adds copy button to code listings in documentation
Adds support for attaching the bank, queue fields from the batch block to the flux jobspec. Adds preliminary support for pass through of arbitrary --setattr flags to the jobspec (attaches to allocation, not flux run's inside the job scripts). Support added only for newest Maestro flux adapter version, 0.49.0; older versions to be deprecated. Change default adapter behavior to 'nested: True' so it uses from_nest_command by default. Fix documentation typos on the cli's pgen args syntax. Update documentation for running with flux.
Expands available options to pass through to flux, replacing the args dictionary in the batch block which only passed through shell options (--setopt/-o) to the launcher (flux run). Adds two dictionaries to the batch block: * allocation_args: enable attaching arbitrary numbers of options to the allocation (jobspec) corresponding to the cli equivalents: -o/--setopt, -S/--setattr, and --conf * launcher_args: replaces args for attaching to launcher, but enables passing through any option prefix, not just -o/--setopt Deprecates args, but leaves it intact as the only option for older flux adapters (< 0.49) pending future removal. Adds flux directives with all allocation options to serialized step scripts to enable reproducibility and direct cli submission by users to match capability of other script adapters. Exposes new exclusive syntax for step keys to enable separate allocation/launcher control: only flux uses the launcher option, other schedulers to follow. Adds script writing and job submission tests for both flux and slurm. Updates info directive in flux scripts from INFO to MAESTRO-INFO for more clearly showing source of info.
* Add autoyes flag to cancel cli command for improved automated usage
Implements MEP 003's proposal for more human readable hashing of workspaces/step id's. Replaces default 'md5' algorithm for hashing step combinations in favor of sortable/human readable 'hash'. This is not a true 'hash', in that it needs all combos for a step to compute the pattern. Pattern is: step_name_instance_<number>, where is a zero padded integer (zero based index) based on the number of used step combinations for a particular step. Zero padding ensures proper sorting on filesystems to improve usability. * replaces original md5 hash algorithm for step nicknames/workspaces * reuses --hashws cli option to activate * adds local integration test * adds test fixtures for checking workspace layouts relative to study root (can't hardcode absolute paths in tests) * adds test fixtures for checking substituted workspace tokens in generated step scripts. * starts implementing rich_repr methods on containers for better debug outputs
…launcher (#467) Updates handling of exclusive keys to modify the batch directives and launcher rendering * Prunes everything but nodes from allocations * If set on launcher, overrides any cores per task, gpu flags, leaving nodes and tasks only * Jobspec creation via Python api now sets num_slots = nodes when exclusive is set, matching flux behavior Updates default renderings: * cores per task omitted from launcher if not set in step keys * removes default 'nodes=1' in batch header/directives to sync with step keys * retains required >=1 for tasks/cores per task in jobspec creation but omits from rendered batch scripts to sync with flux default behaviors Unblocks scheduling to nodes with configurable resources: subdividing gpus happens after allocation startup, and prior behavior of attaching tasks/gpus to allocation would cause flux to reject such jobs (unsatisfiable). Add docs on new exclusive syntax in the study specification Document behavior changes with exclusive flag applied to allocations for flux
Finalize 1.2.0 Version
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge 1.2.0 into main for release/tagging