Add cluv submit command for SLURM job submission#10
Merged
Conversation
- New \`cluv submit <cluster> <command>\` command that enforces a clean git state, injects \`GIT_COMMIT\`, applies \`SBATCH_*\` env vars from config, syncs the project, and runs sbatch on the remote cluster. - Extend \`CluvConfig\` with \`SubmitConfig\`, global \`slurm\` dict, and per-cluster \`cluster_configs\` dict. - Migrate \`[tool.cluv]\` clusters from a flat list to \`[tool.cluv.clusters.*]\` sub-tables (backward compat with old list format preserved). - Global SBATCH_* defaults go in \`[tool.cluv.slurm]\`; per-cluster overrides go in \`[tool.cluv.clusters.<name>]\`. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- test_config.py: 14 unit tests for new/old cluster format, global slurm vars, submit config, and edge cases. - test_submit.py: 9 async unit tests (fully mocked) covering dirty-tree abort, missing job script, env var merging, --no-sync, --job-script override, and GIT_COMMIT injection. - test_integration.py: smoke test for submit reaching rorqual over SSH. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add install_scripts() to sync_task_function(): after cloning the project, all executable files in scripts/ are symlinked into ~/.local/bin/ on each cluster (with .sh extension stripped). Uses ln -sf so re-runs are safe. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix test_submit_rorqual_builds_correct_command: with no_sync=False, submit() uses the mocked sync result rather than calling RemoteV2.connect() internally (which would return a different object). Also switch to direct attribute assignment to avoid instance patching issues with RemoteV2. - Remove partiton-stats_output.txt (stale sample output file). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Untracked files don't affect job reproducibility. Only changes to tracked files (modified, staged, deleted) should block submission. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…sbatch vs program args - `cluv submit <cluster> <job.sh> [--no-sync] [sbatch-flags...] [-- program-args...]` - Remove SubmitConfig and job_script from pyproject.toml config - Replace REMAINDER `command` + `--job-script` with required `job_script` positional and `rest` REMAINDER split on `--` inside submit() - Update all tests accordingly; replace obsolete test_no_job_script_aborts and test_cli_job_script_overrides_config with sbatch flag forwarding tests Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…iption Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
95f1e99 to
87e2b86
Compare
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test_submit_rorqual_real wraps run_async to capture sbatch output and asserts "Submitted batch job" — requires active rorqual connection and clean git tree, mocks sync so the project needn't be freshly pushed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #10 +/- ##
==========================================
+ Coverage 40.00% 40.78% +0.78%
==========================================
Files 11 13 +2
Lines 725 868 +143
==========================================
+ Hits 290 354 +64
- Misses 435 514 +79 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
argparse consumes '--' before REMAINDER sees it, so program args were landing in sbatch_args. Fix by splitting argv on '--' in main() before parse_args(), injecting program_args into args_dict post-parse. Also renames rest → sbatch_args throughout for clarity. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
Signed-off-by: Fabrice Normandin <normandf@mila.quebec>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Config shape:
Usage:
Test plan
🤖 Generated with Claude Code