Skip to content

Dev bilayers#10

Open
TorecLuik wants to merge 8 commits into
masterfrom
dev-bilayers
Open

Dev bilayers#10
TorecLuik wants to merge 8 commits into
masterfrom
dev-bilayers

Conversation

@TorecLuik
Copy link
Copy Markdown
Collaborator

This pull request introduces several improvements and fixes across the Slurm workflow scripts, focusing on parameter type handling, error handling, and robustness. It also adds a new admin-only script for executing safe Slurm commands via OMERO. The most important changes are grouped below:

Parameter Type Handling and Workflow Descriptor Improvements:

  • Replaces usage of convert_cytype_to_omtype with convert_param_type_to_omtype and consistently reads the parameter type from the type key instead of cytype in workflow parameter processing in SLURM_CellPose_Segmentation.py, SLURM_Run_Workflow.py, and SLURM_Run_Workflow_Batched.py. This improves compatibility with updated parameter schemas. [1] [2] [3]
  • Updates workflow descriptor fetching to use generic_descriptor_from_github instead of pull_descriptor_from_github, and consistently accesses descriptor fields, improving maintainability and reducing ambiguity. [1] [2]

Robustness and Error Handling:

  • Adds a pre-flight check in run_workflow to verify that the job script exists on the Slurm cluster before submitting a workflow, providing a clear error if the script is missing.
  • Improves job status polling by initializing the job status dictionary before querying and logging transient errors as warnings instead of failing the workflow immediately, making job monitoring more resilient.
  • Ensures all workflow parameters from other workflows are marked as optional, preventing them from blocking execution if not provided.

Bug Fixes:

  • Fixes a bug in _SLURM_Image_Transfer.py where splitting the target filename for zipping now correctly handles filenames with multiple periods by using rsplit(".", 1).

New Admin Script:

  • Adds admin/Example_Minimal_Slurm_Script.py, a secure admin-only OMERO script for running selected Slurm or Linux commands on the cluster. It includes comprehensive safety checks to block dangerous commands, detailed logging, and user/admin validation to ensure only authorized actions are performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants