User story: As a walkthrough developer, I'd like to have a simple example workflow to demonstrate the main stages in an HPC workflow, so that I can build other walkthroughs around it.
The walkthroughs in this repository typically take three forms:
- Hints and tips that can be read quickly for specific tasks that relate to specific HPC systems.
- Explanations for how to deploy specific packages to a specific HPC systems.
- Generic processes that can be applied to HPC systems for performing specific tasks.
In the case of 3, it's typically the case that some generic workflow is needed for the purposes of demonstration. Using the same example workflow across walkthroughs, or at least something generic that can be built on, is helpful for the walkthrough writer as it makes the job of creating the walkthrough easier. It's also helpful for the reader as they can immediately start with a familiar example.
This task is therefore to create an example workflow that can be used for developing future walkthroughs.
The example should:
- Be deployable to multiple HPC systems (at least Baskerville, DAWN, Isambard-AI and Azure).
- Capture the main features that an HPC workflow typically employs. For example:
- Batch scripts for user with Slurm
sbatch.
- A workflow that also works with
srun.
- Use of PyTorch.
- Potentially both training and inference pipelines.
- The ability to distribute across multiple GPUs and multiple nodes.
- On top of this, the example should be as lightweight, minimal and easily understandable as possible.
User story: As a walkthrough developer, I'd like to have a simple example workflow to demonstrate the main stages in an HPC workflow, so that I can build other walkthroughs around it.
The walkthroughs in this repository typically take three forms:
In the case of 3, it's typically the case that some generic workflow is needed for the purposes of demonstration. Using the same example workflow across walkthroughs, or at least something generic that can be built on, is helpful for the walkthrough writer as it makes the job of creating the walkthrough easier. It's also helpful for the reader as they can immediately start with a familiar example.
This task is therefore to create an example workflow that can be used for developing future walkthroughs.
The example should:
sbatch.srun.