Clean up hpc basics section and add a gams project for tutorial

Author: ptsavol

docs/source/examples/gams_on_hpc.rst

@@ -106,6 +106,14 @@ Running a Spine Toolbox project on an HPC
 
     Line endings in Slurm scripts such as `run_on_hpc.sh` must be Unix style (LF).
 
+**Key parameters**
+    - ``--job-name``: Job name
+    - ``--time``: Maximum runtime
+    - ``--cpus-per-task``: CPU cores
+    - ``--mem``: Memory
+    - ``--output``: Output file
+    - ``--error``: Error file
+
 6. Edit the Slurm script by adding the license
 7. Submit job to Slurm Scheduler
 
@@ -171,6 +179,12 @@ This command returns something like:
 
     watch -n 2 squeue -u $USER
 
+Another option is to use `tail`:
+
+.. code-block:: bash
+
+   tail -f out.txt
+
 Again, if $USER is not defined, replace it with your user name. This function tails the job progress and updates
 every two seconds.
 
@@ -377,6 +391,43 @@ File Not Found
 
    echo $PWD
 
+Job Stuck in Queue
+++++++++++++++++++
+
+- Cluster is full
+- Resource request too large
+
+Memory Errors
++++++++++++++
+
+Increase memory:
+
+.. code-block:: bash
+
+   #SBATCH --mem=16G
+
+
+Solver Not Found
+++++++++++++++++
+
+.. code-block:: bash
+
+   module load gurobi
+
+Check installation:
+
+.. code-block:: bash
+
+   which gurobi_cl
+
+
+Python Module Missing
++++++++++++++++++++++
+
+.. code-block:: bash
+
+   pip install pyomo
+
 Module Not Found
 ++++++++++++++++
 

docs/source/examples/hpc_basics.rst

@@ -28,7 +28,8 @@ Most HPC systems follow a similar architecture composed of three main components
 
    Conceptual structure of an HPC cluster
 
-Users typically interact only with the login node. After a job is submitted, the scheduler assigns it to available compute nodes, where it runs—often in parallel across multiple CPUs or nodes.
+Users typically interact only with the login node. After a job is submitted, the scheduler assigns it to available
+compute nodes, where it runs—often in parallel across multiple CPUs or nodes.
 
 .. warning::
 
@@ -52,9 +53,7 @@ This separation allows HPC systems to efficiently share resources among many use
    :width: 100%
    :align: center
 
-   Overview of the HPC workflow for running an energy optimization model.
-   The user prepares and submits a job from a local machine, which is scheduled
-   and executed on compute nodes. Results are then retrieved back to the local environment.
+   Overview of the HPC workflow
 
 *************
 Job Lifecycle
@@ -66,7 +65,8 @@ When a job is submitted to the scheduler (such as Slurm), it goes through severa
 - **RUNNING**: the job is actively executing on compute nodes
 - **COMPLETED**: the job has finished successfully (or terminated with an error)
 
-Understanding these states helps explain why jobs may not start immediately—waiting in the queue is normal and depends on factors such as resource availability and system load.
+Understanding these states helps explain why jobs may not start immediately—waiting in the queue is normal and
+depends on factors such as resource availability and system load.
 
 .. figure:: ../img/tutorials/job_lifecycle.png
    :width: 100%
@@ -83,163 +83,180 @@ Connecting to the Cluster
 Login
 -----
 
+You need access rights (username/password) for your HPC cluster, which you should request from the HPC administrator.
+Once you have the necessary rights, you can make an SSH connection from your terminal to the HPC login node.
 
 .. code-block:: bash
 
    ssh username@cluster.address
 
+On Windows, we recommend using PuTTY as an SSH client. You can install it from the Windows
+Store or from the `PuTTY site <https://putty.software/>`_.
 
 File Transfer
 -------------
 
+You can use scp to transfer files between your local system and the login node.
+
 .. code-block:: bash
 
    scp -r my_project/ username@cluster:/home/username/
 
-or:
-
-.. code-block:: bash
-
-   rsync -avz my_project/ username@cluster:/home/username/my_project/
+However, in the long run this may become tedious, so it is recommended that you use for example
+`WinSCP <https://winscp.net/>`_, which makes file transfers quicker by providing an easy to use drag-and-drop UI.
 
 *************************************
 Understanding the Cluster Environment
 *************************************
 
-Common Directories
-------------------
+Working on an HPC cluster differs from working on a local machine. The filesystem is typically distributed and
+shared across compute nodes, and different directories are designed for specific purposes. Using them correctly
+is essential for performance, data safety, and efficient workflows.
 
-- ``$HOME``: persistent storage
-- ``$SCRATCH``: fast temporary storage
-
-
-Module System
--------------
+Common Directory Types
+----------------------
 
-.. code-block:: bash
+Running applications on a cluster requires that all compute nodes involved in a job can access the same files.
+This is usually achieved through a shared parallel filesystem. While directory names vary between systems,
+most clusters provide the following *types* of storage:
 
-   module avail
-   module list
+- ``$HOME`` (or home directory)
 
-**************************
-Writing a Slurm Job Script
-**************************
+  Your personal home directory. This is **persistent storage**, meaning files are kept long-term and often backed up.
+  However, it is typically **not optimized for heavy I/O workloads**, so it should mainly be used for:
 
-Create ``job.sh``:
+  - Source code
+  - Configuration files
+  - Small input datasets
+  - Scripts and job submission files
 
-.. code-block:: bash
+- High-performance temporary storage (often called ``$SCRATCH`` or similar)
 
-   #!/bin/bash
-   #SBATCH --job-name=energy_model
-   #SBATCH --output=output.log
-   #SBATCH --error=error.log
-   #SBATCH --time=02:00:00
-   #SBATCH --cpus-per-task=4
-   #SBATCH --mem=8G
+  A fast storage area intended for **temporary data and intensive I/O operations**.
+  The exact location and name vary by system. Examples include:
 
-   module load python
-   module load gurobi
+  - ``$SCRATCH`` (if defined)
+  - ``/scratch``
+  - ``/tmp``
+  - ``/jobs`` (on some systems)
 
-   source venv/bin/activate
+  Consult your system documentation to find the correct path.
 
-   python run_model.py
+  Use this storage for:
 
+  - Large simulation outputs
+  - Intermediate data
+  - Temporary working files
 
-Key Parameters
---------------
+  .. note::
+     These locations are usually not backed up and may be cleaned automatically
+     after a retention period. Always copy important data to persistent storage.
 
-- ``--job-name``: Job name
-- ``--time``: Maximum runtime
-- ``--cpus-per-task``: CPU cores
-- ``--mem``: Memory
-- ``--output``: Output file
-- ``--error``: Error file
+- Project or shared storage (often called ``$PROJECT`` or ``$WORK``)
 
+  A shared directory intended for **collaborative work within a project or research group**.
+  Not all systems provide this, but when available it typically offers more space than ``$HOME``
+  and longer retention than temporary storage.
 
-Submitting the Job
-------------------
+  Common uses include:
 
-.. code-block:: bash
+  - Shared datasets
+  - Group software installations
+  - Results that need to be preserved longer-term
 
-   sbatch job.sh
+  .. note::
+     If your system does not provide a dedicated project directory, you may need
+     to manage shared data manually in agreed-upon locations.
 
+Best Practices
+--------------
 
-Monitoring the Job
-------------------
+Because directory names vary between systems, always adapt these guidelines to the paths provided on your cluster:
 
-Check queue:
+- Copy input data from persistent storage (e.g. ``$HOME`` or project space) to temporary high-performance storage
+  before running jobs.
+- Perform all **compute-intensive tasks** using the high-performance temporary storage (e.g. ``$SCRATCH`` or ``/jobs``).
+- Regularly clean up unnecessary files from temporary storage.
+- Avoid running large jobs directly from your home directory.
 
-.. code-block:: bash
+Example Workflow
+----------------
 
-   squeue -u username
+A typical workflow might look like:
 
-Job details:
+1. Prepare input files and scripts in your home directory (``$HOME``)
+2. Copy necessary data to the system's temporary work directory (e.g. ``$SCRATCH`` or ``/jobs``)
+3. Run the job using the scheduler
+4. Copy final results back to persistent storage (``$HOME`` or project space)
+5. Clean up temporary data
 
-.. code-block:: bash
+This approach ensures efficient use of cluster resources while keeping your data safe and organized. Next section
+provides an actual example workflow for executing Spine Toolbox projects in an HPC environment.
 
-   scontrol show job JOBID
+Module System
+-------------
 
-View logs:
+Environment modules is a system tool to manage the shell environment. It makes it easier to handle the shell
+environment when there are e.g. multiple versions of the same software installed. You can get a list of available
+modules with command
 
 .. code-block:: bash
 
-   tail -f output.log
-
-***********************
-Debugging Common Issues
-***********************
+   module avail
 
-Job Stuck in Queue
-------------------
+The needed module can be loaded with the command
 
-- Cluster is full
-- Resource request too large
+.. code-block:: bash
 
-Memory Errors
--------------
+   module load <module_name>
 
-Increase memory:
+A module can be unloaded with the command
 
 .. code-block:: bash
 
-   #SBATCH --mem=16G
+   module unload <module_name>
 
+You can load any modules you need in a Slurm script with the `module load` command.
 
-Solver Not Found
-----------------
-
-.. code-block:: bash
-
-   module load gurobi
+Slurm - Basic Commands
+----------------------
 
-Check installation:
+These basic commands should be available in the HPC login node to help you get started. There are many more
+commands available, please also read your HPC documentation if there are special circumstances you should be
+aware of when using Slurm.
 
-.. code-block:: bash
+sinfo
+*****
 
-   which gurobi_cl
+This command gives a quick overview of the cluster status. It shows the status of different partitions, time
+limits, and available nodes.
 
+squeue
+******
 
-Python Module Missing
----------------------
+This command outputs information of all queues on all partitions. You can filter the output by partition with the
+**-p** argument, or by username with the **-u** argument. For example, `squeue -u <username>` prints the queued
+jobs of user <username>.
 
-.. code-block:: bash
+sbatch
+******
 
-   pip install pyomo
+This command will submit a job script to the queue. For example, `sbatch job.sh`. Please make sure that the batch
+script file uses **Unix (LF)** line endings.
 
+scancel
+*******
 
-**************
-Best Practices
-**************
+This command cancels a job. It needs a job id as an argument. For example, `scancel 10346`.
 
-Always:
+srun
+****
 
-- Test locally first
-- Run small cases
-- Use version control
-- Log outputs
+This command runs a single command on Slurm. It needs the same arguments for resource reservation as `sbatch`.
+For example `srun -J jobname --mem=64000 --pty -n 36 -p large36 /bin/bash`.
 
-Avoid:
+scontrol show job
+*****************
 
-- Running on login node
-- Over-requesting resources
-- Excessive file writing
+This command outputs detailed information about a job. It needs job id as an argument. For example,
+`scontrol show job 10345`.

execution_tests/gams_on_hpc_tutorial/.spinetoolbox/items/input_data/input.csv

@@ -0,0 +1,3 @@
+a,10
+b,20
+c,30