Skip to content

5. Runtime Job Integration

Jump to bottom
Xavier Tintin edited this page May 13, 2024 · 2 revisions

REANA Test

This REANA reproducible analysis example studies the Higgs-to-four-lepton decay channel that led to the Higgs boson experimental discovery in 2012. The example uses CMS open data released in 2011 and 2012. "This research level example is a strongly simplified reimplementation of parts of the original CMS Higgs to four lepton analysis published in Phys.Lett. B716 (2012) 30-61, arXiv:1207.7235."

Analysis workflow

The analysis workflow is simple and consists of three above-mentioned stages:

                           START
                             |
                             |
                             V
               +-------------------------+
               |           SCRAM         |
               +-------------------------+
                          /     \
                         /       \
                        /         \
+-------------------------+     +------------------------+
| process collision data  |     | process simulated data |
+-------------------------+     +------------------------+
                \                       /
                 \ Higgs4L1file.root   / DoubleMuParked2012C_10000_Higgs.root
                  \                   /
               +-------------------------+
               |    produce final plot   |
               +-------------------------+
                          |
                          | mass4l_combine_userlvl3.pdf
                          V
                         STOP

The steps processing collision data and simulated data can be run in parallel. We shall use the Snakemakeworkflow specification to express the computational workflow by means of the following Snakefile.

Run and monitor

We can now install the REANA command-line client, run the analysis and download the resulting plots:

source ~/.virtualenvs/reana/bin/activate
export REANA_SERVER_URL=https://reana.cern.ch/
export REANA_ACCESS_TOKEN=XXXXXXX
git clone [email protected]:reanahub/reana-demo-cms-h4l.git
cd reana-demo-cms-h4l
reana-client create -n KueueCMS
export REANA_WORKON=KueueCMS
reana-client upload
reana-client start

When the jobs described, you can verify that the jobs are scheduled by Kueue into the Kubernetes Cluster.

(reana) ➜  reana-demo-cms-h4l git:(master) ✗ k get jobs                                     NAME                                                   COMPLETIONS   DURATION   AGE
reana-run-batch-f399ff9f-3459-487d-a14d-3256d4428c91   0/1           3m20s      3m20s
reana-run-job-dee03c98-7216-4ed2-9ea6-3bf2778c88e7     0/1           2m29s      2m29s
reana-run-job-e5a8230f-9e1d-45dc-8168-c8b280aa5413     0/1           2m29s      2m29s

Workflow Submission Job

Check the events of the jobs:

kubectl describe job reana-run-batch-f399ff9f-3459-487d-a14d-3256d4428c91

!Pasted image 20240212223440.png

Job: 
reana-run-batch-f399ff9f-3459-487d-a14d-3256d4428c91 

Workload:
job-reana-run-batch-f399ff9f-3459-487d-a14d-3256d4428c91-c85b8

Pod:
reana-run-batch-f399ff9f-3459-487d-a14d-3256d4428c91-kdhc9 

...
Events:
  Type    Reason            Age    From                        Message
  ----    ------            ----   ----                        -------
  Normal  Suspended         2m58s  job-controller              Job suspended
  Normal  CreatedWorkload   2m58s  batch/job-kueue-controller  Created Workload: default/job-reana-run-batch-f399ff9f-3459-487d-a14d-3256d4428c91-c85b8
  Normal  Started           2m58s  batch/job-kueue-controller  Admitted by clusterQueue cluster-queue-reana-run-batch
  Normal  SuccessfulCreate  2m58s  job-controller              Created pod: reana-run-batch-f399ff9f-3459-487d-a14d-3256d4428c91-kdhc9
  Normal  Resumed           2m58s  job-controller              Job resumed

Pods are created once the workloads of each job are admitted:

!Pasted image 20240212224019.png

Workflow Task Job

kubectl describe job reana-run-job-dee03c98-7216-4ed2-9ea6-3bf2778c88e7
...
Events:
  Type    Reason            Age    From                        Message
  ----    ------            ----   ----                        -------
  Normal  Suspended         4m20s  job-controller              Job suspended
  Normal  CreatedWorkload   4m20s  batch/job-kueue-controller  Created Workload: default/job-reana-run-job-dee03c98-7216-4ed2-9ea6-3bf2778c88e7-0c013
  Normal  Started           4m20s  batch/job-kueue-controller  Admitted by clusterQueue cluster-queue-reana-run-job
  Normal  SuccessfulCreate  4m20s  job-controller              Created pod: reana-run-job-dee03c98-7216-4ed2-9ea6-3bf2778c88e7-tcz6q
  Normal  Resumed           4m20s  job-controller              Job resumed

REANA Validation

Kueue was successfully implemented into REANA locally.

!Pasted image 20240212224749.png

We can also verify jobs respected their dependencies for submission within the workload submission as well

!Pasted image 20240212232453.png

Clone this wiki locally