Skip to content

Commit 4e9ccfb

Browse files
authored
Update README.md
1 parent afb45b7 commit 4e9ccfb

File tree

1 file changed

+0
-45
lines changed

1 file changed

+0
-45
lines changed

README.md

Lines changed: 0 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -17,48 +17,3 @@ flowchart LR
1717
ActionsRunners --> | Poll Queued Jobs | GitHubAPI
1818
ActionsRunners -->| Allocate Actions Runner| Slurm
1919
```
20-
21-
## Enabling Docker Within CI Jobs
22-
23-
```mermaid
24-
graph TD
25-
A[Docker Rootless Daemon] -->| Creates | B[Docker Rootless Socket]
26-
B -->| Creates | C[Custom Actions Runner Image]
27-
C -.->| Calls | B
28-
C --->| Mounts | B
29-
C -->| Creates | E[CI Helper Containers]
30-
E -.->| Calls | B
31-
```
32-
33-
Since CI Docker commands will use the same filesystem, as they have the same Docker socket, you need to configure the working directory of your runners accordingly.
34-
35-
## Speeding up our Actions Runner Image
36-
37-
After we were able to run the actions runner image in as Slurm job using [sbatch](https://slurm.schedmd.com/sbatch.html) and [custom script](https://github.com/WATonomous/run-gha-on-slurm/blob/main/allocate-ephemeral-runner-from-docker.sh) we ran into the issue of having to pull the docker image for every job. From the time the script allocated resources to the time the job began was ~ 2 minutes. When you are running 70+ jobs in a workflow, with some jobs depending on others, this time adds up fast.
38-
39-
Unfortunately, caching the image is not an elegant solution because this would require mounting the filesystem directory to the Slurm job. This means we would need to have multiple directories if we wanted to support multiple concurrent runners. This would require creating a system to manage these directories and would introduce the potenital for starvation and dead locks.
40-
41-
This led us to investegate a [Docker pull through cache](https://docs.docker.com/docker-hub/mirror/).
42-
43-
44-
### References
45-
1. [Docker Rootless](https://docs.docker.com/engine/security/rootless/)
46-
2. [Custom Actions Runner Image](https://github.com/WATonomous/actions-runner-image)
47-
3. [Apptainer](https://apptainer.org/docs/user/main/index.html)
48-
4. [Stargz Snapshotter](https://github.com/containerd/stargz-snapshotter)
49-
5. [CVMFS](https://cvmfs.readthedocs.io/en/stable/)
50-
6. [CVMFS Stratum 0](https://github.com/WATonomous/cvmfs-ephemeral/)
51-
52-
53-
# Issues
54-
- If script needs to be restart and runners are being built, the script will allocate new runners once its back up
55-
56-
# Potential issue:
57-
- job1 requires label1, label2
58-
- job2 requires label1
59-
- runner1 is allocated with label1, label2
60-
- runner1 runs job2
61-
- runner2 is allocated with label1
62-
- runner2 CANT RUN job1
63-
Won't be an issue if we use one label (small, medium, large) per job
64-

0 commit comments

Comments
 (0)