Alpha e2e tests fail often

[afrittoli]

Expected Behavior

Tests are successful most of the times

Actual Behavior

From the latest failure:

arendelle-dx92j              9m55s       Normal    PullImageFailed                taskrun/pipelinerun-consume-tasks-artifactswsxvb-consume-artifacts        build step "step-artifacts-consumer-python" is pending with reason "failed to pull and unpack image \"docker.io/library/python:latest\": failed to extract layer sha256:08e14ec5b7497da231d70d47d1d80440ba7d9997d43c0796a8394923bbc98183: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/631/fs/usr/lib/x86_64-linux-gnu/libicui18n.a: no space left on device"
arendelle-dx92j              9m41s       Normal    PullImageFailed                taskrun/pipelinerun-consume-tasks-artifactswsxvb-consume-artifacts        build step "step-artifacts-consumer-python" is pending with reason "Back-off pulling image \"python:latest\": ErrImagePull: failed to pull and unpack image \"docker.io/library/python:latest\": failed to extract layer sha256:08e14ec5b7497da231d70d47d1d80440ba7d9997d43c0796a8394923bbc98183: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/631/fs/usr/lib/x86_64-linux-gnu/libicui18n.a: no space left on device"
arendelle-dx92j              9m41s       Warning   Failed                         taskrun/pipelinerun-consume-tasks-artifactswsxvb-consume-artifacts        the step "artifacts-consumer-python" in TaskRun "pipelinerun-consume-tasks-artifactswsxvb-consume-artifacts" failed to pull the image "". The pod errored with the message: "Back-off pulling image "python:latest": ErrImagePull: failed to pull and unpack image "docker.io/library/python:latest": failed to extract layer sha256:08e14ec5b7497da231d70d47d1d80440ba7d9997d43c0796a8394923bbc98183: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/631/fs/usr/lib/x86_64-linux-gnu/libicui18n.a: no space left on device."

Since this seems to happen mainly on alpha, I imagine the test in question only runs with alpha.
We could change the specific test to use a smaller image and/or change the configuration of the kind cluster to increase the disk space for the cluster nodes (to be verified if possible).

Steps to Reproduce the Problem

1. Create a PR or check https://github.com/tektoncd/pipeline/actions/runs/18131929619/job/51630667788?pr=9050

Additional Info

• Kubernetes version:

Happens on various k8s versions

• Tekton Pipeline version:

main

[afrittoli]

AI suggests doing some cleanup on the runner beforehand:

# Add before kind cluster creation
sudo rm -rf /usr/share/dotnet
sudo rm -rf /usr/local/lib/android
sudo rm -rf /opt/ghc
docker system prune -a -f
df -h  # Check available space

[afrittoli]

I tried this in the catalog, it does free up ~15GB, it takes a few minutes (3 to 5) to run.

--- Disk space before cleanup ---
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        72G   49G   23G  69% /
tmpfs           7.9G   84K  7.9G   1% /dev/shm
tmpfs           3.2G  1.1M  3.2G   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/sdb16      881M   60M  760M   8% /boot
/dev/sdb15      105M  6.2M   99M   6% /boot/efi
/dev/sda1        74G  4.1G   66G   6% /mnt
tmpfs           1.6G   12K  1.6G   1% /run/user/1001
Total reclaimed space: 0B
--- Disk space after cleanup ---
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        72G   35G   38G  48% /
tmpfs           7.9G   84K  7.9G   1% /dev/shm
tmpfs           3.2G  1.1M  3.2G   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/sdb16      881M   60M  760M   8% /boot
/dev/sdb15      105M  6.2M   99M   6% /boot/efi
/dev/sda1        74G  4.1G   66G   6% /mnt
tmpfs           1.6G   12K  1.6G   1% /run/user/1001

[afrittoli]

I added this to this PR #9050

[vdemeester]

Seems like it help for that error, but other errors keeps creeping up.