Skip to content

Slurm does not clean cgroupv2 #6

@rezib

Description

@rezib

For an unknown reason, Slurm does not properly clean cgroupv2 hierarchy and it ends up being full and fail with no space left in cgroup virtual filesystem.

A workaround is to clean empty cgroups on compute nodes, with:

On Debian:

# clush -bw cn[1-4] 'find /sys/fs/cgroup/system.slice/cn?_slurmstepd.scope -depth -type d -exec rmdir {} +'

On Rocky:

$ clush -bw cn[1-4] 'find /sys/fs/cgroup/system.slice/slurmstepd.scope -depth -type d -exec rmdir {} +

Metadata

Metadata

Assignees

No one assigned

    Labels

    slurmIssues related to Slurm workload manager itself

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions