Disk Hygiene

An interactive disk cleanup tool for HPC/server environments with email reports and a macOS GUI for reviewing and deleting files.

Features

• Interactive CLI cleanup with fzf for file selection
• Weekly email reports with disk usage summary
• macOS integration - click a button in the email to open an interactive checklist in your browser
• Smart file detection:
  • Large files (>500MB)
  • Old files (>90 days)
  • Uncompressed NGS files (.fastq, .fq, .sam)
  • Cache directories (R, pip, conda, renv, jupyter, snakemake, nextflow)
  • Log files and core dumps
• Confirmation view before deletion with file preview
• Bold filenames for easy scanning

Screenshots

Email Report

The weekly email shows disk usage and lists files that could be cleaned up:

Disk Hygiene Report
━━━━━━━━━━━━━━━━━━━
Scratch: 250G    Home: 13G

[Open Cleanup Checklist]  ← Click to review in browser

Browser Checklist

Interactive HTML interface with checkboxes:

┌─────────────────────────────────────────────────────┐
│  Disk Hygiene Review                                │
│  Total: 52 items | 180G                             │
│                                                     │
│  [Select All] [Old >90d] [Large >1GB] [Filter: ___] │
│                                                     │
│  ☐  5.2G  35d  large  /scratch/project/data.tar    │
│  ☐  3.1G  120d old    /scratch/old_analysis/       │
│  ☑  1.2G  0d   cache  ~/.cache/R                   │
│                                                     │
│                           [Delete Selected]         │
└─────────────────────────────────────────────────────┘

Installation

Quick Install (Server)

One-liner with automatic cron setup:

curl -fsSL https://raw.githubusercontent.com/uzh/disk-hygiene/main/install.sh | bash -s -- \
  --email [email protected] \
  --scratch /scratch/username \
  --cron

Or minimal install (no email):

curl -fsSL https://raw.githubusercontent.com/uzh/disk-hygiene/main/install.sh | bash

This will:

• Install scripts to ~/bin
• Add ~/bin to your PATH
• Add login notification for pending reports
• (with --cron) Set up weekly email reports

Manual Install (Server)

# Clone the repository
git clone https://github.com/uzh/disk-hygiene.git
cd disk-hygiene

# Install server scripts
mkdir -p ~/bin
cp server/disk-hygiene ~/bin/
cp server/disk-hygiene-report ~/bin/
chmod +x ~/bin/disk-hygiene ~/bin/disk-hygiene-report

# Add ~/bin to PATH (add to ~/.bashrc)
export PATH="$HOME/bin:$PATH"

# (Optional) Setup cron for weekly reports
crontab -e
# Add: 30 7 * * 1 [email protected] SCRATCH_DIR=/scratch/user ~/bin/disk-hygiene-report --email

macOS Setup (for email button)

Run the installer on your Mac:

# Download and run
curl -O https://raw.githubusercontent.com/uzh/disk-hygiene/main/macos/install.sh
chmod +x install.sh
./install.sh [email protected]

This installs a URL handler so clicking the button in the email opens the interactive checklist.

Usage

Interactive Mode (Terminal)

# Scan and interactively select files to delete
disk-hygiene

# Review a saved report from email
disk-hygiene --review

# Review a specific report
disk-hygiene --review ~/.disk-hygiene/reports/2024-01-15.json

# List available reports
disk-hygiene --list-reports

# Generate report only (no deletion)
disk-hygiene --report

Environment Variables

Variable	Default	Description
SCRATCH_DIR	~/scratch	Path to scratch directory
HOME_DIR	~	Path to home directory
LARGE_FILE_THRESHOLD	500M	Size threshold for large files
OLD_FILE_DAYS	90	Age threshold for old files
DISK_HYGIENE_EMAIL	(none)	Email address for reports

Examples

# Custom scratch directory
SCRATCH_DIR=/cluster/scratch/user disk-hygiene

# Send email report
[email protected] SCRATCH_DIR=/scratch/user disk-hygiene-report --email

# Generate HTML report to stdout
disk-hygiene-report --html > report.html

# Generate JSON report
disk-hygiene-report --json > report.json

How It Works

1. Report Generation (disk-hygiene-report --email)

   • Scans directories for large, old, and unnecessary files
   • Saves findings to ~/.disk-hygiene/reports/YYYY-MM-DD.json
   • Sends HTML email with summary and "Open Checklist" button

2. Email Button Click (requires macOS setup)

   • Opens disk-hygiene://review/YYYY-MM-DD URL
   • macOS handler fetches JSON from server via SSH
   • Generates HTML checklist and opens in browser

3. File Selection

   • Check/uncheck files in the browser
   • Use quick filters: Select All, Old (>90d), Large (>1GB)
   • Filter by path with search box

4. Deletion

   • Click "Delete Selected" to see confirmation view
   • Review files with bold filenames
   • Click "Delete These Files" to execute
   • iTerm opens with SSH command to delete files

File Categories

Category	Criteria	Badge Color
Large	>500MB	Red
Old	>90 days, >10MB	Orange
Uncompressed	.fastq, .fq, .sam >10MB	Teal
Cache	R, pip, conda, etc. >50MB	Purple
Log	.log, .out, .err >50MB	Gray

Requirements

Server

• Bash 4+
• Standard Unix tools (find, du, df, mail)
• fzf (optional, for best CLI experience)

macOS Client

• macOS 10.13+
• iTerm2 (for deletion command execution)
• SSH key authentication to server

Troubleshooting

Email button doesn't work

1. Make sure you ran the macOS installer
2. Test the URL handler: open "disk-hygiene://review/2024-01-15"
3. Check logs: cat ~/.disk-hygiene-local/handler.log

SSH connection fails

1. Ensure SSH key authentication is set up
2. Test manually: ssh user@server "echo ok"
3. Check the SSH host in ~/.disk-hygiene-local/handler.sh

No files found

1. Check SCRATCH_DIR is set correctly
2. Verify directory permissions
3. Run disk-hygiene --report to see what's detected

License

MIT License - see LICENSE for details.

Contributing

Contributions welcome! Please open an issue or pull request.

Credits

Developed at Functional Genomics Center Zurich (FGCZ).