SD Card Optimization Guide

Overview

Raspberry Pi appliances using SD cards as boot volumes face a critical challenge: SD cards have limited write endurance. Consumer-grade SD cards are typically rated for 3,000-10,000 write cycles per block. With continuous logging and disk writes, an SD card can fail within months.

This guide provides comprehensive strategies to minimize SD card writes and maximize the lifespan of your GPS-disciplined NTP server appliance.

Expected Results: 80-90% reduction in SD card writes with proper configuration.

Quick Start

For automated setup of all recommended optimizations:

sudo ./scripts/setup-sd-optimization.sh
sudo reboot

The script configures:

journald volatile storage (RAM-only logging)
tmpfs mounts for temporary directories
Swap disabled
Aggressive log rotation
Application temp directories in RAM

Prerequisites: Ensure InfluxDB is configured in /etc/satclock/monitord.yaml before disabling persistent logging.

Why SD Card Optimization Matters

The Problem

Typical Raspberry Pi write patterns:

System logs: 10-50 MB/day
Application logs: 5-20 MB/day
Temporary files: Variable
Swap activity: 100+ MB/day (if enabled)
Total: 115-170+ MB/day

At this rate, focusing writes on the same blocks, an SD card can wear out in 6-18 months.

The Solution

Move ephemeral data to RAM and preserve only essential persistent data:

System logs → RAM (volatile)
Temporary files → RAM (tmpfs)
Metrics → InfluxDB (remote storage)
Configuration → SD card (minimal writes)

Optimization Strategies

1. journald Volatile Storage

Impact: Eliminates 60-80% of SD card writes

System logs are the primary source of SD card wear. journald can store logs entirely in RAM.

Configuration

The automated script installs this configuration to /etc/systemd/journald.conf.d/10-volatile-storage.conf:

[Journal]
# Store logs only in RAM (/run/log/journal/)
Storage=volatile

# Limit journal size in RAM to 50MB
RuntimeMaxUse=50M

# Ensure 50MB of RAM stays free
RuntimeKeepFree=50M

# Reduce maximum individual log file size
RuntimeMaxFileSize=10M

# Compress logs to save RAM
Compress=yes

Manual Setup

sudo mkdir -p /etc/systemd/journald.conf.d
sudo cp systemd/journald-volatile.conf /etc/systemd/journald.conf.d/10-volatile-storage.conf
sudo systemctl restart systemd-journald

Verification

# Check journal storage location (should show /run/log/journal/)
sudo journalctl --header | grep "File path"

# Verify journal size limit
journalctl --disk-usage

# Check that no journal files exist on disk
ls -lh /var/log/journal/  # Should not exist or be empty

Trade-offs

Pro: Massive reduction in SD card writes
Pro: Reduced disk space usage
Con: Logs lost on reboot
Mitigation: Use InfluxDB for persistent metrics, remote syslog for critical logs

2. tmpfs for Temporary Directories

Impact: Eliminates 10-20% of SD card writes

Mount /tmp and /var/tmp in RAM to prevent temporary file writes.

Configuration

Add to /etc/fstab:

tmpfs /tmp tmpfs defaults,noatime,nosuid,nodev,noexec,mode=1777,size=100M 0 0
tmpfs /var/tmp tmpfs defaults,noatime,nosuid,nodev,noexec,mode=1777,size=50M 0 0
tmpfs /var/run/satclock tmpfs defaults,noatime,nosuid,nodev,noexec,mode=0755,size=10M 0 0

Manual Setup

# Backup fstab
sudo cp /etc/fstab /etc/fstab.backup

# Add tmpfs entries (or run setup-sd-optimization.sh)
echo "tmpfs /tmp tmpfs defaults,noatime,nosuid,nodev,noexec,mode=1777,size=100M 0 0" | sudo tee -a /etc/fstab
echo "tmpfs /var/tmp tmpfs defaults,noatime,nosuid,nodev,noexec,mode=1777,size=50M 0 0" | sudo tee -a /etc/fstab
echo "tmpfs /var/run/satclock tmpfs defaults,noatime,nosuid,nodev,noexec,mode=0755,size=10M 0 0" | sudo tee -a /etc/fstab

# Mount immediately (or reboot)
sudo mount -a

Verification

mount | grep tmpfs
df -h | grep tmpfs

Expected output:

tmpfs           100M     0  100M   0% /tmp
tmpfs            50M     0   50M   0% /var/tmp
tmpfs            10M     0   10M   0% /var/run/satclock

3. Disable Swap

Impact: Eliminates 5-15% of SD card writes (potentially much more under memory pressure)

Swap on SD cards causes significant write amplification and wear. With sufficient RAM (1GB+), swap is unnecessary for this appliance.

Disable Swap

# Disable swap immediately
sudo swapoff -a

# Remove from fstab
sudo sed -i '/swap/d' /etc/fstab

# Disable dphys-swapfile service (Raspberry Pi)
sudo systemctl disable dphys-swapfile
sudo systemctl stop dphys-swapfile

# Optional: Remove swap file
sudo rm -f /var/swap

Verification

# Should show no swap
swapon --show
free -h

Considerations

RAM Requirements: Ensure system has adequate RAM (1GB minimum, 2GB+ recommended)
Monitor Memory: Use free -h and set up alerts if memory usage > 80%
OOM Protection: The kernel will kill processes before running out of memory

4. Filesystem Mount Optimizations

Impact: Eliminates 5-10% of metadata writes

The noatime mount option prevents access time updates on files, reducing write operations.

Configuration

Edit /etc/fstab and add noatime to the root filesystem options:

Before:

/dev/mmcblk0p2  /  ext4  defaults  0  1

After:

/dev/mmcblk0p2  /  ext4  defaults,noatime  0  1

Apply Changes

# Remount root filesystem with new options (or reboot)
sudo mount -o remount,noatime /

Verification

mount | grep " / "

Should show noatime in options.

5. Aggressive Log Rotation

Impact: Reduces log file accumulation

Even with volatile journald, some applications log to /var/log directly. Configure aggressive rotation.

Configuration

The setup script creates /etc/logrotate.d/sd-optimization:

/var/log/syslog
/var/log/messages
{
    rotate 2
    daily
    missingok
    notifempty
    compress
    delaycompress
    sharedscripts
}

/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
{
    rotate 2
    daily
    missingok
    notifempty
    compress
    delaycompress
}

This keeps only 2 days of logs instead of the default 4-7 weeks.

Manual Test

sudo logrotate -f /etc/logrotate.conf

6. Application Configuration

Impact: Eliminates application-specific writes

monitord Configuration

Edit /etc/satclock/monitord.yaml:

# Send metrics to InfluxDB (remote storage)
influxdb:
  enabled: true
  url: "http://your-influxdb-server:8086"
  database: "satclock"
  write_interval: 10s

# Reduce local file logging (rely on journald)
logging:
  level: "INFO"  # Avoid DEBUG level in production

Key Points:

Enable InfluxDB for persistent metrics storage
Use INFO log level (not DEBUG) to reduce log volume
Let journald handle logs (already configured for volatile storage)

webapp Configuration

Edit /etc/satclock/webapp.yaml:

# Webapp typically doesn't write files
# Ensure no file-based session storage or caching
logging:
  level: "INFO"

Monitoring SD Card Health

Check Write Activity

Monitor actual write activity to verify optimizations:

# Install iotop for I/O monitoring
sudo apt-get install iotop

# Monitor real-time disk writes (run for a few minutes)
sudo iotop -oPa

# Check total writes since boot
cat /sys/block/mmcblk0/stat

Estimate SD Card Lifespan

# Total writes in MB (8th column, multiply by 512 bytes per sector)
WRITES_SECTORS=$(awk '{print $7}' /sys/block/mmcblk0/stat)
WRITES_MB=$(echo "$WRITES_SECTORS * 512 / 1024 / 1024" | bc)
echo "Total writes since boot: ${WRITES_MB} MB"

# Calculate daily average after 24+ hours uptime
UPTIME_DAYS=$(awk '{print $1/86400}' /proc/uptime)
DAILY_WRITES=$(echo "$WRITES_MB / $UPTIME_DAYS" | bc)
echo "Average daily writes: ${DAILY_WRITES} MB/day"

Lifespan Estimation:

Good SD cards: 10,000 write cycles
32GB card = 32,000 MB capacity
Total write capacity = 32,000 MB × 10,000 cycles = 320,000,000 MB
Optimized writes: ~20 MB/day
Estimated lifespan: 43+ years

(Note: Actual lifespan varies by SD card quality, wear leveling, and write distribution)

Before and After Comparison

Typical Configuration (No Optimization)

# Daily writes: 115-170 MB
- journald to disk: 80 MB/day
- Swap activity: 20 MB/day
- Temporary files: 10 MB/day
- Application logs: 5 MB/day

# Estimated SD card lifespan: 6-18 months

Optimized Configuration

# Daily writes: 10-20 MB
- Configuration updates: 5 MB/day
- System updates: 5 MB/day
- Critical persistent data: 2 MB/day

# Estimated SD card lifespan: 43+ years

Write Reduction: 85-90%

Verification Checklist

After applying optimizations, verify each component:

# 1. journald volatile storage
sudo journalctl --header | grep "File path"  # Should show /run/log/journal/
journalctl --disk-usage                       # Should show < 50MB
ls /var/log/journal/ 2>/dev/null              # Should not exist or be empty

# 2. tmpfs mounts
mount | grep tmpfs                            # Should show /tmp, /var/tmp, /var/run/satclock

# 3. Swap disabled
swapon --show                                 # Should be empty
free -h | grep Swap                           # Should show 0B

# 4. noatime on root
mount | grep " / "                            # Should show noatime

# 5. Log rotation
ls -lh /var/log/syslog*                       # Should show only 2-3 files

# 6. InfluxDB enabled
grep -A 3 "influxdb:" /etc/satclock/monitord.yaml
curl http://localhost:8080/api/status | jq .influxdb_connected

Troubleshooting

"Cannot find logs after reboot"

Expected behavior - logs are volatile (RAM-only).

Solution: Use InfluxDB for persistent metrics. For critical system logs, configure remote syslog:

# Install rsyslog if needed
sudo apt-get install rsyslog

# Configure remote logging in /etc/rsyslog.conf
echo "*.* @your-log-server:514" | sudo tee -a /etc/rsyslog.conf
sudo systemctl restart rsyslog

"Out of memory errors"

Check RAM usage and adjust tmpfs sizes:

free -h
df -h | grep tmpfs

Reduce tmpfs sizes in /etc/fstab if needed:

/tmp: 100M → 50M
/var/tmp: 50M → 25M

"Application crashes or errors"

Some applications expect persistent /tmp. Check logs:

sudo journalctl -u satclock-monitord -n 100
sudo journalctl -u satclock-webapp -n 100

If needed, create persistent temp directory:

sudo mkdir -p /var/lib/satclock/tmp
# Update application config to use this directory

"Want to check a previous boot's logs"

Not possible with volatile storage. Options:

Configure remote syslog for critical systems
Use InfluxDB dashboards for metrics history
Use persistent journald for important systems (trade SD card lifespan)

Advanced Optimizations

Read-Only Root Filesystem

For maximum SD card protection, mount root filesystem as read-only with overlayfs for writable areas.

Complexity: High Benefit: Near-zero SD card writes Trade-off: More complex updates and configuration changes

See Raspberry Pi documentation for overlayroot setup.

Industrial-Grade SD Cards

Consider industrial/MLC SD cards for critical deployments:

Rated for 100,000+ write cycles (10x consumer cards)
Better wear leveling algorithms
Extended temperature ranges
Examples: SanDisk Industrial, Transcend Industrial

Cost: 3-5x consumer cards Benefit: 10x lifespan even without optimizations

Battery-Backed RAM Disk

For critical applications requiring persistent logs across reboots while minimizing SD writes:

Use battery-backed RAM module
Sync logs to remote storage periodically
Flush critical logs to SD only during shutdown

Best Practices

Always configure InfluxDB before disabling persistent logging
- Metrics must be preserved somewhere
- InfluxDB provides better analytics than text logs
Monitor RAM usage after enabling tmpfs
- Ensure adequate headroom (20%+ free)
- Adjust tmpfs sizes based on actual usage
Test backups and recovery procedures
- Volatile logs mean less forensic data after crashes
- Document known-good configurations
Use quality SD cards
- Prefer SanDisk, Samsung, or industrial-grade cards
- Avoid cheap no-name brands
Keep spare SD cards
- Clone working configurations
- Have hot-swappable backup ready
Monitor SD card health
- Check write statistics weekly
- Replace proactively if write rate increases

Summary

Implementing these optimizations transforms your Raspberry Pi from writing 115-170 MB/day to just 10-20 MB/day - an 85-90% reduction.

Recommended Implementation Order:

Configure InfluxDB (preserve metrics)
Run ./scripts/setup-sd-optimization.sh
Reboot
Verify with checklist
Monitor for 1 week
Adjust as needed

Result: Your GPS-disciplined NTP server appliance will have a reliable, long-lasting SD card with minimal maintenance.

FilesExpand file tree

SD_CARD_OPTIMIZATION.md

Latest commit

History

SD_CARD_OPTIMIZATION.md

File metadata and controls

SD Card Optimization Guide

Overview

Quick Start

Why SD Card Optimization Matters

The Problem

The Solution

Optimization Strategies

1. journald Volatile Storage

Configuration

Manual Setup

Verification

Trade-offs

2. tmpfs for Temporary Directories

Configuration

Manual Setup

Verification

3. Disable Swap

Disable Swap

Verification

Considerations

4. Filesystem Mount Optimizations

Configuration

Apply Changes

Verification

5. Aggressive Log Rotation

Configuration

Manual Test

6. Application Configuration

monitord Configuration

webapp Configuration

Monitoring SD Card Health

Check Write Activity

Estimate SD Card Lifespan

Before and After Comparison

Typical Configuration (No Optimization)

Optimized Configuration

Verification Checklist

Troubleshooting

"Cannot find logs after reboot"

"Out of memory errors"

"Application crashes or errors"

"Want to check a previous boot's logs"

Advanced Optimizations

Read-Only Root Filesystem

Industrial-Grade SD Cards

Battery-Backed RAM Disk

Best Practices

Summary

Related Documentation