Skip to content

bug(shell): SIGPIPE sweep — '| head -1 |' under pipefail kills 4 scripts (multi-GPU, multi-model, duplicate .env) #456

@yasinBursali

Description

@yasinBursali

Severity: High
Category: Shell Compatibility / Error Handling
Platform: See per-site analysis
Confidence: Confirmed

Description

Four shell scripts use the pattern <cmd> | head -1 | ... under set -euo pipefail. When the upstream command produces multiple lines, head -1 terminates after reading line 1 and sends SIGPIPE (exit 141) to the upstream process. With pipefail, the pipeline's exit becomes 141, propagating up and killing the script — even though the intent was "take the first line and continue". This is a known project pattern (memory note: sed -n '1p' is the SIGPIPE-safe replacement). All four sites should be fixed together.

Affected File(s)

  • dream-server/scripts/pre-download.sh:112nvidia-smi --query-gpu=memory.total ... | head -1 | awk ...
  • dream-server/scripts/dream-preflight.sh:87docker exec ... nvidia-smi --query-gpu=memory.free ... | head -1 | tr -d ' '
  • dream-server/scripts/check-offline-models.sh:27ls -1 data/models/*.gguf | head -1
  • dream-server/installers/macos/dream-macos.sh:158grep -E "^${key}=" "$env_file" | head -n 1 | cut -d'=' -f2- | tr -d '\r' || true

Note: dream-server/scripts/dream-test-functional.sh:66,84,230 — already addressed by open PR Light-Heart-Labs#1003 via if ! VAR=$(...) wrapper; NOT in scope of this issue.
Note: dream-server/dream-cli — line 254 already addressed by open PR Light-Heart-Labs#1008. Remaining dream-cli sites use the local VAR=$(...) form which masks the pipeline exit status and is already safe.

Root Cause

set -euo pipefail (enforced at the top of each of the 4 scripts) means the first non-zero pipeline exit kills the script. head -1 reading 1 line and closing its stdin causes SIGPIPE on the upstream producer (nvidia-smi, ls, grep) → exit 141. Under pipefail, the whole pipeline exits 141. The surrounding code was written when set -e alone was active (no pipefail), so it assumed a multi-line input was safe.

Evidence

# pre-download.sh:112
nvidia-smi --query-gpu=memory.total --format=csv,noheader,nounits 2>/dev/null | head -1 | awk '{print int($1/1024)}'

# dream-preflight.sh:87
GPU_MEM=$(docker exec "$LLM_CONTAINER" nvidia-smi --query-gpu=memory.free --format=csv,noheader,nounits 2>/dev/null | head -1 | tr -d ' ')

# check-offline-models.sh:27
MODEL_FILE=$(ls -1 data/models/*.gguf | head -1)

# dream-macos.sh:158
grep -E "^${key}=" "$env_file" 2>/dev/null | head -n 1 | cut -d'=' -f2- | tr -d '\r' || true

Platform Analysis

  • pre-download.sh / dream-preflight.sh: Linux NVIDIA multi-GPU only (nvidia-smi emits N lines on N-GPU hosts). Single-GPU unaffected. macOS / Windows-WSL2 do not run these paths.
  • check-offline-models.sh: Linux and macOS (any user with 2+ .gguf files). Windows-WSL2 not affected (script not invoked there).
  • dream-macos.sh: macOS only. SIGPIPE fires if .env contains a duplicate key (e.g. from a buggy upgrade). Trailing || true catches script abort but VAR still holds truncated/empty output, silently returning wrong value for GPU_BACKEND, LLAMA_CPU_LIMIT, LLAMA_CPU_RESERVATION.

Reproduction

  • pre-download.sh: Any dual-GPU NVIDIA Linux host. Run scripts/pre-download.sh — exits with SIGPIPE-related error before model selection.
  • dream-preflight.sh: Same hardware. Run scripts/dream-preflight.sh — aborts without completing service checks.
  • check-offline-models.sh: Place two .gguf files under data/models/; run scripts/check-offline-models.sh — pipeline abort.
  • dream-macos.sh: Manually duplicate a line in .env (e.g. two GPU_BACKEND= lines); call a dream subcommand that uses read_env_value GPU_BACKEND — returned value silently wrong.

Impact

  • Multi-GPU NVIDIA Linux users cannot complete preflight or model pre-download.
  • Users with multiple downloaded models see offline check crash instead of a clean report.
  • macOS users with an accidentally duplicated .env line see wrong values used silently (no error, no warning).

Suggested Approach

Replace | head -1 | (and | head -n 1 |) with | sed -n '1p' |. sed reads all input and terminates after printing line 1 without generating SIGPIPE to the upstream process. Works identically on BSD (macOS) and GNU (Linux) sed. For check-offline-models.sh, find data/models/ -maxdepth 1 -name "*.gguf" | sed -n '1p' also normalizes ordering (locale-free).

Labels

bug, shell-compatibility, error-handling, pipefail, multi-platform

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions