GOB index corruption on unclean shutdown — crash loop with 'unexpected EOF'

[coolmanns]

Description

When the system shuts down uncleanly (reboot, OOM kill, signal) while grepai watch is running, index.gob gets truncated. On restart, grepai fails with:

[ERROR] failed to load index: failed to decode index: unexpected EOF

This triggers an infinite crash loop since the corrupted file persists and grepai has no recovery path.

Root Cause

index.gob appears to be written directly (non-atomically). If the process is interrupted mid-write, the file is left in a partial/truncated state (e.g., 384 bytes instead of the expected ~14-200MB).

Impact

In my case, a server reboot on March 4 corrupted the index. grepai crash-looped for 3 days (15,000+ restart attempts) before I noticed and manually deleted the index file.

Suggested Fix

Use atomic writes for index.gob:

1. Write to a temporary file (index.gob.tmp)
2. fsync the temp file
3. Rename index.gob.tmp → index.gob (atomic on POSIX)

Alternatively, detect the corruption on startup and automatically rebuild instead of exiting with an error.

Workaround

I wrote a systemd ExecStartPre script that checks index.gob size (<1KB = corrupted) and deletes it before grepai starts:

#!/bin/bash
INDEX="/path/to/.grepai/index.gob"
MIN_SIZE=1024
if [ -f "$INDEX" ]; then
    size=$(stat -c%s "$INDEX" 2>/dev/null || echo 0)
    if [ "$size" -lt "$MIN_SIZE" ]; then
        echo "index.gob is ${size} bytes — likely corrupted. Removing."
        rm -f "$INDEX" "$INDEX.lock"
    fi
fi

Environment

• grepai v0.9.x (latest as of March 2026)
• Backend: GOB
• OS: Ubuntu Linux 6.17
• Service manager: systemd (user unit)