Skip to content

Commit 14ccd84

Browse files
committed
Extend corrupt-gz eviction to all .gz files in cache shards
Earlier scan only looked at .styxcache.stdout.gz / .stderr.gz, but nibabel also reads cached .nii.gz outputs directly from shard paths. A truncated cached BOLD blew up single_session_qc with the same EOFError signature.
1 parent ccbd34e commit 14ccd84

1 file changed

Lines changed: 14 additions & 7 deletions

File tree

.github/workflows/test_full.yaml

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -50,16 +50,23 @@ jobs:
5050
# Purge stale staging dirs from crashed previous runs (safe: atomic-rename commits)
5151
rm -rf "$STYXCACHE_DIR/.incoming" 2>/dev/null || true
5252
53-
# Drop entries with corrupt .gz streams (from crashed runs that committed
54-
# before their stdout finished writing). Reading a truncated gz blows up
55-
# the reading test; cheaper to detect and evict than investigate.
53+
# Drop entries that contain any truncated .gz (styxcache's own
54+
# stdout/stderr streams or cached .nii.gz outputs). Atomic commits
55+
# should prevent these, but crashes before commit have been observed.
5656
corrupt=0
57-
while IFS= read -r gz; do
58-
if ! gzip -t "$gz" 2>/dev/null; then
59-
rm -rf "$(dirname "$gz")"
57+
while IFS= read -r entry; do
58+
bad=0
59+
for gz in $(find "$entry" -type f -name '*.gz'); do
60+
if ! gzip -t "$gz" 2>/dev/null; then
61+
bad=1
62+
break
63+
fi
64+
done
65+
if [ "$bad" = "1" ]; then
66+
rm -rf "$entry"
6067
corrupt=$((corrupt + 1))
6168
fi
62-
done < <(find "$STYXCACHE_DIR" -type f \( -name '.styxcache.stdout.gz' -o -name '.styxcache.stderr.gz' \) 2>/dev/null)
69+
done < <(find "$STYXCACHE_DIR" -mindepth 2 -maxdepth 2 -type d 2>/dev/null)
6370
if [ "$corrupt" -gt 0 ]; then
6471
echo "evicted $corrupt entries with corrupt gzip streams"
6572
fi

0 commit comments

Comments
 (0)