feat: add support for downloading history logs via dlog

Extends the `dlog` command to handle Redis LIST keys (history_* and
failed_tests*) in addition to gzipped string logs. Also adds `yarn ci`
alias for accessing ci.sh from yarn-project and updates agent docs.

Co-Authored-By: Claude Opus 4.5 <[email protected]>

Author: spalladino

ci.sh

@@ -220,7 +220,16 @@ case "$cmd" in
     fi
     pager=${PAGER:-less}
     [ ! -t 0 ] && pager=cat
-    redis_getz $1 | $pager
+    key=$1
+    # Handle list/* URLs or history_* keys (Redis LISTs, not strings)
+    if [[ "$key" == list/* ]]; then
+      key=${key#list/}
+    fi
+    if [[ "$key" == history_* || "$key" == failed_tests* ]]; then
+      redis_cli LRANGE "$key" 0 -1 | $pager
+    else
+      redis_getz "$key" | $pager
+    fi
     ;;
 
   #################

yarn-project/.claude/agents/analyze-logs.md

@@ -228,10 +228,7 @@ Do not use `diff` as an analysis, since timestamps or random values will mean th
 ## Key Behavior
 
 1. **Use local file if provided** - never re-download
-2. If only hash given and file not at `/tmp/<hash>.log`, download via:
-   ```bash
-   ./ci.sh dlog <hash> > /tmp/<hash>.log 2>&1
-   ```
+2. If only hash or `ci.aztec-labs.com` url given and file not at `/tmp/<hash>.log`, then download via `yarn ci dlog <hash> > /tmp/<hash>.log 2>&1`
 3. **Always check Jest report at end first** to identify actual failure
 4. For huge logs (>10k lines): grep for ERROR/WARN first, then expand context
 5. **Return summary, not raw content** - extract only relevant snippets

yarn-project/.claude/agents/identify-ci-failures.md

@@ -64,24 +64,34 @@ gh run view <RUN_ID> --repo AztecProtocol/aztec-packages --log 2>&1 | grep -i "C
 
 ### Step 2: Download Main Log
 
-From repo root:
+Run from `yarn-project`:
 ```bash
-./ci.sh dlog <hash> > /tmp/<hash>.log 2>&1
+yarn ci dlog <hash> > /tmp/<hash>.log 2>&1
 ```
 
 Verify download:
 ```bash
 wc -l /tmp/<hash>.log
 ```
 
+Never use the `Fetch` tool or `curl` for downloading.
+
 ### Step 3: Check for History URL
 
 Look in the first 20 lines for a `History:` URL:
 ```bash
 head -20 /tmp/<hash>.log | grep -i "history"
 ```
 
-If found, this can be used to find successful runs for comparison.
+If found (e.g., `http://ci.aztec-labs.com/list/history_e3d65d7f0d7a03b5_next`), this can be used to find successful runs for comparison.
+
+**To download history:**
+```bash
+# Extract the key from the URL (everything after /list/)
+yarn ci dlog history_e3d65d7f0d7a03b5_next > /tmp/history_e3d65d7f0d7a03b5_next.log 2>&1
+```
+
+The history log contains timestamped entries showing PASSED/FAILED/FLAKED runs with commit info. Use this to find a recent successful run's hash for comparison.
 
 ### Step 4: Find Failure Type
 
@@ -113,7 +123,7 @@ grep -E "compile_all.*failed|FAIL|ERROR|test all" /tmp/<hash>.log
 
 For each nested log URL found (format: `http://ci.aztec-labs.com/<hash>`):
 ```bash
-./ci.sh dlog <hash> > /tmp/<hash>.log 2>&1
+yarn ci dlog <hash> > /tmp/<hash>.log 2>&1
 ```
 
 ### Step 6: Extract Error Details
@@ -142,6 +152,18 @@ FAIL src/e2e_something.test.ts (123.456s)
 error TS2345: Argument of type 'X' is not assignable to parameter of type 'Y'
 ```
 
+**History log format** (from `yarn ci dlog history_*`):
+```
+MM-DD HH:MM:SS: PASSED (http://ci.aztec-labs.com/<hash>): test_command (duration) (author: commit message)
+MM-DD HH:MM:SS: FAILED (http://ci.aztec-labs.com/<hash>): test_command (duration) (code: 1) (author: commit message)
+MM-DD HH:MM:SS: FLAKED (http://ci.aztec-labs.com/<hash>): test_command (duration) (code: 1) (author: commit message)
+```
+Each line represents a past test run. Extract the hash from PASSED entries to download successful run logs for comparison:
+```bash
+grep "PASSED" /tmp/history.log | head -1  # Find a successful run
+yarn ci dlog <hash>  # Download the successful run's logs
+```
+
 ## Key Principles
 
 - **Download once**: Save all logs to `/tmp/<hash>.log` for reuse

yarn-project/.claude/skills/debug-e2e/SKILL.md

@@ -136,7 +136,7 @@ To understand when a test started failing:
 4. Check the PR mentioned in the commit message to understand what changed
 5. Download logs from both passing and failing runs to compare:
    - Use hash from history (e.g., `2614d91ec48f4047` for passed, `10d5f47f04025f1c` for failed)
-   - `./ci.sh dlog <hash>` downloads the log
+   - `yarn ci dlog <hash> > /tmp/<hash>.log 2>&1` downloads the log to a local tmp file
 
 **Important**: Do NOT use `gh run list` - the history in the log file is more accurate for this specific test.
 
@@ -252,4 +252,4 @@ Claude: [Searches codebase for MAXIMUM_GOSSIP_CLOCK_DISPARITY]
         Found in p2p/config.ts, default is 2 seconds. The PR added this validation
         but epoch proofs can take longer to propagate. Recommend increasing the
         disparity threshold for epoch-related messages.
-```
+```

yarn-project/CLAUDE.md

@@ -48,14 +48,6 @@ Always run `yarn build` from the `yarn-project` root. Never run `tsgo` directly,
 yarn build
 ```
 
-### CI Log Access
-
-Download and view CI logs from the repo root:
-
-```bash
-./ci.sh dlog <hash>   # Retrieves logs from CI cache
-```
-
 ### Before Committing (Quality Checklist)
 
 Run from `yarn-project`:

yarn-project/package.json

@@ -3,6 +3,7 @@
   "packageManager": "[email protected]",
   "private": true,
   "scripts": {
+    "ci": "../ci.sh",
     "prepare": "node ./scripts/update_package_jsons.mjs && yarn workspaces foreach -A run prepare && workspaces-to-typescript-project-references --tsconfigPath tsconfig.json && prettier -w */tsconfig.json",
     "prepare:check": "node ./scripts/update_package_jsons.mjs --check && workspaces-to-typescript-project-references --check --tsconfigPath tsconfig.json",
     "docs": "typedoc --out docs/dist && cd docs && yarn serve",