Skip to content

Add VM serial console logging for fast-cycle diagnosis#12

Merged
guifranchi merged 1 commit into
mainfrom
fix/vm-console-logging
May 21, 2026
Merged

Add VM serial console logging for fast-cycle diagnosis#12
guifranchi merged 1 commit into
mainfrom
fix/vm-console-logging

Conversation

@guifranchi

@guifranchi guifranchi commented May 21, 2026

Copy link
Copy Markdown
Contributor

Problem

VMs cycle in ~1–2 seconds with exit_code=0, but we have no visibility into what fails inside them. Firecracker's stdout (the VM serial console, where systemd/journald output goes) was discarded via Stdio::null().

Changes

  • console=ttyS0 added to DEFAULT_BOOT_ARGS — routes the VM kernel and systemd output to Firecracker's stdout (the serial console)
  • {work_dir}/console.log per slot — Firecracker stdout is now written to a file instead of /dev/null
  • Console tail logging in slot.rs — after any cycle shorter than 60 seconds, the last 40 lines of console.log are emitted to the manager journal at WARN level, with the cycle field for correlation
  • duration_secs added to the "Firecracker exited" log line for easier cycle-length filtering

Effect

journalctl -u actions-runner will now show the VM's systemd boot output, service start/stop messages, and any error from config.sh or runner.service immediately after each short cycle — no manual Firecracker test needed to diagnose failures.

VMs cycling in <60 seconds leave no trace of what failed inside them.
Firecracker's stdout (the serial console) was previously discarded.

- Add console=ttyS0 to DEFAULT_BOOT_ARGS so the VM's kernel and
  systemd output goes to Firecracker's stdout
- Redirect Firecracker stdout to {work_dir}/console.log per slot
- After any cycle shorter than 60 seconds, log the last 40 lines of
  the console to the manager journal so failures are visible without
  manual intervention
- Include duration_secs in the "Firecracker exited" log line

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@guifranchi guifranchi self-assigned this May 21, 2026
@guifranchi guifranchi added the bug label May 21, 2026
@backlog-helper

Copy link
Copy Markdown

Hi @guifranchi,

We've found some issues with your Pull Request.

  • This Pull Request is missing reviewers. Either convert this Pull Request into a draft or ignore this rule by adding [skip review] to your Pull Request body. - (More info)

New issue guide | Backlog management | Rules | Feedback

@guifranchi
guifranchi merged commit 8e67ef0 into main May 21, 2026
1 check passed
@guifranchi
guifranchi deleted the fix/vm-console-logging branch May 21, 2026 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant