Skip to content

fix(e2e): enable persistent journal to capture post-reboot logs #1465

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
qinqon wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix(e2e): enable persistent journal to capture post-reboot logs #1465

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions cluster/up.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,7 @@ for node in $(./cluster/kubectl.sh get nodes --no-headers | awk '{print $1}'); d
# keep using the NM internal DHCP client as we always have
./cluster/cli.sh ssh ${node} -- sudo rm -f /etc/NetworkManager/conf.d/002-dhclient.conf
./cluster/cli.sh ssh ${node} -- sudo systemctl restart NetworkManager
# Enable persistent journal so logs survive node reboots
./cluster/cli.sh ssh ${node} -- sudo mkdir -p /var/log/journal
./cluster/cli.sh ssh ${node} -- sudo systemctl restart systemd-journald
Copy link

@gemini-code-assist gemini-code-assist bot Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To improve efficiency and avoid unnecessary service interruptions, consider restarting systemd-journald only if it's not already configured for persistent storage. While mkdir -p /var/log/journal ensures the directory exists, systemd-journald might already be running with persistent storage enabled from a previous run. An unconditional restart on every node in the loop could lead to minor performance overhead and temporary logging unavailability. A conditional restart would be more robust.

Suggested change
./cluster/cli.sh ssh ${node} -- sudo systemctl restart systemd-journald
./cluster/cli.sh ssh ${node} -- sudo bash -c 'if ! journalctl --disk-usage | grep -q "Persistent: yes"; then systemctl restart systemd-journald; fi'

Copy link
Member Author

@qinqon qinqon Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This runs once during cluster-up on freshly provisioned VMs, so there's no "previous run" scenario — the nodes are always new. The restart takes milliseconds and the script already restarts heavier services (NetworkManager, openvswitch) unconditionally in the same loop. Adding a conditional check would be more complex for no practical benefit here.

done