Fix/hibernation transient retry - #4666
Conversation
Hibernation tests share startstop.stop(Hibernate); Azure intermittently returns a transient VMHibernateFailed internal error that it documents as retryable. Retry that exact error up to 3 times, 30s apart, only while the VM is still running. Other failures and non-running states raise immediately as before.
On Debian the hibernation-setup tool writes resume=/resume_offset= to the bootloader, which only take effect after a reboot. Without it the guest accepts the hibernate uevent but never suspends, so Azure returns VMHibernateFailed on every attempt (seen on Debian 13). Extend the existing post-setup reboot workaround (already used for Redhat/AlmaLinux/SLES) to Debian. Exact type() match keeps Ubuntu unaffected.
Ubuntu 24.04 shows the same persistent VMHibernateFailed as Debian: the guest accepts the hibernate uevent but never suspends because the resume=/resume_offset= bootloader entries need a reboot to take effect. Ubuntu is a Debian subclass but the exact type() check did not cover it, so add Ubuntu explicitly. The list now covers all distros is_distro_supported allows.
There was a problem hiding this comment.
Pull request overview
This PR updates the Azure hibernation test common helpers to (1) treat FIPS-enabled kernels as an unmet precondition (skip) and (2) reboot after hibernation setup on additional distros (Debian/Ubuntu) to avoid a known “accepts uevent but never suspends” failure mode.
Changes:
- Add FIPS-kernel detection and skip logic for Azure hibernation scenarios that always fail with
VMHibernateFailed. - Expand the post-setup reboot guard to include Debian and Ubuntu.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
Suppressed comments (2)
lisa/microsoft/testsuites/power/common.py:36
- Please add a description explaining what this PR does and why (the PR description still contains the unfilled template placeholders).
# Some Azure guest/kernel combinations, including certified FIPS kernels,
# do not support hibernation at all. In those cases the platform returns
# VMHibernateFailed on every attempt, so we treat it as an unmet precondition
# rather than a transient failure that can be recovered with retry delay.
_HIBERNATE_FAILED_MARKER = "VMHibernateFailed"
lisa/microsoft/testsuites/power/common.py:172
type(node.os) in (...)is unnecessarily strict and inconsistent with other checks in this module (e.g.isinstance(node.os, Redhat)above). Usingisinstanceis more robust and keeps behavior the same for the current OS classes.
if type(node.os) in (Redhat, AlmaLinux, SLES, Debian, Ubuntu):
AI Test Case SelectionSelected 10 test case(s): verify_hibernation_synthetic_network,verify_hibernation_sriov_network,verify_hibernation_time_sync,verify_hibernation_with_network_workload,verify_hibernation_with_storage_workload,verify_hibernation_with_memory_workload,verify_hibernation_synthetic_network_max_nics,verify_hibernation_sriov_network_max_nics,verify_hibernation_max_data_disks,verify_hibernation_with_vm_extension Marketplace image: Result: Failed |
|
Sandygit01 please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
Suppressed comments (2)
lisa/microsoft/testsuites/power/common.py:55
- Update the call site to use the renamed helper so the skip check continues to work.
if _is_fips_kernel(node):
lisa/microsoft/testsuites/power/common.py:43
- The helper name
_is_fips_kernelis misleading because it returns true when either the kernel or the runtime FIPS mode is enabled (/proc/sys/crypto/fips_enabled == 1). Renaming it to reflect both conditions will make the precondition clearer.
This issue also appears on line 55 of the same file.
def _is_fips_kernel(node: Node) -> bool:
# Azure hibernation is unsupported on the certified FIPS kernel (e.g. Ubuntu
# Pro *-azure-fips): it returns VMHibernateFailed on every attempt, so this
# is an unmet precondition rather than a test failure.
fips_enabled = "0"
|
Hibernation on x86 systems failed in FIPS mode (fips=1) because it used MD5 for E820 memory-map integrity checks, which is prohibited in FIPS. The Linux kernel fixed this upstream by replacing MD5 with CRC32, allowing hibernation to function correctly under FIPS restrictions. |
Description
Skip hibernation tests on FIPS-enabled kernels (Azure returns
VMHibernateFailedon every attempt, so it is an unmet precondition rather than a failure), and treat a missing/proc/sys/crypto/fips_enabledas FIPS-disabled instead of erroring. Also reboot after hibernation setup so the bootloaderresume=/resume_offset=parameters take effect before hibernating.Related Issue
Type of Change
Checklist
Test Validation
Key Test Cases:
verify_hibernation
Impacted LISA Features:
StartStop
Tested Azure Marketplace Images:
Test Results