Skip to content

[AI Generated] BugFix: add MANA dmesg diagnostics - #4678

Open
LiliDeng wants to merge 1 commit into
mainfrom
bugfix/mana-serial-console-check_170826_173354
Open

[AI Generated] BugFix: add MANA dmesg diagnostics#4678
LiliDeng wants to merge 1 commit into
mainfrom
bugfix/mana-serial-console-check_170826_173354

Conversation

@LiliDeng

@LiliDeng LiliDeng commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Description

Add dmesg-based MANA probe failure detection to SR-IOV VF count mismatch errors, so failures can be classified as MANA driver related when applicable.

Related Issue

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Refactoring
  • Documentation update

Checklist

  • Description is filled in above
  • No credentials, secrets, or internal details are included
  • Peer review requested (if not, add required peer reviewers after raising PR)
  • Tests executed and results posted below

Test Validation

Key Test Cases:
verify_sriov_add_max_nics

Impacted LISA Features:
NA

Tested Azure Marketplace Images:

Test Results

Image VM Size Result
microsoftcblmariner cbl-mariner cbl-mariner-2-gen2 2.20241230.02 Standard_L16s_v4 FAILED
Sriov.verify_sriov_add_max_nics: FAILED   failed. AssertionError: [Ethernet VF count inside VM (without IB) is 0, actual sriov nic count is 8. Total PCI NICs: [].  
MANA error found in dmesg: mana 7870:00:00.0: HWC: Failed hw_channel req: 0xc00000bb
 [ 6.737085] mana 7870:00:00.0: VfVerifyVersionOutput: -71, status=0xc00000bb 
 [ 6.740790] mana 7870:00:00.0: gdma probe failed] Expected <0> to be equal to <8>, but was not.

Copilot AI lite review requested due to automatic review settings August 18, 2026 14:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds MANA-related dmesg diagnostics to SR-IOV NIC initialization failures in the network test utilities, to make mismatched VF counts easier to debug.

Changes:

  • Adds MANA/GDMA probe-failure regex patterns and uses them to scan dmesg when SR-IOV PCI NIC counts don’t match expected SR-IOV NIC counts.
  • Augments the failing assertion context to optionally include a MANA-related hint.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread lisa/microsoft/testsuites/network/common.py Outdated
Comment thread lisa/microsoft/testsuites/network/common.py
Copilot AI review requested due to automatic review settings August 18, 2026 14:50
@LiliDeng
LiliDeng force-pushed the bugfix/mana-serial-console-check_170826_173354 branch from 4fcba3f to 805ed09 Compare August 18, 2026 14:51
@LiliDeng

Copy link
Copy Markdown
Collaborator Author

AI Test Case Selection

Selected 43 test case(s): verify_bbr3_available,verify_bbr3_set_and_restore,verify_bbr3_applies_to_live_tcp_flow,verify_bbr3_kernel_config_runtime_consistency,verify_inet_diag_destroy,verify_inet_diag_enabled,verify_lwtunnel_bpf_support,validate_netvsc_reload,verify_network_interface_reload_via_ip_link,verify_ringbuffer_settings_change,verify_device_channels_change,verify_device_enabled_features,verify_device_gro_lro_settings_change,verify_device_rss_hash_key_change,verify_device_rx_hash_level_change,verify_device_msg_level_change,verify_device_statistics,verify_services_state,verify_sriov_basic,verify_sriov_single_vf_connection,verify_sriov_single_vf_connection_max_cpu,verify_sriov_max_vf_connection,verify_sriov_max_vf_connection_max_cpu,verify_sriov_disable_enable,verify_sriov_disable_enable_pci,verify_sriov_disable_enable_on_guest,verify_sriov_add_max_nics,verify_sriov_provision_with_max_nics,verify_sriov_provision_with_max_nics_reboot,verify_sriov_provision_with_max_nics_reboot_from_platform,verify_sriov_provision_with_max_nics_stop_start_from_platform,verify_sriov_reload_modules,verify_sriov_ethtool_offload_setting,verify_irqbalance,verify_sriov_interrupts_change,verify_synthetic_provision_with_max_nics,verify_synthetic_provision_with_max_nics_reboot,verify_synthetic_provision_with_max_nics_reboot_from_platform,verify_synthetic_provision_with_max_nics_stop_start_from_platform,verify_synthetic_add_max_nics_one_time_after_provision,verify_synthetic_add_max_nics_one_by_one_after_provision,verify_xfrm_interface,verify_xfrm_interface_load_unload

Marketplace image: canonical 0001-com-ubuntu-server-jammy 22_04-lts-gen2 latest

Result: Canceled

View full logs in Azure DevOps

@LiliDeng
LiliDeng force-pushed the bugfix/mana-serial-console-check_170826_173354 branch from 805ed09 to 2f18865 Compare August 18, 2026 14:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (1)

lisa/microsoft/testsuites/network/common.py:78

  • Fetching and scanning the full dmesg output on every transient VF count mismatch can add noticeable overhead because initialize_nic_info() is retried up to 30 times. Consider limiting dmesg to the last N lines and simplifying the match check to avoid repeated expensive work.
                dmesg_log = node.tools[Dmesg].get_output(force_run=True)
                mana_errors = find_patterns_in_lines(dmesg_log, _MANA_ERROR_PATTERNS)
                mana_error = next(
                    (match for matches in mana_errors for match in matches), ""
                )

Copilot AI review requested due to automatic review settings August 18, 2026 14:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

Suppressed comments (2)

lisa/microsoft/testsuites/network/common.py:26

  • The inline example says the dmesg line includes an error code (e.g. "error -71"), but the regex currently stops at the word "error" and won’t include the code in the reported match. Expanding the pattern to optionally capture the numeric error code will make the diagnostic more accurate (or alternatively update the comment to match the current behavior).
    # Sample: mana: probe of 7870:00:00.0 failed with error -71
    re.compile(
        r"\bmana:\s+probe of \S+ failed with error\b",
        re.IGNORECASE,
    ),

lisa/microsoft/testsuites/network/common.py:75

  • initialize_nic_info is retried up to 30 times; with force_run=True this will re-run dmesg on every retry when counts mismatch, adding avoidable overhead in a failure/retry loop. Consider using the default cached execution (omit force_run=True) since a single dmesg snapshot is typically enough for diagnostics.
            if pci_nic_count != sriov_count:
                dmesg_log = node.tools[Dmesg].get_output(force_run=True)
                mana_errors = find_patterns_in_lines(dmesg_log, _MANA_ERROR_PATTERNS)

@LiliDeng

Copy link
Copy Markdown
Collaborator Author

AI Test Case Selection

Selected 43 test case(s): verify_bbr3_available,verify_bbr3_set_and_restore,verify_bbr3_applies_to_live_tcp_flow,verify_bbr3_kernel_config_runtime_consistency,verify_inet_diag_destroy,verify_inet_diag_enabled,verify_lwtunnel_bpf_support,validate_netvsc_reload,verify_network_interface_reload_via_ip_link,verify_ringbuffer_settings_change,verify_device_channels_change,verify_device_enabled_features,verify_device_gro_lro_settings_change,verify_device_rss_hash_key_change,verify_device_rx_hash_level_change,verify_device_msg_level_change,verify_device_statistics,verify_services_state,verify_sriov_basic,verify_sriov_single_vf_connection,verify_sriov_single_vf_connection_max_cpu,verify_sriov_max_vf_connection,verify_sriov_max_vf_connection_max_cpu,verify_sriov_disable_enable,verify_sriov_disable_enable_pci,verify_sriov_disable_enable_on_guest,verify_sriov_add_max_nics,verify_sriov_provision_with_max_nics,verify_sriov_provision_with_max_nics_reboot,verify_sriov_provision_with_max_nics_reboot_from_platform,verify_sriov_provision_with_max_nics_stop_start_from_platform,verify_sriov_reload_modules,verify_sriov_ethtool_offload_setting,verify_irqbalance,verify_sriov_interrupts_change,verify_synthetic_provision_with_max_nics,verify_synthetic_provision_with_max_nics_reboot,verify_synthetic_provision_with_max_nics_reboot_from_platform,verify_synthetic_provision_with_max_nics_stop_start_from_platform,verify_synthetic_add_max_nics_one_time_after_provision,verify_synthetic_add_max_nics_one_by_one_after_provision,verify_xfrm_interface,verify_xfrm_interface_load_unload

Marketplace image: canonical 0001-com-ubuntu-server-jammy 22_04-lts-gen2 latest

Result: Succeeded

View full logs in Azure DevOps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants