[system-health]: Honor 'psu' and 'pdb' device ignore independently in hardware checker#28415
Conversation
… hardware checker The PDB-support change (sonic-net#26829) required BOTH 'psu' and 'pdb' to be in devices_to_ignore before _check_psu_status would skip. Platforms that ignore only 'psu' (e.g. DPUs with no PSU_INFO) therefore fell through to the empty-table branch and reported "Failed to get PSU information", flipping system-health Hardware to Not OK. It also silently re-enabled per-PSU checks on switches that ignore category 'psu' but not 'pdb'. Honor 'psu' and 'pdb' independently: skip the whole check only when both are ignored; report the empty PSU_INFO failure only when 'psu' is not ignored; and per row, skip PSU rows when 'psu' is ignored and PDB rows when 'pdb' is ignored. Signed-off-by: Vasundhara Volam <vvolam@microsoft.com>
|
/azp run Azure.sonic-buildimage |
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Pull request overview
Fixes a regression in system-health where the hardware checker would incorrectly report “Failed to get PSU information” (and re-enable PSU checks) when devices_to_ignore contains psu but not pdb. The change refines how PSU vs PDB entries (both sourced from STATE_DB PSU_INFO) are filtered so that psu and pdb ignore behavior works independently and matches the PR/issue expectations.
Changes:
- Update
_check_psu_statusto treatpsuandpdbcategory ignores independently, while still skipping the entire PSU/PDB pass only when both categories are ignored. - Suppress the empty-
PSU_INFOfailure whenpsuis ignored (to support platforms with no PSUs), but keep it for platforms where PSU monitoring is expected. - Add regression unit tests covering the “ignore psu but not pdb” scenarios, including the empty-table case.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/system-health/health_checker/hardware_checker.py |
Refines PSU/PDB ignore logic so empty PSU_INFO and per-row processing honor psu/pdb independently. |
src/system-health/tests/test_system_health.py |
Adds unit tests to prevent regressions for “ignore psu only” behaviors (empty PSU_INFO, and mixed PSU/PDB rows). |
|
@yuazhe please review as well. |
|
This PR has backport request for branch(es): 202605. ---Powered by SONiC BuildBot
|
|
Cherry-pick PR to 202605: #28616 |
|
@vvolam approving cherry pick for 202605. Please:
|
@vaibhavhd Patch is already verified on a testbed with 202605 image and I have updated same in PR description as well. |
Why I did it
show system-health detailreports Hardware: Not OK with reason "Failed to get PSU information" on platforms that have no PSUs and whosesystem_health_monitoring_config.jsonlistspsuindevices_to_ignorebut notpdb— even thoughpsuis shown as ignored.This is a regression from #26829 ("Add PDB support for system-health"), which changed the
_check_psu_statusearly-return guard to require bothpsuandpdbto be ignored. When onlypsuis ignored, the checker proceeds, finds an emptyPSU_INFO, and reports the failure. The same change also silently re-enabled per-PSU checks on switches that ignore thepsucategory but notpdb.fixes #28414
Work item tracking
How I did it
In
src/system-health/health_checker/hardware_checker.py,_check_psu_statusnow honorspsuandpdbindependently:psuandpdbare ignored.PSU_INFO"Failed to get PSU information" failure only whenpsuis not ignored (so a genuinely brokenpsudon a real switch is still detected).psuis ignored and PDB rows (keyedPDB N) whenpdbis ignored.Added regression unit tests in
src/system-health/tests/test_system_health.py.How to verify it
PSU_INFOin STATE_DB, setdevices_to_ignoreto includepsu(andfan) but notpdb, restart thesystem-healthservice, and confirmshow system-health detailreports Hardware: OK (previously Not OK — Failed to get PSU information).psuis not ignored, and that PDB rows are still evaluated when onlypsuis ignored.Verified with both 202605 and master image. PSU is correctly ignored on a DPU, while it is correctly checked on the NPU.
Which release branch to backport (provide reason below if selected)
Reason: the regression (#26829) is present on
masterand202605; the fix should follow it wherever#26829is merged.Tested branch (Please provide the tested image version)
master; the regression was originally observed on20260510.Description for the changelog
[system-health] Fix false "Failed to get PSU information" when only 'psu' (not 'pdb') is in devices_to_ignore
Link to config_db schema for YANG module changes
N/A — no YANG/config_db schema changes.
A picture of a cute animal (not mandatory but encouraged)
🐾🐱