Skip to content

fix: reboot loop and improve devlink parameter reconciliation#134

Merged
e0ne merged 1 commit intoMellanox:network-operator-26.1.xfrom
rollandf:loop
Jan 19, 2026
Merged

fix: reboot loop and improve devlink parameter reconciliation#134
e0ne merged 1 commit intoMellanox:network-operator-26.1.xfrom
rollandf:loop

Conversation

@rollandf
Copy link
Member

  • Avoid unnecessary reboots by checking Mellanox firmware multiport state.
  • Ensure devlink parameter changes trigger interface reconciliation.

- Avoid unnecessary reboots by checking Mellanox firmware multiport state.
- Ensure devlink parameter changes trigger interface reconciliation.

Signed-off-by: Fred Rolland <frolland@nvidia.com>
@github-actions
Copy link

Thanks for your PR,
To run vendors CIs, Maintainers can use one of:

  • /test-all: To run all tests for all vendors.
  • /test-e2e-all: To run all E2E tests for all vendors.
  • /test-e2e-nvidia-all: To run all E2E tests for NVIDIA vendor.

To skip the vendors CIs, Maintainers can use one of:

  • /skip-all: To skip all tests for all vendors.
  • /skip-e2e-all: To skip all E2E tests for all vendors.
  • /skip-e2e-nvidia-all: To skip all E2E tests for NVIDIA vendor.
    Best regards.

@github-actions github-actions bot added the tests label Jan 18, 2026
@greptile-apps
Copy link

greptile-apps bot commented Jan 18, 2026

Greptile Summary

This PR fixes the Mellanox firmware reboot loop issue and ensures devlink parameter changes trigger proper interface reconciliation.

Key Changes

  • Added devlink parameter reconciliation: New NeedToUpdateDevlinkParams function in api/v1/helper.go:207-228 checks if devlink parameters have changed and properly triggers interface updates via the NeedToUpdateSriov function at api/v1/helper.go:309-311.

  • Prevented unnecessary reboots: Enhanced HandleESwitchParams in pkg/vendors/mellanox/mellanox.go:401-445 to check current firmware state (fwCurrent.Multiport) before triggering a reboot. If the firmware already has the desired multiport configuration or doesn't support LagResourceAllocation, the reboot is skipped.

  • Improved firmware state reading: Added LagResourceAllocation to the list of firmware attributes queried in GetMlxNicFwData at pkg/vendors/mellanox/mellanox.go:246 and properly parsed it in mlnxNicFromMap at pkg/vendors/mellanox/mellanox.go:464-470, initializing Multiport to -1 to indicate unsupported firmware.

  • Comprehensive test coverage: Added test cases verifying the new behavior for unsupported firmware and when firmware already matches the desired state.

Confidence Score: 4/5

  • This PR is safe to merge with minimal risk
  • The changes are well-structured with proper firmware state checking and comprehensive test coverage. The logic correctly prevents unnecessary reboots by checking current firmware state before triggering changes. The devlink parameter reconciliation properly integrates into the existing update flow.
  • Pay close attention to pkg/vendors/mellanox/mellanox.go to ensure the multiport state checking logic handles all edge cases correctly

Important Files Changed

Filename Overview
api/v1/helper.go Added NeedToUpdateDevlinkParams function to check devlink parameter changes and integrated it into NeedToUpdateSriov to trigger interface reconciliation when devlink params change
pkg/vendors/mellanox/mellanox.go Enhanced firmware handling to check current multiport state before triggering reboot, added LagResourceAllocation reading, and improved HandleESwitchParams logic to avoid unnecessary reboots

Sequence Diagram

sequenceDiagram
    participant Plugin as MellanoxPlugin
    participant Helper as api/v1/helper
    participant Vendor as mellanox/mlxutils
    participant FW as Firmware (mstconfig)

    Note over Plugin: OnNodeStateChange()
    
    Plugin->>Vendor: GetMlxNicFwData(pciAddress)
    Vendor->>FW: MstConfigReadData()
    FW-->>Vendor: Current & Next FW data (including LagResourceAllocation)
    Vendor->>Vendor: mlnxNicFromMap()
    Note over Vendor: Parse LagResourceAllocation<br/>Set Multiport: -1 if unsupported<br/>Set Multiport: 0 or 1 if supported
    Vendor-->>Plugin: fwCurrent, fwNext
    
    Plugin->>Vendor: HandleESwitchParams(pciPrefix, attrs, fwCurrent, specs, status)
    Vendor->>Vendor: isESwitchParamsRequireChange(spec, status)
    Note over Vendor: Check if esw_multiport<br/>param requested
    
    alt Multiport change needed
        alt fwCurrent.Multiport == -1
            Note over Vendor: LagResourceAllocation not supported<br/>Skip firmware change
            Vendor-->>Plugin: needReboot = false
        else fwCurrent.Multiport == desiredMultiport
            Note over Vendor: Already set in firmware<br/>Skip reboot
            Vendor-->>Plugin: needReboot = false
        else Multiport needs change
            Note over Vendor: Set attrs.Multiport to desired value
            Vendor-->>Plugin: needReboot = true
        end
    else No change needed
        Vendor-->>Plugin: needReboot = false
    end
    
    Plugin->>Helper: NeedToUpdateSriov(ifaceSpec, ifaceStatus)
    Helper->>Helper: NeedToUpdateDevlinkParams(desired, current)
    Note over Helper: Compare each devlink param<br/>Check name and value match
    Helper-->>Plugin: true if devlink params differ
    
    alt needReboot == true
        Plugin->>Vendor: MlxConfigFW(attributesToChange)
        Vendor->>FW: mstconfig set LAG_RESOURCE_ALLOCATION=X
        Plugin->>Vendor: MlxResetFW(pciAddresses)
        Vendor->>FW: mstfwreset --skip_driver
        Note over Plugin: Node will reboot
    end
Loading

@rollandf
Copy link
Member Author

Policy used:

apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodePolicy
metadata:
  name: rail-0
  namespace: network-operator
spec:
  bridge:
    groupingPolicy: all
    ovs:
      bridge:
        datapathType: netdev
        failMode: secure
      uplink:
        interface:
          mtuRequest: 1500
          type: dpdk
  deviceType: netdevice
  devlinkParams:
    params:
    - applyOn: PF
      cmode: runtime
      name: esw_multiport
      value: "true"
  eSwitchMode: switchdev
  isRdma: true
  linkType: ETH
  mtu: 1500
  nicSelector:
    pfNames:
    - eth_p0_r0
  nodeSelector:
    node-role.kubernetes.io/worker: ""
  numVfs: 1
  priority: 99
  resourceName: rail_0

@rollandf
Copy link
Member Author

Failing log:
Note that the current LAG_RESOURCE_ALLOCATION is already on PRE_ALLOCATION, but we still set it and reboot.

*       LAG_RESOURCE_ALLOCATION                     DEVICE_DEFAULT(0)    PRE_ALLOCATION(1)    PRE_ALLOCATION(1)   \n
2026-01-18T11:06:26.724796875Z  INFO    mellanox/mellanox.go:107        MstConfigReadData()     {"device": "0000:c9:00.0"}
2026-01-18T11:06:26.724817168Z  INFO    mellanox/mellanox.go:101        RunCommand()    {"command": "mstconfig", "args": ["-e", "-d", "0000:c9:00.0", "q"]}






2026-01-18T11:06:59.912756397Z  LEVEL(-2)       mellanox/mellanox.go:101        RunCommand()    {"output": "\nDevice #1:\n----------\n\nDevice type:        ConnectX8           \nName:               900-9X81E-00EX-ST0_Ax\nDescription:        NVIDIA ConnectX-8 C8180L HHHL SuperNIC; 800Gbs XDR IB (default mode) / 2x400GbE; Single-cage OSFP; PCIe 6 x16 with x16 PCIe Socket Direct Extension option; Crypto Enabled; Secure Boot Enabled\nDevice:             0000:c9:00.0        \n\nConfigurations:                                          Default             Current         Next Boot\n        ACCURATE_TX_SCHEDULER                       False(0)             False(0)             False(0)            \n        ADVANCED_PCI_SETTINGS                       False(0)             False(0)             False(0)            \n RO     ADVANCED_TESTABILITY                        False(0)             False(0)             False(0)            \n        AES_XTS_TWEAK_INC_64                        False(0)             False(0)             False(0)            \n        ALLOW_RD_COUNTERS                           True(1)              True(1)              True(1)             \n        ATS_ENABLED                                 False(0)             False(0)             False(0)            \n        AUTO_POWER_SAVE_LINK_DOWN_P1                False(0)             False(0)             False(0)            \n        AUTO_POWER_SAVE_LINK_DOWN_P2                False(0)             False(0)             False(0)            \n        BAR_PAGE_ALIGNMENT                          DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        BOOT_DBG_LOG                                False(0)             False(0)             False(0)            \n        BOOT_INTERRUPT_DIS                          False(0)             False(0)             False(0)            \n        BOOT_LACP_DIS                               True(1)              True(1)              True(1)             \n        BOOT_PKEY                                   0                    0                    0                   \n        BOOT_UNDI_NETWORK_WAIT                      0                    0                    0                   \n        BOOT_VLAN_EN                                False(0)             False(0)             False(0)            \n        BOOT_VLAN                                   1                    1                    1                   \n        CLAMP_TGT_RATE_AFTER_TIME_INC_P1            True(1)              True(1)              True(1)             \n        CLAMP_TGT_RATE_AFTER_TIME_INC_P2            True(1)              True(1)              True(1)             \n        CLAMP_TGT_RATE_P1                           False(0)             False(0)             False(0)            \n        CLAMP_TGT_RATE_P2                           False(0)             False(0)             False(0)            \n        CNP_802P_PRIO_P1                            6                    6                    6                   \n        CNP_802P_PRIO_P2                            6                    6                    6                   \n        CNP_DSCP_P1                                 48                   48                   48                  \n        CNP_DSCP_P2                                 48                   48                   48                  \n        CQE_COMPRESSION                             BALANCED(0)          BALANCED(0)          BALANCED(0)         \n        CRYPTO_POLICY                               UNRESTRICTED(1)      UNRESTRICTED(1)      UNRESTRICTED(1)     \n RO     CUSTOMIZATION_NUMBER                        Array[0..15]         Array[0..15]         Array[0..15]        \n        DBR_LESS_SQP                                ENABLE(1)            ENABLE(1)            ENABLE(1)           \n        DCBX_CEE_P1                                 True(1)              True(1)              True(1)             \n        DCBX_CEE_P2                                 True(1)              True(1)              True(1)             \n        DCBX_IEEE_P1                                True(1)              True(1)              True(1)             \n        DCBX_IEEE_P2                                True(1)              True(1)              True(1)             \n        DCBX_WILLING_P1                             True(1)              True(1)              True(1)             \n        DCBX_WILLING_P2                             True(1)              True(1)              True(1)             \n        DCE_TCP_G_P1                                1019                 1019                 1019                \n        DCE_TCP_G_P2                                1019                 1019                 1019                \n        DCE_TCP_RTT_P1                              1                    1                    1                   \n        DCE_TCP_RTT_P2                              1                    1                    1                   \n        DCR_LIFO_SIZE                               16384                16384                16384               \n        DO_NOT_CLEAR_PORT_STATS_P1                  False(0)             False(0)             False(0)            \n        DO_NOT_CLEAR_PORT_STATS_P2                  False(0)             False(0)             False(0)            \n RO     DPA_APP_0_UUID                              Array[0..15]         Array[0..15]         Array[0..15]        \n RO     DPA_APP_1_UUID                              Array[0..15]         Array[0..15]         Array[0..15]        \n RO     DPA_APP_2_UUID                              Array[0..15]         Array[0..15]         Array[0..15]        \n RO     DPA_APP_3_UUID                              Array[0..15]         Array[0..15]         Array[0..15]        \n RO     DPA_APP_4_UUID                              Array[0..15]         Array[0..15]         Array[0..15]        \n RO     DPA_APP_5_UUID                              Array[0..15]         Array[0..15]         Array[0..15]        \n RO     DPA_APP_6_UUID                              Array[0..15]         Array[0..15]         Array[0..15]        \n RO     DPA_APP_7_UUID                              Array[0..15]         Array[0..15]         Array[0..15]        \n        DPA_AUTHENTICATION                          False(0)             False(0)             False(0)            \n        DUP_MAC_ACTION_P1                           LAST_CFG(0)          LAST_CFG(0)          LAST_CFG(0)         \n        DUP_MAC_ACTION_P2                           LAST_CFG(0)          LAST_CFG(0)          LAST_CFG(0)         \n        DYNAMIC_VF_MSIX_TABLE                       False(0)             False(0)             False(0)            \n        ESWITCH_HAIRPIN_DESCRIPTORS                 Array[0..7]          Array[0..7]          Array[0..7]         \n        ESWITCH_HAIRPIN_TOT_BUFFER_SIZE             Array[0..7]          Array[0..7]          Array[0..7]         \n        ETH_ETS_GRANULARITY_P1                      DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        ETH_ETS_GRANULARITY_P2                      DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        ETS_SCHED_MODE_P1                           device_default(0)    device_default(0)    device_default(0)   \n        ETS_SCHED_MODE_P2                           device_default(0)    device_default(0)    device_default(0)   \n        EXP_ROM_PXE_ENABLE                          True(1)              True(1)              True(1)             \n        EXP_ROM_UEFI_ARM_ENABLE                     True(1)              True(1)              True(1)             \n        EXP_ROM_UEFI_x86_ENABLE                     True(1)              True(1)              True(1)             \n        FLEX_IPV4_OVER_VXLAN_PORT                   0                    0                    0                   \n        FLEX_PARSER_PROFILE_ENABLE                  0                    0                    0                   \n        FLOW_CONTROL_P1                             DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        FLOW_CONTROL_P2                             DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        HAIRPIN_DATA_BUFFER_LOCK                    False(0)             False(0)             False(0)            \n        HOST_CHAINING_CACHE_DISABLE                 False(0)             False(0)             False(0)            \n        HOST_CHAINING_DESCRIPTORS                   Array[0..7]          Array[0..7]          Array[0..7]         \n        HOST_CHAINING_MODE                          DISABLED(0)          DISABLED(0)          DISABLED(0)         \n        HOST_CHAINING_TOTAL_BUFFER_SIZE             Array[0..7]          Array[0..7]          Array[0..7]         \n        IB_CC_SHAPER_COALESCE_P1                    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        IB_CC_SHAPER_COALESCE_P2                    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        IB_ETS_GRANULARITY_P1                       DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        IB_ETS_GRANULARITY_P2                       DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        IB_PROTO_WIDTH_EN_MASK_P1                   0                    0                    0                   \n        IB_PROTO_WIDTH_EN_MASK_P2                   0                    0                    0                   \n        IB_ROUTING_MODE_P1                          LID(1)               LID(1)               LID(1)              \n        IB_ROUTING_MODE_P2                          LID(1)               LID(1)               LID(1)              \n        ICM_CACHE_MODE                              DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        INITIAL_ALPHA_VALUE_P1                      1023                 1023                 1023                \n        INITIAL_ALPHA_VALUE_P2                      1023                 1023                 1023                \n        INT_LOG_MAX_PAYLOAD_SIZE                    AUTOMATIC(0)         AUTOMATIC(0)         AUTOMATIC(0)        \n        IP_OVER_VXLAN_EN                            False(0)             False(0)             False(0)            \n        IP_VER                                      IPv4(0)              IPv4(0)              IPv4(0)             \n        KEEP_ETH_LINK_UP_P1                         True(1)              True(1)              True(1)             \n        KEEP_ETH_LINK_UP_P2                         True(1)              True(1)              True(1)             \n        KEEP_IB_LINK_UP_P1                          False(0)             False(0)             False(0)            \n        KEEP_IB_LINK_UP_P2                          False(0)             False(0)             False(0)            \n        KEEP_LINK_UP_ON_BOOT_P1                     False(0)             False(0)             False(0)            \n        KEEP_LINK_UP_ON_BOOT_P2                     False(0)             False(0)             False(0)            \n        KEEP_LINK_UP_ON_STANDBY_P1                  False(0)             False(0)             False(0)            \n        KEEP_LINK_UP_ON_STANDBY_P2                  False(0)             False(0)             False(0)            \n*       LAG_RESOURCE_ALLOCATION                     DEVICE_DEFAULT(0)    PRE_ALLOCATION(1)    PRE_ALLOCATION(1)   \n        LARGE_MTU_TWEAK_64                          False(0)             False(0)             False(0)            \n        LEGACY_BOOT_PROTOCOL                        PXE(1)               PXE(1)               PXE(1)              \n*       LINK_TYPE_P1                                IB(1)                ETH(2)               ETH(2)              \n        LINK_TYPE_P2                                ETH(2)               ETH(2)               ETH(2)              \n        LLDP_NB_DCBX_P1                             False(0)             False(0)             False(0)            \n        LLDP_NB_DCBX_P2                             False(0)             False(0)             False(0)            \n        LLDP_NB_RX_MODE_P1                          OFF(0)               OFF(0)               OFF(0)              \n        LLDP_NB_RX_MODE_P2                          OFF(0)               OFF(0)               OFF(0)              \n        LLDP_NB_TX_MODE_P1                          OFF(0)               OFF(0)               OFF(0)              \n        LLDP_NB_TX_MODE_P2                          OFF(0)               OFF(0)               OFF(0)              \n*       LOAD_BALANCE_MODE_P1                        TRANSPORT(2)         DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n*       LOAD_BALANCE_MODE_P2                        TRANSPORT(2)         DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        LOG_DCR_HASH_TABLE_SIZE                     11                   11                   11                  \n        LOG_MAX_OUTSTANDING_READ_ATOMIC             0                    0                    0                   \n        LOG_MAX_OUTSTANDING_WQE                     7                    7                    7                   \n        LOG_MAX_QUEUE                               17                   17                   17                  \n        LOG_TX_PSN_WINDOW                           9                    9                    9                   \n        LRO_LOG_TIMEOUT0                            6                    6                    6                   \n        LRO_LOG_TIMEOUT1                            7                    7                    7                   \n        LRO_LOG_TIMEOUT2                            8                    8                    8                   \n        LRO_LOG_TIMEOUT3                            13                   13                   13                  \n        MAX_PACKET_LIFETIME                         0                    0                    0                   \n        MCTP_I3C_DISABLE                            False(0)             False(0)             False(0)            \n        MEMIC_ATOMIC_ENDIANESS                      DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        MEMIC_ATOMIC                                MEMIC_ATOMIC_ENABLE(2) MEMIC_ATOMIC_ENABLE(2) MEMIC_ATOMIC_ENABLE(2)\n        MEMIC_BAR_SIZE                              0                    0                    0                   \n        MEMIC_SIZE_LIMIT                            _256KB(1)            _256KB(1)            _256KB(1)           \n        MIN_TIME_BETWEEN_CNPS_P1                    4                    4                    4                   \n        MIN_TIME_BETWEEN_CNPS_P2                    4                    4                    4                   \n        MKEY_BY_NAME                                False(0)             False(0)             False(0)            \n        MKEY_BY_NAME_RANGE                          DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        MODULE_SPLIT_M0                             Array[0..15]         Array[0..15]         Array[0..15]        \n        MPFS_MC_LOOPBACK_DISABLE_P1                 False(0)             False(0)             False(0)            \n        MPFS_MC_LOOPBACK_DISABLE_P2                 False(0)             False(0)             False(0)            \n        MPFS_UC_LOOPBACK_DISABLE_P1                 False(0)             False(0)             False(0)            \n        MPFS_UC_LOOPBACK_DISABLE_P2                 False(0)             False(0)             False(0)            \n        MULTIPATH_DSCP                              DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        MULTI_PCI_RESOURCE_SHARING                  DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        MULTI_PORT_VHCA_EN                          False(0)             False(0)             False(0)            \n        NON_PREFETCHABLE_PF_BAR                     False(0)             False(0)             False(0)            \n        NUM_OF_PFC_P1                               8                    8                    8                   \n        NUM_OF_PFC_P2                               8                    8                    8                   \n*       NUM_OF_PF                                   1                    2                    2                   \n*       NUM_OF_PLANES_P1                            4                    0                    0                   \n        NUM_OF_PLANES_P2                            0                    0                    0                   \n        NUM_OF_TC_P1                                _8_TCs(0)            _8_TCs(0)            _8_TCs(0)           \n        NUM_OF_TC_P2                                _8_TCs(0)            _8_TCs(0)            _8_TCs(0)           \n*       NUM_OF_VFS                                  16                   1                    1                   \n        NUM_OF_VL_P1                                _4_VLs(3)            _4_VLs(3)            _4_VLs(3)           \n        NUM_OF_VL_P2                                _4_VLs(3)            _4_VLs(3)            _4_VLs(3)           \n        NUM_PF_MSIX                                 63                   63                   63                  \n        NUM_PF_MSIX_VALID                           True(1)              True(1)              True(1)             \n        NUM_VF_MSIX                                 11                   11                   11                  \n        NVME_EMU_MNG_CLASS_CODE                     67586                67586                67586               \n        NVME_EMU_MNG_DEVICE_ID                      24577                24577                24577               \n        NVME_EMU_MNG_ENABLE                         False(0)             False(0)             False(0)            \n        NVME_EMU_MNG_MAX_QUEUE_DEPTH                0                    0                    0                   \n        NVME_EMU_MNG_NUM_MSIX                       0                    0                    0                   \n        NVME_EMU_MNG_NUM_PF                         1                    1                    1                   \n        NVME_EMU_MNG_NUM_VF_MSIX                    0                    0                    0                   \n        NVME_EMU_MNG_NUM_VF                         0                    0                    0                   \n        NVME_EMU_MNG_REVISION_ID                    0                    0                    0                   \n        NVME_EMU_MNG_SUBS_ID                        0                    0                    0                   \n        NVME_EMU_MNG_SUBS_VENDOR_ID                 0                    0                    0                   \n        NVME_EMU_MNG_VENDOR_ID                      5555                 5555                 5555                \n        OFF_BOARD_SERIALIZER                        False(0)             False(0)             False(0)            \n        P2P_ORDERING_MODE                           DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        PARTIAL_RESET_EN                            False(0)             False(0)             False(0)            \n        PCC_HANDLE_CORE_UTIL                        DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        PCC_INT_EN                                  False(0)             False(0)             False(0)            \n        PCC_INT_NP_RTT_DATA_MODE                    RTT_V0(64)           RTT_V0(64)           RTT_V0(64)          \n        PCC_INT_NP_RTT_DSCP_EN                      False(0)             False(0)             False(0)            \n        PCC_INT_NP_RTT_DSCP                         26                   26                   26                  \n        PCC_INT_SYSTEM_RTT                          0                    0                    0                   \n        PCC_NP_HANDLE_CORE_UTIL                     DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        PCIE_CONGESTION_MONITOR                     DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        PCIE_CREDIT_TOKEN_TIMEOUT                   0                    0                    0                   \n        PCIE_IN_BAND_VDM_DISABLE                    False(0)             False(0)             False(0)            \n        PCIE_SMBUS_DISABLE                          False(0)             False(0)             False(0)            \n        PCI_ATOMIC_MODE                             PCI_ATOMIC_DISABLED_EXT_ATOMIC_ENABLED(0) PCI_ATOMIC_DISABLED_EXT_ATOMIC_ENABLED(0) PCI_ATOMIC_DISABLED_EXT_ATOMIC_ENABLED(0)\n        PCI_BUS00_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS00_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n*       PCI_BUS00_SPEED                             PCI_GEN_6(5)         PCI_GEN_1(0)         PCI_GEN_6(5)        \n        PCI_BUS00_SWITCH_INDEX                      0                    0                    0                   \n*       PCI_BUS00_WIDTH                             PCI_X16(5)           PCI_INACTIVE(0)      PCI_X16(5)          \n        PCI_BUS01_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS01_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS01_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS01_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS01_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS02_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS02_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS02_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS02_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS02_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS03_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS03_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS03_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS03_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS03_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS04_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS04_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS04_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS04_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS04_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS05_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS05_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS05_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS05_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS05_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS06_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS06_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS06_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS06_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS06_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS07_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS07_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS07_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS07_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS07_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS10_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS10_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS10_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS10_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS10_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS11_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS11_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS11_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS11_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS11_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS12_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS12_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS12_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS12_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS12_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS13_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS13_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS13_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS13_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS13_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS14_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS14_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS14_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS14_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS14_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS15_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS15_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS15_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS15_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS15_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS16_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS16_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS16_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS16_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS16_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS17_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS17_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS17_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS17_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS17_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS20_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS20_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS20_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS20_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS20_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS21_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS21_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS21_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS21_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS21_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS22_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS22_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS22_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS22_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS22_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS23_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS23_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS23_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS23_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS23_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS24_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS24_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS24_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS24_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS24_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS25_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS25_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS25_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS25_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS25_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS26_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS26_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS26_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS26_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS26_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_BUS27_ASPM                              False(0)             False(0)             False(0)            \n        PCI_BUS27_HIERARCHY_TYPE                    PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)     PCIE_ENDPOINT(0)    \n        PCI_BUS27_SPEED                             PCI_GEN_1(0)         PCI_GEN_1(0)         PCI_GEN_1(0)        \n        PCI_BUS27_SWITCH_INDEX                      0                    0                    0                   \n        PCI_BUS27_WIDTH                             PCI_INACTIVE(0)      PCI_INACTIVE(0)      PCI_INACTIVE(0)     \n        PCI_DOWNSTREAM_PORT_OWNER                   Array[0..15]         Array[0..15]         Array[0..15]        \n        PCI_SWITCH0_UPSTREAM_PORT_BUS               0                    0                    0                   \n        PCI_SWITCH0_UPSTREAM_PORT_PEX               0                    0                    0                   \n        PCI_SWITCH1_UPSTREAM_PORT_BUS               0                    0                    0                   \n        PCI_SWITCH1_UPSTREAM_PORT_PEX               0                    0                    0                   \n        PCI_SWITCH2_UPSTREAM_PORT_BUS               0                    0                    0                   \n        PCI_SWITCH2_UPSTREAM_PORT_PEX               0                    0                    0                   \n        PCI_SWITCH_EMULATION_ENABLE                 False(0)             False(0)             False(0)            \n        PCI_SWITCH_EMULATION_NUM_PORT               16                   16                   16                  \n        PCI_SWITCH_EMU_MNG_ENABLE                   False(0)             False(0)             False(0)            \n        PCI_SWITCH_EMU_MNG_NUM_PORT                 16                   16                   16                  \n        PCI_WR_ORDERING                             per_mkey(0)          per_mkey(0)          per_mkey(0)         \n        PCORE0_REVERSAL                             False(0)             False(0)             False(0)            \n        PCORE1_REVERSAL                             False(0)             False(0)             False(0)            \n        PCORE2_REVERSAL                             False(0)             False(0)             False(0)            \n        PER_PF_NUM_SF                               False(0)             False(0)             False(0)            \n        PF_BAR2_ENABLE                              False(0)             False(0)             False(0)            \n        PF_BAR2_SIZE                                0                    0                    0                   \n        PF_DEVICE_ID_ENABLE                         False(0)             False(0)             False(0)            \n        PF_DEVICE_ID                                4131                 4131                 4131                \n        PF_LOG_BAR_SIZE                             5                    5                    5                   \n        PF_NUM_OF_VF_VALID                          False(0)             False(0)             False(0)            \n        PF_NUM_PF_MSIX                              63                   63                   63                  \n        PF_NUM_PF_MSIX_VALID                        False(0)             False(0)             False(0)            \n        PF_SD_GROUP                                 0                    0                    0                   \n        PF_SF_BAR_SIZE                              0                    0                    0                   \n        PF_TOTAL_SF                                 0                    0                    0                   \n        PHY_AUTO_NEG_P1                             DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        PHY_AUTO_NEG_P2                             DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        PHY_RATE_MASK_OVERRIDE_P1                   False(0)             False(0)             False(0)            \n        PHY_RATE_MASK_OVERRIDE_P2                   False(0)             False(0)             False(0)            \n        PLDM_FW_UPDATE_DISABLE                      False(0)             False(0)             False(0)            \n        PORT_OWNER                                  True(1)              True(1)              True(1)             \n        PRIO_TAG_REQUIRED_EN                        False(0)             False(0)             False(0)            \n        PROG_PARSE_GRAPH                            False(0)             False(0)             False(0)            \n        QOS_TRUST_STATE_P1                          TRUST_PCP(1)         TRUST_PCP(1)         TRUST_PCP(1)        \n        QOS_TRUST_STATE_P2                          TRUST_PCP(1)         TRUST_PCP(1)         TRUST_PCP(1)        \n        RATE_REDUCE_MONITOR_PERIOD_P1               4                    4                    4                   \n        RATE_REDUCE_MONITOR_PERIOD_P2               4                    4                    4                   \n        RATE_TO_SET_ON_FIRST_CNP_P1                 0                    0                    0                   \n        RATE_TO_SET_ON_FIRST_CNP_P2                 0                    0                    0                   \n        RBT_DISABLE                                 False(0)             False(0)             False(0)            \n        RDE_DISABLE                                 False(0)             False(0)             False(0)            \n        RDMA_SELECTIVE_REPEAT_EN                    False(0)             False(0)             False(0)            \n        REAL_TIME_CLOCK_ENABLE                      False(0)             False(0)             False(0)            \n        RENEG_ON_CHANGE                             True(1)              True(1)              True(1)             \n        RESET_WITH_HOST_ON_ERRORS                   False(0)             False(0)             False(0)            \n*       ROCE_ADAPTIVE_ROUTING_EN                    False(0)             True(1)              True(1)             \n        ROCE_CC_CNP_MODERATION_P1                   DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        ROCE_CC_CNP_MODERATION_P2                   DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        ROCE_CC_COMPATIBILITY_MODE                  DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        ROCE_CC_DCQCN_COMPATIBILITY_MODE            DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        ROCE_CC_LEGACY_DCQCN_SW                     False(0)             False(0)             False(0)            \n        ROCE_CC_PRIO_MASK_P1                        255                  255                  255                 \n        ROCE_CC_PRIO_MASK_P2                        255                  255                  255                 \n        ROCE_CC_SHAPER_COALESCE_P1                  DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        ROCE_CC_SHAPER_COALESCE_P2                  DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n*       ROCE_CC_STEERING_EXT                        DISABLED(1)          ENABLED(2)           ENABLED(2)          \n        ROCE_CC_TX_EVENT_ACK                        DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        ROCE_CONTROL                                ROCE_ENABLE(2)       ROCE_ENABLE(2)       ROCE_ENABLE(2)      \n        ROCE_NEXT_PROTOCOL                          254                  254                  254                 \n        ROCE_RTT_RESP_DSCP_MODE_P1                  DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        ROCE_RTT_RESP_DSCP_MODE_P2                  DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        ROCE_RTT_RESP_DSCP_P1                       0                    0                    0                   \n        ROCE_RTT_RESP_DSCP_P2                       0                    0                    0                   \n        RPG_AI_RATE_P1                              5                    5                    5                   \n        RPG_AI_RATE_P2                              5                    5                    5                   \n        RPG_BYTE_RESET_P1                           32767                32767                32767               \n        RPG_BYTE_RESET_P2                           32767                32767                32767               \n        RPG_GD_P1                                   11                   11                   11                  \n        RPG_GD_P2                                   11                   11                   11                  \n        RPG_HAI_RATE_P1                             50                   50                   50                  \n        RPG_HAI_RATE_P2                             50                   50                   50                  \n        RPG_MAX_RATE_P1                             0                    0                    0                   \n        RPG_MAX_RATE_P2                             0                    0                    0                   \n        RPG_MIN_DEC_FAC_P1                          50                   50                   50                  \n        RPG_MIN_DEC_FAC_P2                          50                   50                   50                  \n        RPG_MIN_RATE_P1                             1                    1                    1                   \n        RPG_MIN_RATE_P2                             1                    1                    1                   \n        RPG_THRESHOLD_P1                            1                    1                    1                   \n        RPG_THRESHOLD_P2                            1                    1                    1                   \n        RPG_TIME_RESET_P1                           300                  300                  300                 \n        RPG_TIME_RESET_P2                           300                  300                  300                 \n        RT_PPS_ENABLED_ON_POWERUP                   False(0)             False(0)             False(0)            \n        SAFE_MODE_ENABLE                            True(1)              True(1)              True(1)             \n        SAFE_MODE_THRESHOLD                         10                   10                   10                  \n        SILENT_MODE                                 False(0)             False(0)             False(0)            \n        SM_DISABLE                                  False(0)             False(0)             False(0)            \n        SRIOV_EN                                    True(1)              True(1)              True(1)             \n        SRIOV_IB_ROUTING_MODE_P1                    LID(1)               LID(1)               LID(1)              \n        SRIOV_IB_ROUTING_MODE_P2                    LID(1)               LID(1)               LID(1)              \n        STEERING_CACHE_REFRESH                      0                    0                    0                   \n        STRAP_SD_OR_MH                              False(0)             False(0)             False(0)            \n        STRICT_VF_MSIX_NUM                          False(0)             False(0)             False(0)            \n        SWITCH_COMPT_FEATURE_MASK                   0x0(0)               0x0(0)               0x0(0)              \n        SWP_L4_CHECKSUM_MODE                        DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        TRACER_ENABLE                               True(1)              True(1)              True(1)             \n        TUNNEL_ECN_COPY_DISABLE                     False(0)             False(0)             False(0)            \n        TUNNEL_IP_PROTO_ENTROPY_DISABLE             False(0)             False(0)             False(0)            \n        TX_PCI_DATA_FETCH_LATENCY                   40                   40                   40                  \n        TX_SCHEDULER_BURST                          0                    0                    0                   \n        TX_SCHEDULER_FWS_REACTIVITY                 DIRECT(1)            DIRECT(1)            DIRECT(1)           \n        TX_SCHEDULER_LOCALITY_FACTOR                0                    0                    0                   \n*       TX_SCHEDULER_LOCALITY_MODE                  DEVICE_DEFAULT(0)    ACCUMULATIVE(2)      ACCUMULATIVE(2)     \n        UCTX_EN                                     True(1)              True(1)              True(1)             \n        UEFI_HII_EN                                 True(1)              True(1)              True(1)             \n        UEFI_LOGS                                   DISABLED(0)          DISABLED(0)          DISABLED(0)         \n        UNKNOWN_UPLINK_MAC_FLOOD_P1                 False(0)             False(0)             False(0)            \n        UNKNOWN_UPLINK_MAC_FLOOD_P2                 False(0)             False(0)             False(0)            \n        UPSTREAM_PORT0_GEN3_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT0_GEN3_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT0_GEN4_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT0_GEN4_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT0_GEN5_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT0_GEN5_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT0_GEN6_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT0_GEN6_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT0_PCIE_BUS                     0                    0                    0                   \n        UPSTREAM_PORT0_TX_AMP                       0                    0                    0                   \n        UPSTREAM_PORT1_GEN3_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT1_GEN3_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT1_GEN4_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT1_GEN4_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT1_GEN5_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT1_GEN5_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT1_GEN6_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT1_GEN6_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT1_PCIE_BUS                     0                    0                    0                   \n        UPSTREAM_PORT1_TX_AMP                       0                    0                    0                   \n        UPSTREAM_PORT2_GEN3_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT2_GEN3_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT2_GEN4_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT2_GEN4_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT2_GEN5_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT2_GEN5_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT2_GEN6_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT2_GEN6_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT2_PCIE_BUS                     0                    0                    0                   \n        UPSTREAM_PORT2_TX_AMP                       0                    0                    0                   \n        UPSTREAM_PORT3_GEN3_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT3_GEN3_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT3_GEN4_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT3_GEN4_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT3_GEN5_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT3_GEN5_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT3_GEN6_PRESET0                 0                    0                    0                   \n        UPSTREAM_PORT3_GEN6_PRESET1                 0                    0                    0                   \n        UPSTREAM_PORT3_PCIE_BUS                     0                    0                    0                   \n        UPSTREAM_PORT3_TX_AMP                       0                    0                    0                   \n        UPT_EMULATION_ENABLE                        False(0)             False(0)             False(0)            \n*       USER_PROGRAMMABLE_CC                        False(0)             True(1)              True(1)             \n        VF_LOG_BAR_SIZE                             1                    1                    1                   \n        VF_MIGRATION_MODE                           DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        VF_NODNIC_ENABLE                            False(0)             False(0)             False(0)            \n        VF_VPD_ENABLE                               False(0)             False(0)             False(0)            \n        VIRTIO_BLK_EMU_MNG_ENABLE                   False(0)             False(0)             False(0)            \n        VIRTIO_BLK_EMU_MNG_NUM_MSIX                 2                    2                    2                   \n        VIRTIO_BLK_EMU_MNG_NUM_PF                   0                    0                    0                   \n        VIRTIO_BLK_EMU_MNG_NUM_VF_MSIX              0                    0                    0                   \n        VIRTIO_BLK_EMU_MNG_NUM_VF                   0                    0                    0                   \n        VIRTIO_BLK_EMU_MNG_SUBS_ID                  4162                 4162                 4162                \n        VIRTIO_BLK_EMU_MNG_SUBS_VENDOR_ID           6900                 6900                 6900                \n        VIRTIO_FS_EMU_MNG_ENABLE                    False(0)             False(0)             False(0)            \n        VIRTIO_FS_EMU_MNG_NUM_MSIX                  2                    2                    2                   \n        VIRTIO_FS_EMU_MNG_NUM_PF                    0                    0                    0                   \n        VIRTIO_FS_EMU_MNG_NUM_VF_MSIX               0                    0                    0                   \n        VIRTIO_FS_EMU_MNG_NUM_VF                    0                    0                    0                   \n        VIRTIO_FS_EMU_MNG_SUBS_ID                   4186                 4186                 4186                \n        VIRTIO_FS_EMU_MNG_SUBS_VENDOR_ID            6900                 6900                 6900                \n        VIRTIO_NET_EMU_MNG_ENABLE                   False(0)             False(0)             False(0)            \n        VIRTIO_NET_EMU_MNG_NUM_MSIX                 2                    2                    2                   \n        VIRTIO_NET_EMU_MNG_NUM_PF                   0                    0                    0                   \n        VIRTIO_NET_EMU_MNG_NUM_VF_MSIX              0                    0                    0                   \n        VIRTIO_NET_EMU_MNG_NUM_VF                   0                    0                    0                   \n        VIRTIO_NET_EMU_MNG_SUBS_ID                  4161                 4161                 4161                \n        VIRTIO_NET_EMU_MNG_SUBS_VENDOR_ID           6900                 6900                 6900                \n        VL15_BUFFER_SIZE_P1                         0                    0                    0                   \n        VL15_BUFFER_SIZE_P2                         0                    0                    0                   \n        VL_BUFFER_ALLOCATION_P1                     FULL_DYNAMIC(1)      FULL_DYNAMIC(1)      FULL_DYNAMIC(1)     \n        VL_BUFFER_ALLOCATION_P2                     FULL_DYNAMIC(1)      FULL_DYNAMIC(1)      FULL_DYNAMIC(1)     \n        VL_ISOLATION_MODE_P1                        DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \n        VL_ISOLATION_MODE_P2                        DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)    DEVICE_DEFAULT(0)   \nThe '*' shows parameters with next value different from default/current value.\n\nThe 'RO' shows parameters which are for read only and cannot be changed\n", "error": null}
2026-01-18T11:06:59.913152815Z  INFO    mellanox/mellanox.go:118        ParseMstconfigOutput()  {"attributes": ["INTERNAL_CPU_PAGE_SUPPLIER", "INTERNAL_CPU_ESWITCH_MANAGER", "INTERNAL_CPU_IB_VPORT0", "INTERNAL_CPU_OFFLOAD_ENGINE", "INTERNAL_CPU_MODEL"]}
2026-01-18T11:06:59.913355049Z  LEVEL(-2)       mellanox/mellanox_plugin.go:224 mellanox-plugin: configFW(): can't get DPU mode for NIC {"pciAddress": "0000:c9:00.0"}
2026-01-18T11:06:59.913370377Z  LEVEL(-2)       mellanox/mellanox_plugin.go:224 mellanox-plugin: configFW()     {"cmd-args": ["-d", "0000:c9:00.0", "-y", "set", "LAG_RESOURCE_ALLOCATION=1"]}
2026-01-18T11:06:59.913377568Z  INFO    mellanox/mellanox.go:235        RunCommand()    {"command": "mstconfig", "args": ["-d", "0000:c9:00.0", "-y", "set", "LAG_RESOURCE_ALLOCATION=1"]}
2026-01-18T11:07:00.351834128Z  LEVEL(-2)       mellanox/mellanox.go:235        RunCommand()    {"output": "\nDevice #1:\n----------\n\nDevice type:        ConnectX8           \nName:               900-9X81E-00EX-ST0_Ax\nDescription:        NVIDIA ConnectX-8 C8180L HHHL SuperNIC; 800Gbs XDR IB (default mode) / 2x400GbE; Single-cage OSFP; PCIe 6 x16 with x16 PCIe Socket Direct Extension option; Crypto Enabled; Secure Boot Enabled\nDevice:             0000:c9:00.0        \n\nConfigurations:                                          Next Boot       New\n        LAG_RESOURCE_ALLOCATION                     PRE_ALLOCATION(1)    PRE_ALLOCATION(1)   \n\n Apply new Configuration? (y/n) [n] : y\nApplying... Done!\n-I- Please reboot machine to load new configurations.\n", "error": null}
2026-01-18T11:07:00.351932862Z  INFO    daemon/daemon.go:377    k8s plugin Apply()
2026-01-18T11:07:00.351952976Z  INFO    Apply   daemon/daemon.go:286    reboot node     {"controller": "sriovnetworknodestate", "controllerGroup": "sriovnetwork.openshift.io", "controllerKind": "SriovNetworkNodeState", "SriovNetworkNodeState": {"name":"worker2","namespace":"network-operator"}, "namespace": "network-operator", "name": "worker2", "reconcileID": "9ec4ac93-ade4-425e-ad9e-6d7d789634ce"}
2026-01-18T11:07:00.35997882Z   INFO    rebootNode      daemon/daemon.go:397    trigger node reboot
2026-01-18T11:07:00.360055394Z  INFO    daemon/daemon.go:633    RunCommand()    {"command": "systemd-run", "args": ["--unit", "sriov-network-config-daemon-reboot", "--description", "sriov-network-config-daemon reboot node", "/bin/sh", "-c", "systemctl stop kubelet.service; reboot"]}
E0118 11:07:00.360468    6462 token_source.go:188] "Unable to rotate token" err="failed to read token file \"/var/run/secrets/kubernetes.io/serviceaccount/token\": open /var/run/secrets/kubernetes.io/serviceaccount/token: no such file or directory"
2026-01-18T11:07:00.384415505Z  LEVEL(-2)       daemon/daemon.go:633    RunCommand()    {"output": "", "error": null}

@rollandf rollandf changed the title fix: Mellanox reboot loop and improve devlink parameter reconciliation fix: reboot loop and improve devlink parameter reconciliation Jan 18, 2026
Copy link
Collaborator

@heyvister1 heyvister1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@e0ne e0ne merged commit 94dfa2a into Mellanox:network-operator-26.1.x Jan 19, 2026
11 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants