`opx-config-fanout` should fail if interfaces are part of bridges or port groups

[vnam1]

OPX version 2.3.1
Platform S6000

I split a 40g interface into 4x1 10g and applied some interface config on the split ports.
Later on, when I try to unsplit the command back to 40g, it fails:
root@leaf-1:/home/admin# opx-config-fanout e101-007-1 1x1 40g
Deleting.. e101-007-1
Deleting.. e101-007-2
Deleting.. e101-007-3
Deleting.. e101-007-4
Failed...
Creating interface e101-007-1
Creating interface e101-007-2
Creating interface e101-007-3
Creating interface e101-007-4
Failed to create interfaces

syslog has following error messages:

Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], Interface 39 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: Cannot find device "e101-007-1"
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NEIGH-UPD], cmd:ip neigh flush to fe80::eef4:bbff:fefc:7931/64 dev e101-007-1 failed
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], npu:port = 0:1 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Not interface config set event
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Not interface config set event
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [QOS:QOS-if-delete], ifindex: 39 is being deleted
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Not interface config set event
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Not interface config set event
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [QOS:QOS-if-delete], ifindex: 40 is being deleted
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-COM-INT-GET], Get request handler not present
Jul 24 19:23:47 leaf-1 kernel: [ 379.826963] br10: port 1(e101-007-3.10) entered disabled state
Jul 24 19:23:47 leaf-1 kernel: [ 379.827796] br11: port 1(e101-007-3.11) entered disabled state
Jul 24 19:23:47 leaf-1 kernel: [ 379.828209] device e101-007-3 left promiscuous mode
Jul 24 19:23:47 leaf-1 python[1345]: Failed to get Interface attributes by ifindex from NAS interface
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 kernel: [ 379.828890] br12: port 1(e101-007-3.12) entered disabled state
Jul 24 19:23:47 leaf-1 kernel: [ 379.830206] br10: port 1(e101-007-3.10) entered disabled state
Jul 24 19:23:47 leaf-1 kernel: [ 379.831807] br11: port 1(e101-007-3.11) entered disabled state
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-COM-INT-GET], Get request handler not present
Jul 24 19:23:47 leaf-1 python[1345]: Failed to get Interface attributes by ifindex from NAS interface
Jul 24 19:23:47 leaf-1 kernel: [ 379.834904] br12: port 1(e101-007-3.12) entered disabled state
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [DSAPI:CPS IPC], Was not able to send the full message body.
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NEIGH-UPD], cmd:ip neigh flush to ff00::/8 dev failed
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: Command line is not complete. Try option "help"
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], Interface 39 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], Interface 40 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: Command line is not complete. Try option "help"
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NEIGH-UPD], cmd:ip neigh flush to 172.16.0.3/32 dev failed
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:IP-PUB], Interface 40 to if_name from OS returned error
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-LINUX-INTERFACE], Invalid Interface Index 48
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-LINUX-INTERFACE], Invalid Interface Index 46
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-LINUX-INTERFACE], Invalid Interface Index 44
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], Interface 40 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], Interface 41 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Not interface config set event
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Not interface config set event
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [QOS:QOS-if-delete], ifindex: 41 is being deleted
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], Interface 48 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [ROUTE:HAL-RT-DR], Intf name get failed for if_index:48
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [ROUTE:HAL-RT-DR], Intf name get failed for if_index:48
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-COM-INT-GET], Get request handler not present
Jul 24 19:23:47 leaf-1 python[1345]: Failed to get Interface attributes by ifindex from NAS interface
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], npu:port = 0:2 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], npu:port = 😇 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], npu:port = 0:4 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: Command line is not complete. Try option "help"
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NEIGH-UPD], cmd:ip neigh flush to ff00::/8 dev failed
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], Interface 46 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [ROUTE:HAL-RT-DR], Intf name get failed for if_index:46
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [ROUTE:HAL-RT-DR], Intf name get failed for if_index:46
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: Command line is not complete. Try option "help"
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NEIGH-UPD], cmd:ip neigh flush to 172.16.0.7/32 dev failed
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: Command line is not complete. Try option "help"
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NEIGH-UPD], cmd:ip neigh flush to ff00::/8 dev failed
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], Interface 44 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [ROUTE:HAL-RT-DR], Intf name get failed for if_index:44
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [ROUTE:HAL-RT-DR], Intf name get failed for if_index:44
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: Command line is not complete. Try option "help"
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NEIGH-UPD], cmd:ip neigh flush to 172.16.0.12/32 dev failed
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-LINUX-INTERFACE], Invalid Interface Index 48
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-LINUX-INTERFACE], Invalid Interface name e101-007-3
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-UPD-VLAN], Add tag vlan-index:47 sub-mbr-index:1946168280 to mbr-index mapping does not exist!
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-LINUX-INTERFACE], Invalid Interface Index 46
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-Vlan], Error finding index 0 in intf_ctrl
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-LINUX-INTERFACE], Invalid Interface name e101-007-3
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-UPD-VLAN], Add tag vlan-index:45 sub-mbr-index:1946168280 to mbr-index mapping does not exist!
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-LINUX-INTERFACE], Invalid Interface Index 44
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-Vlan], Error finding index 0 in intf_ctrl
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-LINUX-INTERFACE], Invalid Interface name e101-007-3
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:NAS-UPD-VLAN], Add tag vlan-index:43 sub-mbr-index:1946168280 to mbr-index mapping does not exist!
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NAS_OS:IP-PUB], Interface 42 to if_name from OS returned error
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-Vlan], Error finding index 0 in intf_ctrl
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], Interface 41 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], Interface 42 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:INTF-C], Failed to get if_info
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-INT], Interface 42 returned error -2013167616
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [ev_log_t_SAI_BRIDGE:Switch Id: 0], Error - bridge port 0x3a000000000023 in use - ref count 3
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [NDI:NDI-PORT], Delete bridge ports failed for bridge port oid 3a000000000023
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [ev_log_t_SAI_PORT:Switch Id: 0], port id 0x100000000001b is in use
Jul 24 19:23:47 leaf-1 opx_nas_daemon[1159]: [INTERFACE:NAS-PHY-DELETE], Failure on NDI port delete
Jul 24 19:23:47 leaf-1 python[1341]: [DSAPI:COMMIT], Failed to commit request at 0 out of 1
Jul 24 19:23:47 leaf-1 python[1341]: Failed to delete physical port
Jul 24 19:23:47 leaf-1 python[6150]: [DSAPI:COMMIT], Failed to commit request at 0 out of 1
Jul 24 19:23:47 leaf-1 python[1341]: Physical port object not found
Jul 24 19:23:47 leaf-1 python[6150]: [DSAPI:COMMIT], Failed to commit request at 0 out of 4

[jeff-yin]

@vnam1 it would help to know what interface configs you had applied to the fanned-out ports.

[vnam1]

config is:

auto e101-007-1
iface e101-007-1
address 172.16.0.3/31
alias facing_spine-1:swp1
mtu 9050

auto e101-007-2
iface e101-007-2
address 172.16.0.7/31
alias facing_spine-2:swp1
mtu 9050

L3Edge interfaces

auto e101-007-4
iface e101-007-4
address 172.16.0.12/31
alias facing_external-router:eth1

Also e101-007-3 is a tagged port in bridges br10, br11 and br12

[jeff-yin]

If the ports are part of some bridge or bond, you have to remove them from the bridge/bond prior to changing fanout settings. Closing the issue for now.

[vnam1]

I feel OPX needs to handle this gracefully by giving an error message and not proceeding ahead.
Crashing and interfaces getting disappeared leaves the box in unusable state (only workaround I found is to reboot)

[jeff-yin]

Ok, let's leave it open then. We can work on building more intelligence into the opx-config-fanout tool