NodeNetworkInterfaceDown alert seems to mix interface state from different nodes

[bernhardloos]

What happened:
I tested the updated expression from #3539 and some of the NodeNetworkInterfaceDown alerts vanish but not all.
If two nodes have an interface with the same name but one is in use and one isn't, the alerts still appear. This seems to be related to the "on(device) node_network_up == 1" part of the expression.

What you expected to happen:
No NodeNetworkInterfaceDown alerts.

How to reproduce it (as minimally and precisely as possible):
Node1 has eno1 (in use) and eno2 (in use)
Node2 has eno1 (in use) and enp5s0 (not in use)
Node3 has eno1 (in use) and eno2 (not in use)

There is an NodeNetworkInterfaceDown alert for Node3 but not Node2.

Environment:

• OKD 4.19.0-okd-scos.15
• KubeVirt Operator 1.15.2 from Community Operators

[machadovilaca]

I think this is fixed by #3736

[sradco]

@avlitman please see if the PR you created fixes this issue.
Also, did we backport the fix tonolder versions?

[nunnatsa]

did we backport the fix tonolder versions?

@sradco - not really - only to the new release-1.16 branch.

[sradco]

We should backport it. Alerts create noise in the cluster and we must trybto reduce this node if its due to a bug.

[bernhardloos]

I can see that this was backported to various old releases. This is nice but not particularly useful as there are basically no patch releases as far as I can see?

[kubevirt-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[kubevirt-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten