route calculation performed too frequently under high load

[rade]

The route calculation contains some clever logic to coalesce multiple recalculation requests into one. However, I am quite sure this isn't effective when receiving a deluge of topology updates on a single connection, such as can happen under load when backlogs can accumulate in tcp buffers. Each update will trigger a separate recalculation. With #1761 in mind, this is hugely costly.

[rade]

We could perhaps attempt to drain connections of gossip traffic before processing it in bulk. Or maybe make GossipChannels active components that have separate process and propagation phases.

[rade]

Simply dropping the EnsureRecalculated calls does have quite a beneficial effect. At the cost of accuracy. Does that matter? I reckon the place where this could do the most harm is in broadcastPeerUpdate when we've just added a connection - there is a risk that the broadcast won't reach the new peer. OTOH, if it really is a new peer then we actually SendAllGossipDown the connection. And if it's not a new peer then the broadcast might still reach it via the old route.

[rade]

Simply dropping the EnsureRecalculated calls does have quite a beneficial effect. At the cost of accuracy. Does that matter?

Some of the gossip unit tests fail. Not all that surprising, I suppose.

[rade]

Since we solved #1761, this is less important.