Description
In certain conditions, Ravel fails to write iptables via the iptables-restore command.
The effect of this is putting the current rules in stasis. Ravel spins on iptables-restore attempting to write the new rules, and the failure behavior is to error out and leave the iptables as they are, meaning new pods scheduled will not be added to the service chain and will not be able to receive traffic. The erroneous rules are written to the container.
The conditions for reproducing this bug are unknown. This has happened a myriad of times, in numerous environments. It is unclear what state triggers this event. The erroneous iptables are typically many thousands of lines long, making isolating where the bad line is difficult. iptables-restore is unhelpful because while it notes a syntax error, it only says the error is on the last line.