Make sure containers don't exit with non zero status during e2e runs

[mflendrich]

Issue originally authored by tnozicka as #1383

We should wire an invariant checker that verifies that no container exits with non zero status during an e2e run. There likely needs to be an annotation to except from this in cases where we do it on purpose in the e2e test.

This would have caught some of the bugs and flakes (like #1231 (comment)) we had and save debugging time.