Better handling of non interactive runs

[maurerle]

When using a NON_INTERACTIVE flag everywhere, we have a strict default behavior, but we might want to test how the program reacts to different inputs, using different behaviors. So I think a general NON_INTERACTIVE flag might be not suited as a long-term solution

Originally posted by @jrasko in #648 (comment)

[jrasko]

For now, the NON_INTERACTIVE flag does the trick. If we implement tests with non-default behavior in future, we can invent boolean flags for that case.