Data dependency on subsequent actions with `FailureMode::Allow`

[adam-cattermole]

With the changes in #158 we no longer resume and ignore subsequent actions when an action with FailureMode::Allow fails. This creates an "undesirable" behaviour for some cases.

An example is performing authenticated rate-limiting where the auth service is set to FailureMode::Allow; on failure there is no data returned from the auth service, so if your rate limit action is predicated on data expected from the auth request, or the data is used to determine which limit to apply, the rate limiting would not take place and all requests would "silently" go through.

This can be easily reproduced with the *.a.multi.com example in our test environment https://github.com/Kuadrant/wasm-shim/blob/main/utils/deploy/envoy.yaml#L296-L323, by setting authorino failureMode: allow and scaling down both the authorino-operator and authorino; all requests go through the rate-limiting service and are allowed with 200.

[maleck13]

@adam-cattermole where would this configuration naturally surface? It seems we want to allow a user to define the behaviour in these circumstances. Would it be perhaps reasonable to define a global behaviour at the Kuadrant CR level and then allow it to be overridden in the AP and RLP for example?
It would seem on the surface at least that a user defining an Authenticated RLP would know they are doing that and so it would be reasonable to expect them to decide on the behaviour in case of failure