Do not fail requests with readonly middleware

[C-Otto]

Background

I'm using the middleware feature to "spy" on my node and extract useful information out of the requests/responses caused by my non-middleware software. As I'm only interested in the information provided to me as part of the requests/responses, I'm using the readonly middleware mode.

However, I notice two issues:

• when I stop my middleware client, requests might fail (because the middleware disconnected)
• when I do not "acknowledge" requests/responses in time, requests fail (or responses do not reach their destinations)

As my middleware client is unable to cancel or alter any request (or response), I expect that my non-middleware gRPC clients are unaffected by the middleware (connection, disconnection, not acknowledging messages, ...).

Your environment

• version of lnd: v0.14.3-beta-rc1
• version of btcd, bitcoind, or other backend: bitcoind v22.0.0

Steps to reproduce

Connect middleware with "readonly" registration. Ignore incoming messages. Send gRPC request with some other program.

Expected behaviour

Other program is unaffected.

Actual behaviour

Other program's request fails.

[Roasbeef]

The current architecture is based on "interception" hence the blocking traits it has today. In the future, we could introduce a non-blocking mode, where it just attempts a best effort send, and never tries to wait for a response.

[gkrizek]

I think this could be related to #6839

With a specific ReadOnly RPC you'd be able to see the middleware but not affect any of the requests.