Docs: Improve retry policy and speculative execution policy

[Lorak-mmk]

@fee-mendes noticed that our docs are a bit lacking with regards when and how those policies will be applied.

• Speculative execution will only work if statement is marked as idempotent. It is not mentioned in speculative execution chapter of the book ( https://rust-driver.docs.scylladb.com/stable/speculative-execution/speculative.html ), and neither in SpeculativeExecutionPolicy trait docs https://docs.rs/scylla/latest/scylla/policies/speculative_execution/trait.SpeculativeExecutionPolicy.html
• RetryPolicy in contrary does not strictly require idempotent requests - its the policy that decides what to do, and can take idempotency into consideration. We should say that explicitly in the docs
• DefaultRetryPolicy has quite complex logic - it takes into consideration what error occured, what happened in previous attempt and wheter the statement is idempotent. Our docs are imo a bit laconic - we should describe the logic a bit more, instead of just linking java driver docs and saying "It retries when there is a high chance that it might help".
• Speculative execution also has a logic that decides whether to fail early based on the error returned - it is not documented anywhere.

[fee-mendes]

Our docs are imo a bit laconic - we should describe the logic a bit more, instead of just linking java driver docs and saying "It retries when there is a high chance that it might help".

Yeah, I agree - I was just now reading the impl to figure out what does it even mean lol

Btw, orthogonal to this issue as it is only docs related, we should perhaps have a way to log Retries and their reason. I ran into a situation where I am almost certain Retries occurred, but I don't know what led to it nor which events it received from the caller to decide on that. Having a logger would help with making more sense of things. (And maybe there's a logger - I didn't check, on which case guidance would be appreciated :-)

[wprzytula]

Btw, orthogonal to this issue as it is only docs related, we should perhaps have a way to log Retries and their reason. I ran into a situation where I am almost certain Retries occurred, but I don't know what led to it nor which events it received from the caller to decide on that. Having a logger would help with making more sense of things. (And maybe there's a logger - I didn't check, on which case guidance would be appreciated :-)

Check out HistoryCollector. I think this is what you ask for.