Improve offline algo performance

[MischaPanch]

For @arnaujc91

Background: currently the pre-processing of most offline learning algorithms is done in _preprocess_batch, which is highly suboptimal. Instead it should be done in process_buffer.

In this PR, a general class that implements process_buffer using _preprocess_batch is introduced that allows converting an OffPolicyAlgorithm into an efficient OfflineAlgorithm.

Current status: when using it to improve performance of TD3BC, the algorithm converges and tests pass, but the determinism test fails, meaning something changed in the processing or at least in the random number generation. If it's the latter, the failure is not a problem, but I currently don't see why any rng related things should have changed.

The implementation that changes the buffer's managed batch is rather hacky. I suspect something goes wrong with the indexing but after 20 mins of debugging I haven't yet pinned down what causes the determinism test to fail. Understanding this will require reading through the sample_indices in ReplayBuffer and ReplayBufferManager. Note that there is inconsistency in how sample_indices(None) is handled between the two, but it shouldn't play a role for this PR.

In the course of this PR, determinism snapshots should be created from the dev-v2 branch, then after switching to this branch, the determinism test of td3bc should succeed. See docstring of AlgorithmDeterminismTest in determinism_test.py for further details.

The PR is finished when all relevant offline algorithms inherit from OfflineAlgorithmFromOffPolicyAlgorithm and either the determinism tests pass, or a source of different random number generation caused by the refactoring has been identified.

[MischaPanch]

After a long investigation, we understood that this is fundamentally the wrong approach. preprocess_batch of the off-policy algo uses the learned value function to estimate returns, and the value function changes all the time with gradient updates. So it's not possible to just compute the returns once, or rather, it gives a poo estimate. Would work with purely monte-carlo estimates of returns, but we don't control how an algorithm would estimate returns, and we currently don't have a single algorithm implementation of that kind. Closing this PR