Update metrics to be able to access score per task/output

[radka-j]

At the moment we average all metrics over tasks. It would be useful to also have access scores per tasks.

[EdwinB12]

Starting scoping this issue out.

bootstrap returns a dictionary of dict of mean and std for each metric..

Each metric has a single entry and the mean and std must be a single float value for each bootstrap sample. The mean and std are then taken for each bootstrap sample to get a single value. The computation of the metric is handled in the evaluate function.

For torchmetrics, we can use the keyword argument 'reduction' that is set to 'mean' by default to 'none'. This means it will do no reduction and return an array of the same size of input (except for the first dimension). As all tensors should have 2 axis, this is what we want.

CRPS we would have to look at in a bit more detail.

We might have to think a little about how Result handles this; maybe we start by just returning an array in the pandas dataframe.

In short, I see no blocker for now within the compare loop at least. Will start implenting this tomorrow.

[radka-j]

Just a note here that I have added a reduction kwarg to a new metric in #924. As part of the PR, I also update evaluate and boostrap functions in model_selection.py to accept metric specific kwargs.