New metrics for ensemble tuning objective

[dylanbouchard]

Is your feature request related to a problem? Please describe.
Currently, our UQEnsemble.tune and UQEnsemble.tune_from_graded methods offer the following metrics:
['fbeta_score', 'accuracy_score', 'balanced_accuracy_score', 'roc_auc', 'log_loss']. We we would like to include additional metrics, including, but not limited to, area under precision-recall curve and brier score.

Describe the solution you'd like
These metrics should be added to the uqlm.utils.Tuner class first, and offered by UQEnsemble methods by class composition (Tuner is stored as an attribute of UQEnsemble).

Describe alternatives you've considered
Current metrics

Additional context
Ideally, this issue should be worked on after #3 is completed and coded from scratch rather than relying on scikit-learn.

[vaifai]

Hey @dylanbouchard, can I work on this ? I actually stumbled upon UQLM through your post Eulether AI and I am really interested in contributing to it.

[vaifai]

Also, do I need to wait upon #3 to get completed ?

[dylanbouchard]

@vaifai thanks for your interest!

I believe @mohitcek is almost finished with #3. For now, please feel free to begin writing the code for computation of the new metrics (without relying on scikit-learn) and Mohit's PR can inform how those metrics get integrated after you have that code written.

[vaifai]

Thanks Dylan, I will get started with it.

[vaifai]

Hey @dylanbouchard , while making changes, I noticed Mohit has raised a PR to remove scikit-learn. I have a couple of new metrics but I think I will wait for Mohit's PR to get merged first and would raise my PR following that.