Skip to content

Conversation

@sgreenbury
Copy link
Collaborator

@sgreenbury sgreenbury commented Oct 29, 2025

Contributes towards #884.

This PR:

  • Adds ProbabilisticMetric base class (no new methods for the moment, but allows extension)
  • Adds CRPS metric with handling for OutputLike
  • Updates the Metric and evaluate() API to support passing OutputLike
  • Revises model selection to use .predict() so that probabilistic or deterministic metrics can be used interchangeably
  • Adds einops dep

This might close #884 if no other probabilistic metrics are planned for the now.

@sgreenbury sgreenbury marked this pull request as ready for review October 30, 2025 11:37
Copy link
Collaborator

@EdwinB12 EdwinB12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great Sam!

@radka-j
Copy link
Member

radka-j commented Oct 30, 2025

Lets close #884 when this is merged.

@sgreenbury sgreenbury merged commit bf210f5 into main Oct 31, 2025
5 checks passed
@sgreenbury sgreenbury mentioned this pull request Nov 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add metrics for probabilistic outputs

4 participants