Skip to content

[ENH]: Make PinballLoss missing alpha error message more informative#978

Open
MOHITKOURAV01 wants to merge 1 commit intosktime:mainfrom
MOHITKOURAV01:enh/pinball-loss-error
Open

[ENH]: Make PinballLoss missing alpha error message more informative#978
MOHITKOURAV01 wants to merge 1 commit intosktime:mainfrom
MOHITKOURAV01:enh/pinball-loss-error

Conversation

@MOHITKOURAV01
Copy link

Reference Issues/PRs

Fixes #977

What does this implement/fix? Explain your changes.

This PR enhances the ValueError message raised by the PinballLoss metric when one or more requested alpha quantiles are missing from the given predictions (y_pred).

Previously, the error message was generic and did not specify which alphas were missing, requiring manual debugging. This resolves an active TODO in skpro/metrics/_classes.py to make the error message more informative.

Changes:

  • Modified PinballLoss._evaluate_by_index to compute the set difference between requested alpha and available y_pred_alphas.
  • Updated the error string to explicitly list the missing values.
  • Old Error Message: ValueError: "not all quantile values in alpha are available in y_pred"
  • New Error Message: ValueError: "not all quantile values in alpha are available in y_pred. Missing alphas: [0.1, 0.9]"

Does your contribution introduce a new dependency? If yes, which one?

No.

What should a reviewer concentrate their feedback on?

  • The string formatting of the new ValueError message in PinballLoss.
  • The logic used to identify missing alphas (set difference).

Did you add any tests for the change?

Yes, I have updated test_evaluate_alpha_negative in test_probabilistic_metrics.py to use pytest.raises(ValueError, match=".*Missing alphas:.*"). This ensures the new informative message is correctly triggered and verified.

Any other comments?

All probabilistic metrics tests passed successfully on my local machine using pytest.

PR checklist

For all contributions
  • I've added myself to the list of contributors with any new badges I've earned :-) (Badge: code)
  • The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. (Using [ENH])
For new estimators
  • I've added the estimator to the API reference. (N/A - existing estimator)
  • I've added illustrative usage examples to the docstring. (N/A)
  • I've set the python_dependencies tag for soft dependencies. (N/A)

@MOHITKOURAV01 MOHITKOURAV01 changed the title ENH: Make PinballLoss missing alpha error message more informative [ENH]: Make PinballLoss missing alpha error message more informative Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ENH] Make missing alpha error message in PinballLoss more informative.

1 participant