Skip to content

Conversation

@glemaitre
Copy link
Member

closes #1061

image

We are more flexible using a regular expression to check the score names. In addition, we take care to test first the neg_ part that would mean that negative score are therefore "higher is greater" convention.

@augustebaum augustebaum self-requested a review January 9, 2025 09:00
Copy link
Contributor

@augustebaum augustebaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "neg_ means higher is better" rule makes me a bit wary of false positives.
For example, neg_accuracy gets falsely considered as higher-is-better.

@glemaitre
Copy link
Member Author

It will never happen in practice. The neg_ is only a scikit-learn convention used to negate the loss/error/deviance such that whatever metrics used in a grid-search will be maximized.

So if you are a user, you are never going to defined your own neg_ metric to make a score that lower is better.

@augustebaum
Copy link
Contributor

Sounds good.

Copy link
Contributor

@augustebaum augustebaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@augustebaum augustebaum merged commit b7ab74a into probabl-ai:main Jan 9, 2025
19 checks passed
waridrox pushed a commit to waridrox/skore that referenced this pull request Apr 15, 2025
…robabl-ai#1063)

closes probabl-ai#1061 


![image](https://github.com/user-attachments/assets/98d002c1-874d-4de2-bb26-5fc16838b2f1)

We are more flexible using a regular expression to check the score
names. In addition, we take care to test first the `neg_` part that
would mean that negative score are therefore "higher is greater"
convention.

---------

Co-authored-by: Auguste Baum <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The check for metric name and show lower/higher is better is too strict

2 participants