-
Notifications
You must be signed in to change notification settings - Fork 101
feat(ComparisonReport[CVReport]): Add ROC curve #1669
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(ComparisonReport[CVReport]): Add ROC curve #1669
Conversation
9f2d56b to
dc43c1f
Compare
dc43c1f to
e008c11
Compare
e397976 to
82468ec
Compare
Coverage Report for |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| File | Stmts | Miss | Cover | Missing |
|---|---|---|---|---|
| venv/lib/python3.12/site-packages/skore | ||||
| __init__.py | 23 | 0 | 100% | |
| _config.py | 28 | 0 | 100% | |
| exceptions.py | 4 | 4 | 0% | 4, 15, 19, 23 |
| venv/lib/python3.12/site-packages/skore/project | ||||
| __init__.py | 2 | 0 | 100% | |
| metadata.py | 67 | 0 | 100% | |
| project.py | 43 | 0 | 100% | |
| reports.py | 11 | 0 | 100% | |
| widget.py | 138 | 5 | 96% | 375–377, 447–448 |
| venv/lib/python3.12/site-packages/skore/sklearn | ||||
| __init__.py | 6 | 0 | 100% | |
| _base.py | 169 | 14 | 91% | 45, 58, 126, 129, 182, 185–186, 188–191, 224, 227–228 |
| find_ml_task.py | 61 | 0 | 100% | |
| types.py | 20 | 0 | 100% | |
| venv/lib/python3.12/site-packages/skore/sklearn/_comparison | ||||
| __init__.py | 5 | 0 | 100% | |
| metrics_accessor.py | 205 | 3 | 98% | 165, 329, 1283 |
| report.py | 95 | 0 | 100% | |
| utils.py | 58 | 0 | 100% | |
| venv/lib/python3.12/site-packages/skore/sklearn/_cross_validation | ||||
| __init__.py | 5 | 0 | 100% | |
| metrics_accessor.py | 204 | 1 | 99% | 321 |
| report.py | 108 | 0 | 100% | |
| venv/lib/python3.12/site-packages/skore/sklearn/_estimator | ||||
| __init__.py | 7 | 0 | 100% | |
| feature_importance_accessor.py | 143 | 2 | 98% | 216–217 |
| metrics_accessor.py | 367 | 8 | 97% | 181, 183, 190, 281, 350, 354, 369, 404 |
| report.py | 155 | 0 | 100% | |
| venv/lib/python3.12/site-packages/skore/sklearn/_plot | ||||
| __init__.py | 2 | 0 | 100% | |
| base.py | 5 | 0 | 100% | |
| style.py | 28 | 0 | 100% | |
| utils.py | 118 | 5 | 95% | 50, 74–76, 80 |
| venv/lib/python3.12/site-packages/skore/sklearn/_plot/metrics | ||||
| __init__.py | 5 | 0 | 100% | |
| confusion_matrix.py | 69 | 4 | 94% | 90, 98, 120, 228 |
| precision_recall_curve.py | 174 | 2 | 98% | 521, 524 |
| prediction_error.py | 160 | 0 | 100% | |
| roc_curve.py | 242 | 4 | 98% | 380, 497, 598, 791 |
| venv/lib/python3.12/site-packages/skore/sklearn/train_test_split | ||||
| __init__.py | 0 | 0 | 100% | |
| train_test_split.py | 49 | 0 | 100% | |
| venv/lib/python3.12/site-packages/skore/sklearn/train_test_split/warning | ||||
| __init__.py | 8 | 0 | 100% | |
| high_class_imbalance_too_few_examples_warning.py | 17 | 1 | 94% | 80 |
| high_class_imbalance_warning.py | 18 | 0 | 100% | |
| random_state_unset_warning.py | 10 | 0 | 100% | |
| shuffle_true_warning.py | 10 | 1 | 90% | 46 |
| stratify_is_set_warning.py | 10 | 0 | 100% | |
| time_based_column_warning.py | 21 | 1 | 95% | 73 |
| train_test_split_warning.py | 4 | 0 | 100% | |
| venv/lib/python3.12/site-packages/skore/utils | ||||
| __init__.py | 6 | 2 | 66% | 8, 13 |
| _accessor.py | 52 | 2 | 96% | 67, 108 |
| _environment.py | 27 | 0 | 100% | |
| _fixes.py | 8 | 0 | 100% | |
| _index.py | 5 | 0 | 100% | |
| _logger.py | 22 | 4 | 81% | 15–17, 19 |
| _measure_time.py | 10 | 0 | 100% | |
| _parallel.py | 38 | 3 | 92% | 23, 33, 124 |
| _patch.py | 13 | 5 | 61% | 21, 23–24, 35, 37 |
| _progress_bar.py | 45 | 0 | 100% | |
| _show_versions.py | 33 | 2 | 93% | 65–66 |
| _testing.py | 12 | 0 | 100% | |
| TOTAL | 3145 | 73 | 97% | |
| Tests | Skipped | Failures | Errors | Time |
|---|---|---|---|---|
| 786 | 5 💤 | 0 ❌ | 0 🔥 | 59.970s ⏱️ |
848f8c7 to
fb72191
Compare
62ec965 to
5f8adf6
Compare
5f8adf6 to
99d2c62
Compare
glemaitre
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is a partial review. I still have to look in details the roc_curve file and the tests but I thought that it could already be useful to look at those.
skore/src/skore/sklearn/_plot/metrics/precision_recall_curve.py
Outdated
Show resolved
Hide resolved
skore/src/skore/sklearn/_plot/metrics/precision_recall_curve.py
Outdated
Show resolved
Hide resolved
15d93fd to
47cd9d4
Compare
Starting from probabl-ai#1669, it is sometimes possible to draw plots onto (a `ndarray` of) several Axes rather than just a single Axes. In fact, starting from probabl-ai#1720, the same data might be drawn on one or the other depending on user parameters. In this context, allowing the user to pass `ax` might add too much complexity compared to how useful it might be; it is considered an advanced use-case which is difficult to motivate. This PR thus removes the ability to pass user-created `ax`.
Starting from probabl-ai#1669, it is sometimes possible to draw plots onto (a `ndarray` of) several Axes rather than just a single Axes. In fact, starting from probabl-ai#1720, the same data might be drawn on one or the other depending on user parameters. In this context, allowing the user to pass `ax` might add too much complexity compared to how useful it might be; it is considered an advanced use-case which is difficult to motivate. This PR thus removes the ability to pass user-created `ax`.
Starting from probabl-ai#1669, it is sometimes possible to draw plots onto (a `ndarray` of) several Axes rather than just a single Axes. In fact, starting from probabl-ai#1720, the same data might be drawn on one or the other depending on user parameters. In this context, allowing the user to pass `ax` might add too much complexity compared to how useful it might be; it is considered an advanced use-case which is difficult to motivate. This PR thus removes the ability to pass user-created `ax`.
Starting from probabl-ai#1669, it is sometimes possible to draw plots onto (a `ndarray` of) several Axes rather than just a single Axes. In fact, starting from probabl-ai#1720, the same data might be drawn on one or the other depending on user parameters. In this context, allowing the user to pass `ax` might add too much complexity compared to how useful it might be; it is considered an advanced use-case which is difficult to motivate. This PR thus removes the ability to pass user-created `ax`.
glemaitre
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, I check in depth the roc_curve file. I have a couple of nit pickings.
Otherwise, I think that we can add a couple of tests for the kwargs (error and behaviour) and checking the dataframe as well.
I need to figure out some details for the legend.
skore/tests/unit/sklearn/plot/roc_curve/test_comparison_cross_validation.py
Outdated
Show resolved
Hide resolved
skore/tests/unit/sklearn/plot/roc_curve/test_comparison_cross_validation.py
Outdated
Show resolved
Hide resolved
skore/tests/unit/sklearn/plot/roc_curve/test_comparison_cross_validation.py
Show resolved
Hide resolved
|
[automated comment] Please update your PR with main, so that the |
For consistency with precision-recall curve, where we had to migrate because some style kwargs are not valid for LineCollection.
7e8d4cc to
4b3530c
Compare
glemaitre
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. We can go ahead and revisit the legend all at once.
|
Nice, two plots to go. @sylvaincom You could start to look at creating the |

Demo:
Depends on #1668, #1682
TODO:
n_curvesis not calculated correctly in the comparison-cv case (unclear in what way)pos_label is not Nonecase:pos_labelhas no effect