feat(EstimatorReport): Display the feature permutation importance and mean decrease impurity (#1365)

Closes #1319 and #1323

Todo:
- ~[ ] Add example~ deferred to #1390 
- [x] Coverage at 100%
- [x] Add API docs
- [x] Check for `# TODO` in code
- [x] Check what happens if `scoring` is a callable (is it cached?)
    - scoring was not included in the cache key at all! Fixed
- [x] Check what happens if `random_state` is a `RandomState` instance
(is it cached?)
- If `random_state` is a RandomState then the call should not be cached,
because reusing a `RandomState` gives a different result.

---------

Co-authored-by: Sylvain Combettes <[email protected]>

Author: auguste-probabl

README.md

@@ -32,7 +32,7 @@ skore is a Python open-source library designed to help data scientists apply rec
 - **Diagnose**: catch methodological errors before they impact your models.
   - `train_test_split` supercharged with methodological guidance: the API is the same as scikit-learn's, but skore displays warnings when applicable. For example, it warns you against shuffling time series data or when you have class imbalance.
 - **Evaluate**: automated insightful reports.
-  - `EstimatorReport`: feed your scikit-learn compatible estimator and dataset, and it generates recommended metrics and plots to help you analyze your estimator. All these are computed and generated for you in 1 line of code. Under the hood, we use efficient caching to make the computations blazing fast.
+  - `EstimatorReport`: feed your scikit-learn compatible estimator and dataset, and it generates recommended metrics, feature importance, and plots to help you evaluate and inspect your estimator. All these are computed and generated for you in 1 line of code. Under the hood, we use efficient caching to make the computations blazing fast.
   - `CrossValidationReport`: get a skore estimator report for each fold of your cross-validation.
   - `ComparisonReport`: benchmark your skore estimator reports.
 

examples/getting_started/plot_skore_getting_started.py

@@ -12,7 +12,8 @@
 # #.    Get assistance when developing your ML/DS projects to avoid common pitfalls
 #       and follow recommended practices.
 #
-#       *   :class:`skore.EstimatorReport`: get an insightful report on your estimator
+#       *   :class:`skore.EstimatorReport`: get an insightful report on your estimator,
+#           for evaluation and inspection
 #
 #       *   :class:`skore.CrossValidationReport`: get an insightful report on your
 #           cross-validation results
@@ -82,11 +83,19 @@
 roc_plot.plot()
 plt.tight_layout()
 
+# %%
+# Furthermore, we can inspect the model using the permutation feature importance:
+
+# %%
+log_reg_report.feature_importance.feature_permutation().T.boxplot(vert=False)
+plt.tight_layout()
+
 # %%
 # .. seealso::
 #
 #   For more information about the motivation and usage of
-#   :class:`skore.EstimatorReport`, see :ref:`example_estimator_report`.
+#   :class:`skore.EstimatorReport`, see :ref:`example_estimator_report` for evaluation
+#   and :ref:`example_feature_importance` for inspection.
 
 
 # %%