Added Type-hinting to methods + Update to Numpy v2

.github/workflows/unit_tests.yml

@@ -22,46 +22,46 @@ jobs:
             os: windows-latest
         python-version: [3.9, "3.10", "3.11", "3.12"]
     steps:
-    - uses: actions/checkout@master
+      - uses: actions/checkout@master
 
-    - name: Get latest CMake and Ninja
-      uses: lukka/get-cmake@latest
-      with:
-        cmakeVersion: latest
-        ninjaVersion: latest
+      - name: Get latest CMake and Ninja
+        uses: lukka/get-cmake@latest
+        with:
+          cmakeVersion: latest
+          ninjaVersion: latest
 
-    - name: Install LIBOMP on Macos runners
-      if: runner.os == 'macOS'
-      run: |
-        brew install libomp
+      - name: Install LIBOMP on Macos runners
+        if: runner.os == 'macOS'
+        run: |
+          brew install libomp
 
-    - name: Setup Python
-      uses: actions/setup-python@master
-      with:
-        python-version: ${{ matrix.python-version }}
+      - name: Setup Python
+        uses: actions/setup-python@master
+        with:
+          python-version: ${{ matrix.python-version }}
 
-    - name: Install Python dependencies
-      run: |
-        pip3 install --upgrade setuptools pip
-        pip3 install ".[all]"
+      - name: Install Python dependencies
+        run: |
+          pip3 install --upgrade setuptools pip
+          pip3 install ".[all]"
 
-    - name: Run linters
-      run: |
-        pre-commit install
-        pre-commit run --all-files
+      - name: Run linters
+        run: |
+          pre-commit install
+          pre-commit run --all-files
 
-    - name: Run (unit) tests
-      env:
-        TEST_NOTEBOOKS: 0
-      run: |
-        pytest --cov=probatus/binning --cov=probatus/metric_volatility --cov=probatus/missing_values --cov=probatus/sample_similarity --cov=probatus/stat_tests --cov=probatus/utils --cov=probatus/interpret/ --ignore==tests/interpret/test_inspector.py --cov-report=xml
-        pyflakes probatus
+      - name: Run (unit) tests
+        env:
+          TEST_NOTEBOOKS: 0
+        run: |
+          pytest --save-plots true
+          pyflakes probatus
 
-    - name: Upload coverage to Codecov
-      if: github.ref == 'refs/heads/main'
-      uses: codecov/codecov-action@v1
-      with:
-        token: ${{ secrets.CODECOV_TOKEN }}
-        file: ./coverage.xml
-        flags: unittests
-        fail_ci_if_error: false
+      - name: Upload coverage to Codecov
+        if: github.ref == 'refs/heads/main'
+        uses: codecov/codecov-action@v1
+        with:
+          token: ${{ secrets.CODECOV_TOKEN }}
+          file: ./coverage.xml
+          flags: unittests
+          fail_ci_if_error: false

.gitignore

@@ -4,7 +4,7 @@ notebooks/explore
 .idea/
 docs/source/.ipynb_checkpoints/
 notebooks/.ipynb_checkpoints/
-
+tests/**/*.png
 
 # Created by https://www.gitignore.io/api/macos,python,pycharm,jupyternotebooks,visualstudiocode
 # Edit at https://www.gitignore.io/?templates=macos,python,pycharm,jupyternotebooks,visualstudiocode

docs/api/core.md

@@ -0,0 +1,53 @@
+# Core Components
+
+The core module provides fundamental building blocks and interfaces used throughout the probatus package.
+
+## Overview
+
+The core module establishes the architectural foundation for probatus components through abstract base classes and custom exceptions. This module defines consistent interfaces that ensure all probatus components follow similar patterns for fitting, computing results, and visualization.
+
+### Key Components
+
+#### Abstract Base Classes
+
+##### BaseFitComputeClass
+- Defines the standard interface for components that need to be fitted to data and compute results
+- Implements a consistent pattern with three essential methods:
+  - `fit()`: Trains the component on input data
+  - `compute()`: Calculates results based on fitted state
+  - `fit_compute()`: Combines fitting and computation in a single step
+- Provides error checking to ensure methods are called in the correct sequence
+
+##### BaseFitComputePlotClass
+- Extends BaseFitComputeClass with visualization capabilities
+- Adds the required `plot()` method for result visualization
+- Used as the base class for components that need to generate visual insights
+- Ensures consistent interface for all components with plotting functionality
+
+#### Exceptions
+
+##### NotFittedError
+- Custom exception raised when a method requiring a fitted state is called prematurely
+- Provides clear error messages indicating which operation failed
+- Helps prevent incorrect usage of probatus components
+
+### Core Design Principles
+
+The core module implements several key design principles that are reflected throughout the probatus package:
+
+1. **Consistent Interfaces**: All analytical components share a common interface pattern, making the library easier to learn and use.
+
+2. **Method Chaining**: Base classes support method chaining (returning `self` from fit methods) for more concise code.
+
+3. **Explicit State Validation**: Components explicitly check their fitted state before performing operations that require prior fitting.
+
+4. **Separation of Concerns**: Clear separation between data fitting, result computation, and visualization.
+
+5. **Extensibility**: Abstract base classes make it straightforward to add new components while maintaining consistent behavior.
+
+These core components serve as the foundation for more specialized classes throughout probatus, ensuring that all parts of the library work together seamlessly.
+
+## Implementation
+
+::: probatus.core.base
+::: probatus.core.exceptions

docs/api/dataset.md

@@ -0,0 +1,91 @@
+# Dataset
+
+The goal of dataset module is understanding how different two samples are from a multivariate perspective.
+
+One of the ways to indicate this is Resemblance Model. Having two datasets -  say X1 and X2 - one can analyse how easy it is to recognize which dataset a randomly selected row comes from. The Resemblance model assigns label 0 to the dataset X1, and label 1 to X2 and trains a binary classification model to predict which sample a given row comes from.
+By looking at the test AUC, one can conclude that the samples have a different distribution if the AUC is significantly higher than 0.5. Furthermore, by analysing feature importance one can understand which of the features have predictive power.
+
+<img src="../img/resemblance_model_schema.png"/>
+
+## Overview
+
+The sample similarity module provides sophisticated tools for analyzing and comparing multivariate datasets using machine learning approaches. This module is particularly valuable for:
+
+- Detecting distribution shifts between datasets
+- Identifying features that contribute to dataset differences
+- Understanding data drift in production environments
+- Validating data quality and consistency
+- Supporting data monitoring and maintenance
+
+### Key Components
+
+#### BaseResemblanceModel
+- Abstract base class that provides the foundation for all resemblance models
+- Handles data preparation, model training, and evaluation
+- Supports both standalone estimators and sklearn Pipelines
+- Features:
+  - Flexible scoring metrics
+  - Train/test split configuration
+  - Comprehensive performance reporting
+  - Parallel processing support
+
+#### SHAPImportanceResemblance (Recommended)
+- Leverages SHAP (SHapley Additive exPlanations) for feature importance analysis
+- Key features:
+  - Tree-based model interpretation
+  - Detailed feature importance analysis
+  - Support for complex feature interactions
+  - Multiple visualization options (bar, dot, violin plots)
+  - Advanced SHAP configuration options
+
+#### PermutationImportanceResemblance
+- Uses permutation feature importance for analysis
+- Key features:
+  - Direct measurement of feature impact on model performance
+  - Configurable number of permutation iterations
+  - Simple and interpretable results
+  - Robust to feature interactions
+  - Efficient computation with parallel processing
+
+### Features
+- Multiple analysis approaches:
+  - SHAP-based importance (recommended for detailed analysis)
+  - Permutation-based importance (faster for initial screening)
+- Comprehensive model support:
+  - Binary classification models
+  - Tree-based algorithms
+  - Custom model integration
+  - Pipeline compatibility
+- Advanced analysis capabilities:
+  - Feature importance ranking
+  - Distribution shift detection
+  - Feature interaction analysis
+- Detailed reporting and visualization:
+  - Feature importance plots
+  - Performance metrics
+  - Distribution comparisons
+
+### Use Cases
+- Data drift detection
+- Feature importance analysis
+- Dataset comparison
+- Quality assurance
+- Production monitoring
+- Data validation
+
+The module is designed to provide robust tools for understanding and comparing multivariate datasets, with a focus on interpretability and practical application.
+
+The following features are implemented:
+
+- [BaseResemblanceModel][probatus.dataset.resemblance_modeler.BaseResemblanceModel]:
+  Abstract base class that provides core functionality for all resemblance models. Handles data preparation, model training, and performance evaluation.
+
+- [SHAPImportanceResemblance (Recommended)][probatus.dataset.resemblance_modeler.SHAPImportanceResemblance]:
+  The class applies SHAP library to interpret tree-based resemblance models. Features multiple visualization options and detailed feature importance analysis.
+
+- [PermutationImportanceResemblance][probatus.dataset.resemblance_modeler.PermutationImportanceResemblance]:
+  The class applies permutation feature importance to understand which features the model relies on most. The higher the importance of the feature, the more a given feature possibly differs in X2 compared to X1. The importance indicates how much the test performance drops if a given feature is permuted.
+
+## Implementation
+
+::: probatus.dataset.resemblance_modeler

docs/api/features.md

@@ -0,0 +1,80 @@
+# Features Elimination
+
+This module focuses on feature elimination and it contains the following:
+
+- [ShapRFECV][probatus.feature_elimination.feature_elimination.ShapRFECV]: Perform Backwards Recursive Feature Elimination, using SHAP feature importance. It supports binary classification, multi-class classification, regression models and hyperparameter optimization at every feature elimination step. For XGBoost, LightGBM and CatBoost it supports early stopping of the model fitting process. It can be an alternative regularization technique to hyperparameter optimization of the number of base trees in gradient boosted tree models. Particularly useful when dealing with large datasets.
+
+## Overview
+
+The feature elimination module provides sophisticated tools for selecting the most important features in machine learning models using SHAP-based importance metrics. This module is particularly valuable for:
+
+- Reducing model complexity while maintaining performance
+- Identifying the most impactful features
+- Optimizing model training time
+- Improving model interpretability
+
+### Key Components
+
+#### ShapRFECV (Recursive Feature Elimination with Cross-Validation)
+- Implements backward feature elimination using SHAP importance
+- Key features:
+  - Supports both classification and regression tasks
+  - Integrates with hyperparameter optimization via SearchCV, RandomizedSearchCV, or BayesSearchCV, etc...
+  - Provides cross-validation at each elimination step
+  - Generates detailed feature selection reports
+  - Supports custom feature selection strategies
+  - Includes advanced methods to select optimal feature subsets:
+    - Best performance-based selection
+    - Coherent performance with minimal variance
+    - Parsimonious selection (fewest features within a performance threshold)
+  - Supports sklearn Pipeline objects for seamless preprocessing integration
+
+#### Early Stopping Support
+- Directly integrated into ShapRFECV (no separate class needed)
+- Optimized for popular gradient boosting frameworks:
+  - LightGBM
+  - XGBoost
+  - CatBoost
+- Benefits:
+  - Faster training process
+  - Prevents overfitting
+  - Reduces computational resources
+  - Maintains model performance
+
+### Features
+- Flexible feature elimination strategies:
+  - Fixed number of features per step
+  - Percentage-based elimination (adaptive step sizes)
+  - Custom elimination criteria
+- Comprehensive model support:
+  - Binary classification
+  - Multi-class classification
+  - Regression models
+- Advanced optimization capabilities:
+  - Hyperparameter tuning integration
+  - Cross-validation at each step
+  - Early stopping for gradient boosted models
+  - Support for sample weighting
+  - Optional SHAP variance penalization for more stable feature selection
+- Detailed reporting and visualization:
+  - Feature importance tracking
+  - Performance metrics across steps
+  - Selection process visualization
+  - Error bars showing model stability
+
+### Use Cases
+- Large-scale feature selection
+- Model optimization
+- Feature importance analysis
+- Computational resource optimization
+- Model interpretability enhancement
+- Production model preparation
+
+The module is designed to provide a robust and efficient approach to feature selection using RFECV in combination with SHAP, with a focus on practical application in real-world scenarios.
+
+## Implementation
+
+::: probatus.features.shap_recursive_feature_elimination
+::: probatus.features.shap_recursive_feature_elimination_helper
+::: probatus.features.shap_early_stopping_recursive_feature_elimination
+::: probatus.features.shap_early_stopping_recursive_feature_elimination_helper