Merge remote-tracking branch 'upstream/HEAD' into doc-clarity-improvements

Author: rcap107

CHANGES.rst

@@ -31,6 +31,15 @@ New Features
   :meth:`DataOp.skb.eval`, :meth:`SkrubLearner.predict`, etc., or in
   :meth:`DataOp.skb.find` or :meth:`SkrubLearner.truncated_after`. :pr:`2062` by
   :user:`Jérôme Dockès <jeromedockes>`.
+- The :class:`SessionEncoder` is now available. This encoder adds a `session_id`
+  column, which groups together events that occur within the given session gap.
+  Additionally, it is possible to provide a ``split_by`` column or list of columns
+  (e.g., user ID or (user ID, user device)) to compute sessions for each grouping
+  value.
+  :pr:`1930` by  :user:`Riccardo Cappuzzo <rcap107>`.
+-  A new synthetic dataset generator for timestamped data and session-based
+  operations has been added: :meth:`~skrub.datasets.make_retail_events`.
+  :pr:`1930` by  :user:`Riccardo Cappuzzo <rcap107>`.
 - The :class:`DropSimilar` transformer has been added, for removing columns in a
   dataframe that present high correlation with other columns. :pr:`2023` by
   :user:`Eloi Massoulié <emassoulie>`.
@@ -65,6 +74,19 @@ Changes
   :pr:`2048` by :user:`Riccardo Cappuzzo <rcap107>`.
 - The minimum required version of matplotlib has been increased from 3.4.3 to 3.6.1.
   :pr:`2159` by :user:`Riccardo Cappuzzo <rcap107>`.
+- :meth:`SkrubLearner.score` has been enhanced when the DataOp used
+  :meth:`DataOp.skb.with_scoring`. During scoring, predict(), predict_proba()
+  etc. are cached to avoid recomputation when multiple scorers are used (or one
+  scorer calls them several times). Moreover it is possible to pass
+  ``return_predictions=True`` to also retrieve any predictions that have been
+  computed during scoring, in addition to the scores. Finally, in cases where we
+  already have the predictions but want the result of score() without
+  recomputing them, it is possible to provide them in the environment passed to
+  ``score({..., "_skrub_predictions": {"predict_proba": ...}})``.
+  :pr:`2195` by :user:`Jérôme Dockès <jeromedockes>`.
+- :meth:`SkrubLearner.find_fitted_estimator` now supports searching for the
+  apply node by ID or callable predicate as alternatives to the node name.
+  :pr:`2194` by :user:`Jérôme Dockès <jeromedockes>`.
 
 Bugfixes
 --------

doc/_templates/index.html

@@ -152,7 +152,7 @@ <h5 class="feature-title">Inspect it, apply it to new data</h5>
       </div>
       <p class="feature-text-see-also">
         <!-- name of the userguide file on DataOps -->
-        <a href="{{ pathto('auto_tutorials/1110_data_ops_intro') }}">Discover the skrub DataOps →</a>
+        <a href="{{ pathto('data_ops') }}">Discover the skrub DataOps →</a>
       </p>
     </div>
     <div class="col-md-7">

doc/api_reference.py

@@ -90,6 +90,7 @@
                     "SimilarityEncoder",
                     "ToCategorical",
                     "DatetimeEncoder",
+                    "SessionEncoder",
                     "ToDatetime",
                     "ToFloat",
                 ],
@@ -341,6 +342,9 @@
                     "datasets.get_data_dir",
                     "datasets.make_deduplication_data",
                     "datasets.toy_orders",
+                    "datasets.toy_products",
+                    "datasets.toy_cities",
+                    "datasets.make_retail_events",
                 ],
             }
         ],

doc/data_ops.rst

@@ -56,6 +56,18 @@ they can be saved in a file, loaded, applied to new data as easily as a single
   etc. . Skrub DataOps remove those limitations and add several useful features
   such as interactive previews and integration with Optuna.
 
+A quick overview of DataOps
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This tutorial walks through the main components of the DataOps on a simple
+example:
+
+.. toctree::
+   :maxdepth: 1
+
+   auto_tutorials/1111_data_ops_quick_tour
+
+
 Data Ops basic concepts
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -64,7 +76,6 @@ Data Ops basic concepts
 
    modules/data_ops/basics/what_are_data_ops
    modules/data_ops/basics/building_data_ops_plan
-   auto_tutorials/1110_data_ops_intro
    modules/data_ops/basics/using_previews
    modules/data_ops/basics/direct_access_methods
    modules/data_ops/basics/control_flow

doc/data_ops_report.py

@@ -74,18 +74,16 @@ def create_employee_salaries_report():
     if (output_dir / "index.html").exists():
         return output_dir
 
-    dataset = skrub.datasets.fetch_employee_salaries(split="train").employee_salaries
-    data_var = skrub.var("data", dataset)
-    X = data_var.drop("current_annual_salary", axis=1).skb.mark_as_X()
-    y = data_var["current_annual_salary"].skb.mark_as_y()
-
-    vectorizer = TableVectorizer()
-    X_vec = X.skb.apply(vectorizer)
+    pred = (
+        skrub.var("employee_data")
+        .skb.apply(TableVectorizer())
+        .skb.apply(HistGradientBoostingRegressor(), y=skrub.var("salary"))
+    )
 
-    hgb = HistGradientBoostingRegressor()
-    predictor = X_vec.skb.apply(hgb, y=y)
+    dataset = skrub.datasets.fetch_employee_salaries(split="train")
 
-    predictor.skb.full_report(
+    pred.skb.full_report(
+        {"employee_data": dataset.X, "salary": dataset.y},
         output_dir=output_dir,
         overwrite=True,
         open=False,

doc/modules/data_ops/basics/building_data_ops_plan.rst

@@ -90,5 +90,5 @@ By working only on Data Ops we ensure that all the operations done on the data
 are added correctly to the computational graph, which then allows the resulting
 learner to execute all steps as intended.
 
-See :ref:`sphx_glr_auto_tutorials_1110_data_ops_intro.py` for an introductory
+See :ref:`sphx_glr_auto_tutorials_1111_data_ops_quick_tour.py` for an introductory
 example on how to use skrub DataOps on a single dataframe.

doc/modules/data_ops/basics/control_flow.rst

@@ -168,7 +168,7 @@ Finally, there are other situations where using :func:`deferred` can be helpful:
 
 .. rubric:: Examples
 
-- See :ref:`sphx_glr_auto_tutorials_1110_data_ops_intro.py` for an introductory
+- See :ref:`sphx_glr_auto_examples_data_ops_1111_data_ops_quick_tour.py` for an introductory
   example on how to use skrub DataOps on a single dataframe.
 - See :ref:`sphx_glr_auto_examples_02_data_ops_1120_multiple_tables.py` for an example
   of how skrub DataOps can be used to process multiple tables using dataframe APIs.

doc/modules/data_ops/validation/tuning_validating_data_ops.rst

@@ -256,11 +256,75 @@ SkrubLearner(data_op=<Scoring <Apply DummyClassifier> (1 scorers)>
 Note that the score above is negative: it is the negative log loss we passed to
 ``with_scoring``, and not the default score (accuracy, which would be positive).
 
+If we also want to recover the default score that would be returned by the
+applied estimator's ``score()`` method (what we would get if we did not use
+:meth:`DataOp.skb.with_scoring`), we can pass ``None`` as the scorer, and the
+default corresponding key in the result is ``"score"`` (exactly like in
+:func:`~sklearn.model_selection.cross_validate`):
+
+>>> (
+...     pred.skb.with_scoring(None)
+...     .skb.with_scoring('accuracy')
+...     .skb.with_scoring('roc_auc')
+...     .skb.make_learner()
+...     .fit(split["train"])
+...     .score(split["test"])
+... )
+{'score': 0.6666666666666666, 'accuracy': 0.6666666666666666, 'roc_auc': 0.5}
+
 :meth:`DataOp.skb.with_scoring` only changes how scoring is performed
 (the outputs of :meth:`DataOp.skb.cross_validate`,
 :meth:`DataOp.skb.make_randomized_search`, :class:`SkrubLearner.score <SkrubLearner>` etc.),
-**not** the actual outputs of the learner (it does _not_ affect the outputs of
+**not** the actual outputs of the learner (it does *not* affect the outputs of
 :meth:`DataOp.skb.eval`, :class:`SkrubLearner.predict <SkrubLearner>`, etc.)
 
 This method can be called several times to add scorers that take different
 kwargs. See the reference documentation for details.
+
+Avoiding computing predictions multiple times
+=============================================
+
+This section gives a few tips to avoid recomputing predictions, which is
+particularly important for pipelines for which inference is expensive, such as
+those using the :class:`TextEncoder` or Tabular Foundation Models such as
+`TabICL <https://tabicl.readthedocs.io/en/latest/>`_.
+
+When :meth:`DataOp.skb.with_scoring` is used, predictions are cached during
+scoring so that if multiple scorers call the same function (e.g. ``predict()``
+or ``predict_proba()``) the computation runs only once. Moreover, if we also
+need the predictions in addition to the scores (for example to create plots,
+study calibration, etc.) we can pass ``return_predictions=True`` to the
+learner's ``score()`` method.
+
+Instead of doing the following, which computes the predictions twice:
+
+>>> scores = learner.score(split['test'])
+>>> predictions = learner.predict(split['test']) # used for plotting etc.
+
+We can write:
+
+>>> scores, predictions = learner.score(split['test'], return_predictions=True)
+>>> scores  # doctest: +SKIP
+{'neg_log_loss': -0.6365141682948128}
+>>> predictions
+{'predict_proba': array([[0.66666667, 0.33333333],
+       [0.66666667, 0.33333333],
+       [0.66666667, 0.33333333]])}
+
+The returned dictionary contains any predictions of the pipeline that have been
+computed as part of scoring. When :meth:`DataOp.skb.with_scoring` has not been
+used, it will always be empty.
+
+If we happen to already have the predictions for the data we are scoring on, but
+would like to use the learner's ``score`` method to get the scores, we can
+inject the precomputed predictions in the input environment, under the special
+key ``"_skrub_predictions"``:
+
+>>> predict_proba = learner.predict_proba(split['test'])
+>>> learner.score(split['test'] | {'_skrub_predictions': {'predict_proba': predict_proba}})
+{'neg_log_loss': -0.63...}
+
+With the above, ``predict()`` is not called during scoring. As for
+``return_predictions``, this only applies to scorers added with
+:meth:`DataOp.skb.with_scoring`. If no scorer has been configured
+``'_skrub_predictions'`` will be ignored.

doc/modules/multi_column_operations/sessionization.rst

@@ -0,0 +1,106 @@
+.. _sessionization:
+
+.. |SessionEncoder| replace:: :class:`~skrub.SessionEncoder`
+.. |BaseEstimator| replace:: :class:`~sklearn.base.BaseEstimator`
+.. |TransformerMixin| replace:: :class:`~sklearn.base.TransformerMixin`
+
+
+Detecting sessions in timestamped data with the SessionEncoder
+----------------------------------------------------------------
+
+When dealing with timestamped data (data that includes at least a timestamp column),
+it may be beneficial to try and identify groups of events as
+`"sessions" <https://en.wikipedia.org/wiki/Session_(web_analytics)>`__,
+through **sessionization**.
+
+Sessionization is the process of grouping a sequence of events (like user
+interactions) into meaningful sessions.
+For example, in an online retail context you might define a new session whenever
+more than 30 minutes pass with no activity from the user. On a website, a session may
+define a sequence of requests made by a single end-user within a certain time duration.
+
+While definitions may vary depending on the specific use case, being able to detect
+such "bursts" of activity by a user can often help with building features that have
+greater predictive power than raw individual events, such as number of sessions or
+average session duration.
+
+The |SessionEncoder| addresses this problem by detecting sessions based on
+a timestamp column, other session-related columns (e.g., user and device) that should be
+used to distinguish between sessions, and a ``session_gap``. Session-related columns
+-- identified by the ``split_by`` parameter -- allow to split sessions based on
+the provided parameters, for example to group user actions only if they were conducted
+on the same device.
+
+A session is then defined as a sequence of events that share the same value in the
+``split_by`` columns, and whose events are closer to each other than the
+``session_gap``.
+
+>>> from skrub import SessionEncoder
+>>> from skrub.datasets import make_retail_events
+>>> events = make_retail_events(n_events=100, random_state=0)
+>>> X, y = events.X, events.y
+
+Once the necessary features are provided, the |SessionEncoder|
+returns a dataframe that includes a ``timestamp_session_id`` column, which is
+composed of a distinct integer ID for each session:
+
+>>> se = SessionEncoder(timestamp_col="timestamp", split_by="user_id", session_gap=30 * 60)
+>>> res = se.fit_transform(X)
+>>> res.head(5) # doctest: +SKIP
+     user_id                        timestamp device_type page_category event_type  time_on_page  price_viewed  timestamp_session_id
+0  user_0164 2024-01-01 03:29:07.708922+00:00      mobile       fashion  page_view         134.1        309.80                    59
+1  user_0164 2024-01-01 03:29:42.185048+00:00      tablet         books     search         103.4         11.00                    59
+2  user_0164 2024-01-01 03:32:38.352703+00:00     desktop          home   wishlist         180.3          4.80                    59
+3  user_0008 2024-01-02 10:49:56.974375+00:00      mobile         books  page_view           7.0         33.94                     2
+4  user_0149 2024-01-04 10:00:15.882835+00:00     desktop   electronics  page_view         108.5          4.44                    49
+
+With the session ID, it becomes possible to compute aggregations on
+each session, for example to find the duration or number of sessions
+by a user.
+
+.. warning::
+
+Caution! Aggregation can introduce data leakage. Records should only be aggregated from
+within the training set at training time, and the test set at predict time. To
+ensure this is the case, any code that performs aggregation can be wrapped in a
+scikit-learn |BaseEstimator| (as shown in the
+:ref:`SessionEncoder example <sphx_glr_auto_examples_0110_session_encoder.py>`),
+otherwise the pipeline should use the skrub :ref:`Data Ops framework<user_guide_data_ops_plan>`.
+
+The |SessionEncoder| includes the ``suffix`` parameter (by default
+``suffix="session_id"``) to specify what the name of the new column should be.
+This can help with creating multiple session IDs based on the same timestamp.
+For example, we might want to create sessions based on users, and based on users
+and their device:
+
+>>> se = SessionEncoder(timestamp_col="timestamp",
+... split_by="user_id",
+... session_gap=30 * 60,
+... suffix="user"
+... )
+>>> res = se.fit_transform(X)
+>>> res.head(5) # doctest: +SKIP
+     user_id                        timestamp  ... price_viewed timestamp_user
+0  user_0164 2024-01-01 03:29:07.708922+00:00  ...       309.80             59
+1  user_0164 2024-01-01 03:29:42.185048+00:00  ...        11.00             59
+2  user_0164 2024-01-01 03:32:38.352703+00:00  ...         4.80             59
+3  user_0008 2024-01-02 10:49:56.974375+00:00  ...        33.94              2
+4  user_0149 2024-01-04 10:00:15.882835+00:00  ...         4.44             49
+
+>>> se = SessionEncoder(timestamp_col="timestamp",
+... split_by=["user_id", "device_type"],
+... session_gap=30 * 60,
+... suffix="user_device"
+... )
+>>> res = se.fit_transform(X)
+>>> res.head(5) # doctest: +SKIP
+     user_id                        timestamp  ... price_viewed timestamp_user_device
+0  user_0164 2024-01-01 03:29:07.708922+00:00  ...       309.80                    75
+1  user_0164 2024-01-01 03:29:42.185048+00:00  ...        11.00                    76
+2  user_0164 2024-01-01 03:32:38.352703+00:00  ...         4.80                    74
+3  user_0008 2024-01-02 10:49:56.974375+00:00  ...        33.94                     2
+4  user_0149 2024-01-04 10:00:15.882835+00:00  ...         4.44                    59
+
+The |SessionEncoder| has additional features that are detailed in the relevant docstring.
+You can also find a working :ref:`example <sphx_glr_auto_examples_0110_session_encoder.py>`
+in the gallery.

doc/multi_column_operations.rst

@@ -15,3 +15,4 @@ multiple columns.
    modules/multi_column_operations/selectors
    modules/multi_column_operations/type_of_selectors
    modules/multi_column_operations/advanced_selectors
+   modules/multi_column_operations/sessionization