From 1615550d14e00be2232266b14e816f676bf49109 Mon Sep 17 00:00:00 2001 From: Riccardo Cappuzzo Date: Thu, 2 Jul 2026 15:23:53 +0200 Subject: [PATCH 1/4] MAINT - preparing changelog for 0.10 --- CHANGES.rst | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/CHANGES.rst b/CHANGES.rst index 631e87319..faede2b66 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -13,7 +13,7 @@ New Features ------------ - New methods :meth:`SkrubLearner.get_named_params` and :meth:`SkrubLearner.set_named_params` allow getting and setting the outcomes for - choices contained in the DataOp, keyed by choice name. It provides a more + choices contained in the DataOp, keyed by choice name. They provide a more robust way of transferring selected hyperparameters from one DataOp to a different one than :meth:`SkrubLearner.get_params` and :meth:`SkrubLearner.set_params`. @@ -26,12 +26,13 @@ New Features - It is now possible to attach new preview values to the variables in a DataOp with :meth:`DataOp.skb.set_data`. :pr:`2081` by :user:`Jérôme Dockès `. -- :class:`DataOp` objects have a new attribute :attr:`DataOp.skb.id` which - provides an alternative for referring to a node, in the environment passed to +- :class:`DataOp` objects have a new attribute :attr:`DataOp.skb.id`. + :attr:`DataOp.skb.id` + provides an alternative for referring to a node in the environment passed to :meth:`DataOp.skb.eval`, :meth:`SkrubLearner.predict`, etc., or in :meth:`DataOp.skb.find` or :meth:`SkrubLearner.truncated_after`. :pr:`2062` by :user:`Jérôme Dockès `. -- The :class:`SessionEncoder` is now available. This encoder adds a `session_id` +- The :class:`SessionEncoder` is now available. This encoder adds a ``session_id`` column, which groups together events that occur within the given session gap. Additionally, it is possible to provide a ``split_by`` column or list of columns (e.g., user ID or (user ID, user device)) to compute sessions for each grouping @@ -40,8 +41,8 @@ New Features - A new synthetic dataset generator for timestamped data and session-based operations has been added: :meth:`~skrub.datasets.make_retail_events`. :pr:`1930` by :user:`Riccardo Cappuzzo `. -- The :class:`DropSimilar` transformer has been added, for removing columns in a - dataframe that present high correlation with other columns. :pr:`2023` by +- The :class:`DropSimilar` transformer has been added, for removing columns that + present high correlation with other columns in a dataframe . :pr:`2023` by :user:`Eloi Massoulié `. - :class:`ToFloat32` now allows users to specify ``decimal`` and ``thousand`` separators to parse numerical columns that use formatting different from the default @@ -51,26 +52,29 @@ New Features Gómez Jiménez `. - :meth:`TableReport.json` now includes histogram data for numeric and datetime columns (the bin count and edges, and numbers of low and high outliers). Now - ``json()`` contains all the information shown in the report html rendering, - including the plots. :pr:`2164` by :user:`Jérôme Dockès `. + ``json()`` contains all the information shown in the report HTML rendering, + including the plots. The schema of the generated JSON is available at :ref:``. + :pr:`2164` by :user:`Jérôme Dockès `. - Added :func:`skrub.selectors.object` to select columns with the ``object`` (pandas) or ``pl.Object`` (polars) dtype. :pr:`2171` by :user:`Omkar Kabde `. Changes ------- -- Grouped Examples into subject-specific sections. :pr:`2102` by +- Gallery examples have been grouped into subject-specific sections. :pr:`2102` by :user:`Maureen Githaiga `. -- :meth:`choose_from` now transparently converts `outcomes` to a list when it is +- :meth:`choose_from` now transparently converts ``outcomes`` to a list when it is another type of sequence. :pr:`2100` by :user:`aidbar `. - An unnecessary warning that was raised when passing a numpy array to the TableVectorizer has been removed. :pr:`1908` by :user:`Sandrine Henry `. -- Improving the association tab error message when only one column is present +- The error message displayed in the Associations tab of the :class:`TableReport` + has been improved for reports where only one column is present. :pr:`2094` by :user:`Alicja Kosak `. - Added support for numpy arrays in :meth:`DataOp.skb.concat`. :pr:`2096` by :user:`Ayesha Siddiqua `. -- The :class:`TableReport` can now be exported in markdown format with ``.markdown``. +- The :class:`TableReport` can now be exported in markdown format with + func:`~skrub.TableReport.markdown`. :pr:`2048` by :user:`Riccardo Cappuzzo `. - The minimum required version of matplotlib has been increased from 3.4.3 to 3.6.1. :pr:`2159` by :user:`Riccardo Cappuzzo `. From 2f0f1ee7e18a1dd34d30eafb529ea168c998a882 Mon Sep 17 00:00:00 2001 From: Riccardo Cappuzzo Date: Mon, 6 Jul 2026 13:15:47 +0200 Subject: [PATCH 2/4] sorting, splitting --- CHANGES.rst | 128 ++++++++++++++++++++++++++++------------------------ README.rst | 2 +- 2 files changed, 71 insertions(+), 59 deletions(-) diff --git a/CHANGES.rst b/CHANGES.rst index d9f966d7f..39caf0432 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -11,42 +11,14 @@ Ongoing development New Features ------------ -- :func:`set_config` and :func:`config_context` now accept a - ``table_report_n_rows`` parameter to globally control the default number of - rows displayed in :class:`TableReport`. - :pr:`2193` by :user:`Mann `. -- :meth:`TableReport.dict` now allows exporting the report data as a Python - dictionary. :pr:`2188` by :user:`m4nn2609-dot `. -- New methods :meth:`SkrubLearner.get_named_params` and - :meth:`SkrubLearner.set_named_params` allow getting and setting the outcomes for - choices contained in the DataOp, keyed by choice name. They provide a more - robust way of transferring selected hyperparameters from one DataOp to a - different one than :meth:`SkrubLearner.get_params` and - :meth:`SkrubLearner.set_params`. - :pr:`2090` by :user:`Jérôme Dockès `. -- A parameter ``becomes_default`` has been added to :func:`var`. It allows - indicating that the provided preview ``value`` should also be treated as a - default value for this variable in all contexts (for example in a - SkrubLearner's method like ``fit`` or ``predict``). - :pr:`2082` by :user:`Jérôme Dockès `. -- It is now possible to attach new preview values to the variables in a DataOp - with :meth:`DataOp.skb.set_data`. :pr:`2081` by - :user:`Jérôme Dockès `. -- :class:`DataOp` objects have a new attribute :attr:`DataOp.skb.id`. - :attr:`DataOp.skb.id` - provides an alternative for referring to a node in the environment passed to - :meth:`DataOp.skb.eval`, :meth:`SkrubLearner.predict`, etc., or in - :meth:`DataOp.skb.find` or :meth:`SkrubLearner.truncated_after`. :pr:`2062` by - :user:`Jérôme Dockès `. +**Transformers**: + - The :class:`SessionEncoder` is now available. This encoder adds a ``session_id`` column, which groups together events that occur within the given session gap. Additionally, it is possible to provide a ``split_by`` column or list of columns (e.g., user ID or (user ID, user device)) to compute sessions for each grouping value. :pr:`1930` by :user:`Riccardo Cappuzzo `. -- A new synthetic dataset generator for timestamped data and session-based - operations has been added: :meth:`~skrub.datasets.make_retail_events`. - :pr:`1930` by :user:`Riccardo Cappuzzo `. - The :class:`DropSimilar` transformer has been added, for removing columns that present high correlation with other columns in a dataframe . :pr:`2023` by :user:`Eloi Massoulié `. @@ -56,39 +28,65 @@ New Features Additionally, negative numbers indicated with parentheses can be converted to the regular numeric format (``(432)`` becomes ``-432``). :pr:`1772` by :user:`Gabriela Gómez Jiménez `. +- The transformer :class:`DurationToFloat` has been added. :pr:`2069` by + :user:`Riccardo Cappuzzo `. +**TableReport**: + - :meth:`TableReport.json` now includes histogram data for numeric and datetime columns (the bin count and edges, and numbers of low and high outliers). Now ``json()`` contains all the information shown in the report HTML rendering, including the plots. The schema of the generated JSON is available at :ref:``. :pr:`2164` by :user:`Jérôme Dockès `. -- Added :func:`skrub.selectors.object` to select columns with the ``object`` - (pandas) or ``pl.Object`` (polars) dtype. :pr:`2171` by :user:`Omkar Kabde - `. +- The :class:`TableReport` can now be exported in markdown format with + :func:`~skrub.TableReport.markdown`. + :pr:`2048` by :user:`Riccardo Cappuzzo `. +- :meth:`TableReport.dict` now allows exporting the report data as a Python + dictionary. :pr:`2188` by :user:`m4nn2609-dot `. + +**Data Ops**: + - :meth:`ParamSearch.plot_results` and :meth:`OptunaParamSearch.plot_results` accept new parameters ``show_scores``, ``show_choices``, and ``show_times`` to control respectively which scores, choices (params), and times (fit or score durations) should be included in the figure. :pr:`2202` by :user:`Jérôme Dockès `. +- New methods :meth:`SkrubLearner.get_named_params` and + :meth:`SkrubLearner.set_named_params` allow getting and setting the outcomes for + choices contained in the DataOp, keyed by choice name. They provide a more + robust way of transferring selected hyperparameters from one DataOp to a + different one than :meth:`SkrubLearner.get_params` and + :meth:`SkrubLearner.set_params`. + :pr:`2090` by :user:`Jérôme Dockès `. +- A parameter ``becomes_default`` has been added to :func:`var`. It allows + indicating that the provided preview ``value`` should also be treated as a + default value for this variable in all contexts (for example in a + SkrubLearner's method like ``fit`` or ``predict``). + :pr:`2082` by :user:`Jérôme Dockès `. +- It is now possible to attach new preview values to the variables in a DataOp + with :meth:`DataOp.skb.set_data`. :pr:`2081` by + :user:`Jérôme Dockès `. +- :class:`DataOp` objects have a new attribute :attr:`DataOp.skb.id`. + :attr:`DataOp.skb.id` + provides an alternative for referring to a node in the environment passed to + :meth:`DataOp.skb.eval`, :meth:`SkrubLearner.predict`, etc., or in + :meth:`DataOp.skb.find` or :meth:`SkrubLearner.truncated_after`. :pr:`2062` by + :user:`Jérôme Dockès `. + +**Misc**: + +- A new synthetic dataset generator for timestamped data and session-based + operations has been added: :meth:`~skrub.datasets.make_retail_events`. + :pr:`1930` by :user:`Riccardo Cappuzzo `. +- Added the :func:`~skrub.selectors.object` selector to select columns with the + ``object`` (pandas) or ``pl.Object`` (polars) dtype. :pr:`2171` by + :user:`Omkar Kabde `. +- :func:`set_config` and :func:`config_context` now accept a + ``table_report_n_rows`` parameter to globally control the default number of + rows displayed in :class:`TableReport`. + :pr:`2193` by :user:`Mann `. Changes ------- -- Gallery examples have been grouped into subject-specific sections. :pr:`2102` by - :user:`Maureen Githaiga `. -- :meth:`choose_from` now transparently converts ``outcomes`` to a list when it is - another type of sequence. :pr:`2100` by :user:`aidbar `. -- An unnecessary warning that was raised when passing a numpy array to the - TableVectorizer has been removed. :pr:`1908` by - :user:`Sandrine Henry `. -- The error message displayed in the Associations tab of the :class:`TableReport` - has been improved for reports where only one column is present. - :pr:`2094` by :user:`Alicja Kosak `. -- Added support for numpy arrays in :meth:`DataOp.skb.concat`. - :pr:`2096` by :user:`Ayesha Siddiqua `. -- The :class:`TableReport` can now be exported in markdown format with - func:`~skrub.TableReport.markdown`. - :pr:`2048` by :user:`Riccardo Cappuzzo `. -- The minimum required version of matplotlib has been increased from 3.4.3 to 3.6.1. - :pr:`2159` by :user:`Riccardo Cappuzzo `. - :meth:`SkrubLearner.score` has been enhanced when the DataOp used :meth:`DataOp.skb.with_scoring`. During scoring, predict(), predict_proba() etc. are cached to avoid recomputation when multiple scorers are used (or one @@ -102,25 +100,39 @@ Changes - :meth:`SkrubLearner.find_fitted_estimator` now supports searching for the apply node by ID or callable predicate as alternatives to the node name. :pr:`2194` by :user:`Jérôme Dockès `. +- The minimum required version of matplotlib has been increased from 3.4.3 to 3.6.1. + :pr:`2159` by :user:`Riccardo Cappuzzo `. +- Gallery examples have been grouped into subject-specific sections. :pr:`2102` by + :user:`Maureen Githaiga `. +- :meth:`choose_from` now transparently converts ``outcomes`` to a list when it is + another type of sequence. :pr:`2100` by :user:`aidbar `. +- An unnecessary warning that was raised when passing a numpy array to the + TableVectorizer has been removed. :pr:`1908` by + :user:`Sandrine Henry `. +- The error message displayed in the Associations tab of the :class:`TableReport` + has been improved for reports where only one column is present. + :pr:`2094` by :user:`Alicja Kosak `. +- Added support for numpy arrays in :meth:`DataOp.skb.concat`. + :pr:`2096` by :user:`Ayesha Siddiqua `. Bugfixes -------- -- :class:`MinHashEncoder` with the default ``hashing="fast"`` now uses every - n-gram size in ``ngram_range`` (the upper bound is inclusive, as documented - and as already done by ``hashing="murmur"``). Previously the largest size was - dropped, so the default ``ngram_range=(2, 4)`` ignored 4-grams and a - single-size range such as ``(3, 3)`` produced the same constant encoding for - every string. :pr:`2168` by :user:`José Maia `. - A bug in how the :class:`TableVectorizer` and :class:`Cleaner` treated columns duration columns in pandas and polars has been fixed. Now, both classes convert durations to the total number of seconds (with fractional part). This is done by the new transformer :class:`DurationToFloat`. :pr:`2069` by :user:`Riccardo Cappuzzo `. -- An error that could arise when running ``TableReport`` on dataframes containing +- An error that could arise when running :class:`TableReport` on dataframes containing double dollar (``$$``) signs has been fixed. :pr:`2154` by :user:`Katerina Michenina `, :user:`CecilyTS `, :user:`Eve Rabin `. -- An error that happened when running ``TableReport`` or ``column_associations`` +- :class:`MinHashEncoder` with the default ``hashing="fast"`` now uses every + n-gram size in ``ngram_range`` (the upper bound is inclusive, as documented + and as already done by ``hashing="murmur"``). Previously the largest size was + dropped, so the default ``ngram_range=(2, 4)`` ignored 4-grams and a + single-size range such as ``(3, 3)`` produced the same constant encoding for + every string. :pr:`2168` by :user:`José Maia `. +- An error that happened when running :class:`TableReport` or ``column_associations`` on some dataframes with non-string column names has been fixed in :pr:`2179` by :user:`Jérôme Dockès `. - An error that could arise in histograms when running :class:`TableReport` on diff --git a/README.rst b/README.rst index f8322a91e..ad78a7788 100644 --- a/README.rst +++ b/README.rst @@ -17,7 +17,7 @@ skrub .. |black| image:: https://img.shields.io/badge/code%20style-black-000000.svg -**skrub** (formerly *dirty_cat*) is a Python +**skrub** is a Python library that facilitates machine learning with dataframes. If you like the package, spread the word and ⭐ this repository! From 8971109e980059a259ef805e4fd17f9207ce9170 Mon Sep 17 00:00:00 2001 From: Riccardo Cappuzzo Date: Mon, 6 Jul 2026 13:20:27 +0200 Subject: [PATCH 3/4] _ --- CHANGES.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGES.rst b/CHANGES.rst index 39caf0432..ba574899f 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -30,6 +30,7 @@ New Features Gómez Jiménez `. - The transformer :class:`DurationToFloat` has been added. :pr:`2069` by :user:`Riccardo Cappuzzo `. + **TableReport**: - :meth:`TableReport.json` now includes histogram data for numeric and datetime From 993f88e09c65de2f76ee6a5ee2068b207964b0a8 Mon Sep 17 00:00:00 2001 From: Riccardo Cappuzzo Date: Mon, 6 Jul 2026 13:44:17 +0200 Subject: [PATCH 4/4] _ --- CHANGES.rst | 14 +++++++++----- doc/api_reference.py | 4 +++- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/CHANGES.rst b/CHANGES.rst index ba574899f..2e6d8e6ed 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -22,7 +22,7 @@ New Features - The :class:`DropSimilar` transformer has been added, for removing columns that present high correlation with other columns in a dataframe . :pr:`2023` by :user:`Eloi Massoulié `. -- :class:`ToFloat32` now allows users to specify ``decimal`` and ``thousand`` +- :class:`ToFloat` now allows users to specify ``decimal`` and ``thousand`` separators to parse numerical columns that use formatting different from the default formatting used in Python, such as ``1'234,5``. Additionally, negative numbers indicated with parentheses can be converted to the @@ -36,7 +36,8 @@ New Features - :meth:`TableReport.json` now includes histogram data for numeric and datetime columns (the bin count and edges, and numbers of low and high outliers). Now ``json()`` contains all the information shown in the report HTML rendering, - including the plots. The schema of the generated JSON is available at :ref:``. + including the plots. The schema of the generated JSON is available at + :ref:`table_report_json_schema`. :pr:`2164` by :user:`Jérôme Dockès `. - The :class:`TableReport` can now be exported in markdown format with :func:`~skrub.TableReport.markdown`. @@ -51,8 +52,10 @@ New Features control respectively which scores, choices (params), and times (fit or score durations) should be included in the figure. :pr:`2202` by :user:`Jérôme Dockès `. -- New methods :meth:`SkrubLearner.get_named_params` and - :meth:`SkrubLearner.set_named_params` allow getting and setting the outcomes for +- New :class:`SkrubLearner` methods + :meth:`SkrubLearner.get_named_params` and + :meth:`SkrubLearner.set_named_params` allow getting and + setting the outcomes for choices contained in the DataOp, keyed by choice name. They provide a more robust way of transferring selected hyperparameters from one DataOp to a different one than :meth:`SkrubLearner.get_params` and @@ -88,7 +91,8 @@ New Features Changes ------- -- :meth:`SkrubLearner.score` has been enhanced when the DataOp used +- :class:`SkrubLearner`'s :meth:`SkrubLearner.score` + has been enhanced when the DataOp used :meth:`DataOp.skb.with_scoring`. During scoring, predict(), predict_proba() etc. are cached to avoid recomputation when multiple scorers are used (or one scorer calls them several times). Moreover it is possible to pass diff --git a/doc/api_reference.py b/doc/api_reference.py index c7f9056ca..7e3a4ec63 100644 --- a/doc/api_reference.py +++ b/doc/api_reference.py @@ -122,10 +122,12 @@ "description": None, "autosummary": [ "SquashingScaler", - "deduplicate", "Cleaner", "DropUninformative", + "DropSimilar", + "DurationToFloat", "to_datetime", + "deduplicate", ], }, ],