[ci skip] MAINT Replace R^2 with MAE in exercise M3.02 (#830) e62b23c

Author: ArturoAmorQ

.buildinfo

@@ -1,4 +1,4 @@
 # Sphinx build info version 1
 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
-config: 804f24a7d6da2bb21a214583fed15567
+config: 08c157dd73b0a8da763c180621a73e08
 tags: 645f666f9bcd5a90fca523b33c5a78b7

_sources/python_scripts/parameter_tuning_ex_03.py

@@ -40,8 +40,9 @@
 # Write your code here.
 
 # %% [markdown]
-# Use `RandomizedSearchCV` with `n_iter=20` to find the best set of
-# hyperparameters by tuning the following parameters of the `model`:
+# Use `RandomizedSearchCV` with `n_iter=20` and
+# `scoring="neg_mean_absolute_error"` to tune the following hyperparameters
+# of the `model`:
 #
 # - the parameter `n_neighbors` of the `KNeighborsRegressor` with values
 #   `np.logspace(0, 3, num=10).astype(np.int32)`;
@@ -50,6 +51,11 @@
 # - the parameter `with_std` of the `StandardScaler` with possible values `True`
 #   or `False`.
 #
+# The `scoring` function is expected to return higher values for better models,
+# since grid/random search objects **maximize** it. Because of that, error
+# metrics like `mean_absolute_error` must be negated (using the `neg_` prefix)
+# to work correctly (remember lower errors represent better models).
+#
 # Notice that in the notebook "Hyperparameter tuning by randomized-search" we
 # pass distributions to be sampled by the `RandomizedSearchCV`. In this case we
 # define a fixed grid of hyperparameters to be explored. Using a `GridSearchCV`

_sources/python_scripts/parameter_tuning_sol_03.py

@@ -40,8 +40,9 @@
 model = make_pipeline(scaler, KNeighborsRegressor())
 
 # %% [markdown]
-# Use `RandomizedSearchCV` with `n_iter=20` to find the best set of
-# hyperparameters by tuning the following parameters of the `model`:
+# Use `RandomizedSearchCV` with `n_iter=20` and
+# `scoring="neg_mean_absolute_error"` to tune the following hyperparameters
+# of the `model`:
 #
 # - the parameter `n_neighbors` of the `KNeighborsRegressor` with values
 #   `np.logspace(0, 3, num=10).astype(np.int32)`;
@@ -50,6 +51,11 @@
 # - the parameter `with_std` of the `StandardScaler` with possible values `True`
 #   or `False`.
 #
+# The `scoring` function is expected to return higher values for better models,
+# since grid/random search objects **maximize** it. Because of that, error
+# metrics like `mean_absolute_error` must be negated (using the `neg_` prefix)
+# to work correctly (remember lower errors represent better models).
+#
 # Notice that in the notebook "Hyperparameter tuning by randomized-search" we
 # pass distributions to be sampled by the `RandomizedSearchCV`. In this case we
 # define a fixed grid of hyperparameters to be explored. Using a `GridSearchCV`
@@ -79,6 +85,7 @@
 model_random_search = RandomizedSearchCV(
     model,
     param_distributions=param_distributions,
+    scoring="neg_mean_absolute_error",
     n_iter=20,
     n_jobs=2,
     verbose=1,
@@ -107,6 +114,13 @@
 
 cv_results = pd.DataFrame(model_random_search.cv_results_)
 
+# %% [markdown] tags=["solution"]
+# As we used `neg_mean_absolute_error` as score metric, we should multiply the
+# score results with minus 1 to get mean absolute error values:
+
+# %% tags=["solution"]
+cv_results["mean_test_score"] *= -1
+
 # %% [markdown] tags=["solution"]
 # To simplify the axis of the plot, we rename the column of the dataframe and
 # only select the mean test score and the value of the hyperparameters.
@@ -121,7 +135,7 @@
 
 cv_results = cv_results.rename(columns=column_name_mapping)
 cv_results = cv_results[column_name_mapping.values()].sort_values(
-    "mean test score", ascending=False
+    "mean test score"
 )
 
 # %% [markdown] tags=["solution"]
@@ -153,7 +167,7 @@
 # holding on any axis of the parallel coordinate plot. You can then slide (move)
 # the range selection and cross two selections to see the intersections.
 #
-# Selecting the best performing models (i.e. above R2 score of ~0.68), we
+# Selecting the best performing models (i.e. below MEA score of ~47 k$), we
 # observe that **in this case**:
 #
 # - scaling the data is important. All the best performing models use scaled

appendix/notebook_timings.html

@@ -1175,9 +1175,9 @@ <h1>Notebook timings<a class="headerlink" href="#notebook-timings" title="Link t
 <td><p>✅</p></td>
 </tr>
 <tr class="row-odd"><td><p><a class="xref doc reference internal" href="../python_scripts/parameter_tuning_sol_03.html"><span class="doc">python_scripts/parameter_tuning_sol_03</span></a></p></td>
-<td><p>2025-04-01 09:09</p></td>
+<td><p>2025-04-22 15:13</p></td>
 <td><p>cache</p></td>
-<td><p>19.48</p></td>
+<td><p>21.24</p></td>
 <td><p>✅</p></td>
 </tr>
 <tr class="row-even"><td><p><a class="xref doc reference internal" href="../python_scripts/trees_classification.html"><span class="doc">python_scripts/trees_classification</span></a></p></td>

python_scripts/parameter_tuning_ex_03.html

@@ -728,8 +728,9 @@ <h1>📝 Exercise M3.02<a class="headerlink" href="#exercise-m3-02" title="Link
 </div>
 </div>
 </div>
-<p>Use <code class="docutils literal notranslate"><span class="pre">RandomizedSearchCV</span></code> with <code class="docutils literal notranslate"><span class="pre">n_iter=20</span></code> to find the best set of
-hyperparameters by tuning the following parameters of the <code class="docutils literal notranslate"><span class="pre">model</span></code>:</p>
+<p>Use <code class="docutils literal notranslate"><span class="pre">RandomizedSearchCV</span></code> with <code class="docutils literal notranslate"><span class="pre">n_iter=20</span></code> and
+<code class="docutils literal notranslate"><span class="pre">scoring=&quot;neg_mean_absolute_error&quot;</span></code> to tune the following hyperparameters
+of the <code class="docutils literal notranslate"><span class="pre">model</span></code>:</p>
 <ul class="simple">
 <li><p>the parameter <code class="docutils literal notranslate"><span class="pre">n_neighbors</span></code> of the <code class="docutils literal notranslate"><span class="pre">KNeighborsRegressor</span></code> with values
 <code class="docutils literal notranslate"><span class="pre">np.logspace(0,</span> <span class="pre">3,</span> <span class="pre">num=10).astype(np.int32)</span></code>;</p></li>
@@ -738,6 +739,10 @@ <h1>📝 Exercise M3.02<a class="headerlink" href="#exercise-m3-02" title="Link
 <li><p>the parameter <code class="docutils literal notranslate"><span class="pre">with_std</span></code> of the <code class="docutils literal notranslate"><span class="pre">StandardScaler</span></code> with possible values <code class="docutils literal notranslate"><span class="pre">True</span></code>
 or <code class="docutils literal notranslate"><span class="pre">False</span></code>.</p></li>
 </ul>
+<p>The <code class="docutils literal notranslate"><span class="pre">scoring</span></code> function is expected to return higher values for better models,
+since grid/random search objects <strong>maximize</strong> it. Because of that, error
+metrics like <code class="docutils literal notranslate"><span class="pre">mean_absolute_error</span></code> must be negated (using the <code class="docutils literal notranslate"><span class="pre">neg_</span></code> prefix)
+to work correctly (remember lower errors represent better models).</p>
 <p>Notice that in the notebook “Hyperparameter tuning by randomized-search” we
 pass distributions to be sampled by the <code class="docutils literal notranslate"><span class="pre">RandomizedSearchCV</span></code>. In this case we
 define a fixed grid of hyperparameters to be explored. Using a <code class="docutils literal notranslate"><span class="pre">GridSearchCV</span></code>