Release Candidate 1.4.0 (#632)

Author: EmilHvitfeldt

.claude/skills/tidy-deprecate-function/SKILL.md

@@ -6,7 +6,7 @@ description: Guide for deprecating R functions/arguments. Use when a user asks t
 # Deprecate functions and function arguments
 
 Use this skill when deprecating functions or function parameters in dbplyr.
-
+<!--  -->
 ## Overview
 
 This skill guides you through the complete process of deprecating a function or parameter, ensuring all necessary changes are made consistently:

DESCRIPTION

@@ -1,7 +1,7 @@
 Type: Package
 Package: yardstick
 Title: Tidy Characterizations of Model Performance
-Version: 1.3.2.9000
+Version: 1.4.0.9000
 Authors@R: c(
     person("Max", "Kuhn", , "max@posit.co", role = "aut"),
     person("Davis", "Vaughan", , "davis@posit.co", role = "aut"),
@@ -23,7 +23,7 @@ Imports:
     cli,
     dplyr (>= 1.1.0),
     generics (>= 0.1.2),
-    hardhat (>= 1.4.2.9000),
+    hardhat (>= 1.4.3),
     lifecycle (>= 1.0.3),
     rlang (>= 1.1.4),
     tibble,
@@ -137,5 +137,3 @@ Collate:
     'template.R'
     'validation.R'
     'yardstick-package.R'
-Remotes:
-    tidymodels/hardhat

NEWS.md

@@ -1,64 +1,86 @@
 # yardstick (development version)
 
-* `sedi()` was added to compute the Symmetric Extremal Dependence Index, a prevalence-independent skill metric for classification. SEDI remains reliable at extreme class imbalance (prevalence < 2.5%) where TSS and MCC degrade. Supports binary and multiclass (macro, macro-weighted, micro averaging via one-vs-all decomposition). Based on Ferro & Stephenson (2011) and recommended by Wunderlich et al. (2019) for species distribution models with rare events.
+# yardstick 1.4.0
 
-* `brier_class()` has gained the `event_level` argument. (#515)
+## Breaking Changes
 
-* `gini_coef()` was added to compute the normalized Gini coefficient for regression, which measures ranking ability based on the Lorenz curve. This is useful for evaluating loss cost models and risk predictions. (#147)
+* The global option `yardstick.event_first` (deprecated in 0.0.7) now throws an error. Use the `event_level` argument of individual metric functions instead. (#632)
 
-* `roc_dist()` was added to compute the Euclidean distance from (sensitivity, specificity) to the ideal point (1, 1) in ROC space. (#148)
+* `conf_mat()` now throws an error if anything is passed to `...` (deprecated in 1.0.0). This argument has had no effect since case weight support was added. (#632)
 
-* `rmse_relative()` was added to compute relative root mean squared error, which normalizes RMSE by the range of the true values. (#527)
+* `roc_auc()`, `roc_aunp()`, `roc_aunu()`, and `roc_curve()` now throw an error if a non-empty list is passed to `options` (deprecated in 1.0.0). Use the pROC package directly if you need these features. (#632)
 
-* `mse()` was added to compute the mean squared error. (#560)
+## Deprecations
 
-* Added documentation pages for each metric type (e.g., `?class-metrics`, `?numeric-metrics`) that list all available metrics with their direction and range. (#547, #540)
+* `dots_to_estimate()`, `metric_summarizer()`, and `metric_vec_template()` (soft-deprecated in 1.2.0) now warn for all users. See the yardstick 1.2.0 release notes for recommended replacements. (#632)
 
-* For metrics with alternate argument values that will be used in a metric set, the documentation pages emphasize doing this via `metric_tweak()` #626   
+## New Metrics
 
-* `get_metrics()` was added to return a `metric_set()` containing all metrics of a specified type. (#534)
+* `gini_coef()` computes the normalized Gini coefficient for regression, which measures ranking ability based on the Lorenz curve. (#147)
 
-* All class metrics and probability metrics now include mathematical formulas in their documentation. (#605)
+* `mse()` computes the mean squared error. (#560)
 
-* `mpe()` documentation now includes the formula and clarifies the interpretation of positive and negative values. (#345)
+* `rmse_relative()` computes the relative root mean squared error, normalizing RMSE by the range of the true values. (#527)
 
-* `classification_cost()` documentation now correctly refers to the `cost` column of the data.frame that can be passed to the `costs` arguemtn. (#343)
+* `fall_out()` and `miss_rate()` compute the false positive rate and false negative rate respectively. (#336)
 
-* `new_metric()` and related functions gain an optional `range` argument to store the valid output range of a metric. This is a developer-facing change. (#572)
+* `markedness()` computes the markedness metric (PPV + NPV - 1), the predictive power analog of `j_index()`. (#27)
 
-* `markedness()` calculates the markedness metric (PPV + NPV - 1), which is the predictive power analog of informedness/j_index (#27).
+* `roc_dist()` computes the Euclidean distance from the (sensitivity, specificity) point to the ideal point (1, 1) in ROC space. (#148)
 
-* `metric_set()` now provides a more informative error message when `estimate` is not explicitly named for class/prob or survival metric sets. (#504)
+* `sedi()` computes the Symmetric Extremal Dependence Index, a prevalence-independent skill metric for binary classification that remains reliable at extreme class imbalance. (#630)
 
-* Added `thresholds` argument to `roc_curve()` to allow for custom thresholds to calculate curves for. (#488)
+* `ranked_prob_score()` computes the ranked probability score for ordinal classification data. (#524)
 
-* Speed up survival metrics performance. Some of this performance comes from slightly less strict input checking. (#576)
+* `weighted_interval_score()` is a new quantile metric. (#569)
+
+## Improvements
 
-* All metrics now have documented ranges of possible values in addition to what direction is the best. (#572)
+* Added checks to all metrics for the `na_rm` argument. (#349)
 
-* The ranked probability score for ordinal classification data was added with `ranked_prob_score()`. (#524)
+* Added improved argument checking for metrics with additional arguments. (#519)
 
-* Fixed bug where `brier_class()` returns NaN with extreme value case weights. (#614)
+* Added documentation pages for each metric type (e.g., `?class-metrics`, `?numeric-metrics`) listing all available metrics with their direction and range. (#547, #540)
 
-* `poisson_log_loss()` has been enhanced to handle 0 valued estimates, no longer returning `Inf` or `NaN`. (#513)
+* All class metrics and probability metrics now include mathematical formulas in their documentation. (#605)
 
-* Fixed bug where ranked probability metrics didn't work in combination with other classification metrics in `metric_set()`. (#539)
+* All metrics now document their valid range of output values. (#572)
 
-* Added infrastructure for survival metrics on the linear predictor. (#551)
+* Documentation pages for metrics with alternate argument values now emphasize using `metric_tweak()` when building metric sets. (#626)
 
-* Added infrastructure for quantile metrics. (#569)
+* Survival metrics performance has been improved. (#576)
 
-* Added quantile metric `weighted_interval_score()`. (#569)
+* `brier_class()` has gained the `event_level` argument. (#515)
 
-* Added checks to all metrics for `na_rm` argument. (#349)
+* `get_metrics()` has been added to return a `metric_set()` containing all metrics of a specified type. (#534)
 
-* Removed crayon as a suggested package. (#574)
+* `metric_set()` now provides a more informative error message when `estimate` is not explicitly named for class/prob or survival metric sets. (#504)
 
-* Added improved argument checking for metrics with additional arguments. (#519)
+* `roc_curve()` has gained a `thresholds` argument for specifying custom thresholds at which the curve is evaluated. (#488)
+
+## Bug Fixes
+
+* `brier_class()` no longer returns `NaN` with extreme value case weights. (#614)
+
+* `classification_cost()` documentation now correctly refers to the `cost` column of the costs data frame. (#343)
+
+* `mpe()` documentation now includes the formula and clarifies the interpretation of positive and negative values. (#345)
+
+* `poisson_log_loss()` now handles 0-valued estimates without returning `Inf` or `NaN`. (#513)
+
+* Fixed a bug where ranked probability metrics didn't work in combination with other classification metrics in `metric_set()`. (#539)
 
 * Fixed documentation to show equations correctly. (#541)
 
-* `fall_out()` and `miss_rate()` have been added to compute the false positive rate and false negative rate respectively (#336).
+## Developer
+
+* Added infrastructure for quantile metrics. (#569)
+
+* Added infrastructure for survival metrics on the linear predictor. (#551)
+
+* `new_metric()` and related functions gain an optional `range` argument to store the valid output range of a metric. (#572)
+
+* Removed crayon as a suggested package. (#574)
 
 # yardstick 1.3.2
 

R/aaa-new.R

@@ -155,6 +155,10 @@ metric_direction <- function(x) {
 metric_range <- function(x) {
   attr(x, "range", exact = TRUE)
 }
+metric_range_chr <- function(x, i) {
+  val <- metric_range(x)[[i]]
+  if (is.infinite(val)) paste0(if (val < 0) "-", "Inf") else as.character(val)
+}
 `metric_range<-` <- function(x, value) {
   attr(x, "range") <- value
   x

R/class-accuracy.R

@@ -25,8 +25,8 @@
 #' \deqn{\text{Accuracy} = \frac{A + D}{A + B + C + D}}
 #'
 #' Accuracy is a metric that should be `r attr(accuracy, "direction")`d. The
-#' output ranges from `r metric_range(accuracy)[1]` to
-#' `r metric_range(accuracy)[2]`, with `r metric_optimal(accuracy)` indicating
+#' output ranges from `r metric_range_chr(accuracy, 1)` to
+#' `r metric_range_chr(accuracy, 2)`, with `r metric_optimal(accuracy)` indicating
 #' perfect predictions.
 #'
 #' @author Max Kuhn

R/class-bal_accuracy.R

@@ -27,7 +27,7 @@
 #'
 #' Balanced accuracy is a metric that should be
 #' `r attr(bal_accuracy, "direction")`d. The output ranges from
-#' `r metric_range(bal_accuracy)[1]` to `r metric_range(bal_accuracy)[2]`, with
+#' `r metric_range_chr(bal_accuracy, 1)` to `r metric_range_chr(bal_accuracy, 2)`, with
 #' `r metric_optimal(bal_accuracy)` indicating perfect predictions.
 #'
 #' @author Max Kuhn

R/class-detection_prevalence.R

@@ -24,7 +24,7 @@
 #'
 #' Detection prevalence is a metric that should be
 #' `r attr(detection_prevalence, "direction")`d. The output ranges from
-#' `r metric_range(detection_prevalence)[1]` to `r metric_range(detection_prevalence)[2]`.
+#' `r metric_range_chr(detection_prevalence, 1)` to `r metric_range_chr(detection_prevalence, 2)`.
 #' The "optimal" value depends on the true prevalence of positive events in the data.
 #'
 #' @author Max Kuhn

R/class-f_meas.R

@@ -36,8 +36,8 @@
 #' \deqn{F_{meas} = \frac{(1 + \beta^2) \cdot \text{Precision} \cdot \text{Recall}}{(\beta^2 \cdot \text{Precision}) + \text{Recall}}}
 #'
 #' F measure is a metric that should be `r attr(f_meas, "direction")`d. The
-#' output ranges from `r metric_range(f_meas)[1]` to
-#' `r metric_range(f_meas)[2]`, with `r metric_optimal(f_meas)` indicating
+#' output ranges from `r metric_range_chr(f_meas, 1)` to
+#' `r metric_range_chr(f_meas, 2)`, with `r metric_optimal(f_meas)` indicating
 #' perfect precision and recall.
 #'
 #' @references

R/class-fall_out.R

@@ -35,8 +35,8 @@
 #' \deqn{\text{Fall-out} = \frac{B}{B + D}}
 #'
 #' Fall-out is a metric that should be `r attr(fall_out, "direction")`d. The
-#' output ranges from `r metric_range(fall_out)[1]` to
-#' `r metric_range(fall_out)[2]`, with `r metric_optimal(fall_out)` indicating
+#' output ranges from `r metric_range_chr(fall_out, 1)` to
+#' `r metric_range_chr(fall_out, 2)`, with `r metric_optimal(fall_out)` indicating
 #' that all actual negatives were correctly predicted as negative (no false
 #' positives).
 #'

R/class-j_index.R

@@ -22,8 +22,8 @@
 #' \deqn{\text{J-index} = \text{Sensitivity} + \text{Specificity} - 1}
 #'
 #' J-index is a metric that should be `r attr(j_index, "direction")`d. The
-#' output ranges from `r metric_range(j_index)[1]` to
-#' `r metric_range(j_index)[2]`, with `r metric_optimal(j_index)` indicating no
+#' output ranges from `r metric_range_chr(j_index, 1)` to
+#' `r metric_range_chr(j_index, 2)`, with `r metric_optimal(j_index)` indicating no
 #' false positives and no false negatives.
 #'
 #' The binary version of J-index is equivalent to the binary concept of