Deploying to gh-pages from @ 18a5299 🚀

Author: sebffischer

dev/LICENSE-text.html

@@ -0,0 +1,167 @@
+# License
+
+                       GNU LESSER GENERAL PUBLIC LICENSE
+                           Version 3, 29 June 2007
+
+     Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
+     Everyone is permitted to copy and distribute verbatim copies
+     of this license document, but changing it is not allowed.
+
+
+      This version of the GNU Lesser General Public License incorporates
+    the terms and conditions of version 3 of the GNU General Public
+    License, supplemented by the additional permissions listed below.
+
+      0. Additional Definitions.
+
+      As used herein, "this License" refers to version 3 of the GNU Lesser
+    General Public License, and the "GNU GPL" refers to version 3 of the GNU
+    General Public License.
+
+      "The Library" refers to a covered work governed by this License,
+    other than an Application or a Combined Work as defined below.
+
+      An "Application" is any work that makes use of an interface provided
+    by the Library, but which is not otherwise based on the Library.
+    Defining a subclass of a class defined by the Library is deemed a mode
+    of using an interface provided by the Library.
+
+      A "Combined Work" is a work produced by combining or linking an
+    Application with the Library.  The particular version of the Library
+    with which the Combined Work was made is also called the "Linked
+    Version".
+
+      The "Minimal Corresponding Source" for a Combined Work means the
+    Corresponding Source for the Combined Work, excluding any source code
+    for portions of the Combined Work that, considered in isolation, are
+    based on the Application, and not on the Linked Version.
+
+      The "Corresponding Application Code" for a Combined Work means the
+    object code and/or source code for the Application, including any data
+    and utility programs needed for reproducing the Combined Work from the
+    Application, but excluding the System Libraries of the Combined Work.
+
+      1. Exception to Section 3 of the GNU GPL.
+
+      You may convey a covered work under sections 3 and 4 of this License
+    without being bound by section 3 of the GNU GPL.
+
+      2. Conveying Modified Versions.
+
+      If you modify a copy of the Library, and, in your modifications, a
+    facility refers to a function or data to be supplied by an Application
+    that uses the facility (other than as an argument passed when the
+    facility is invoked), then you may convey a copy of the modified
+    version:
+
+       a) under this License, provided that you make a good faith effort to
+       ensure that, in the event an Application does not supply the
+       function or data, the facility still operates, and performs
+       whatever part of its purpose remains meaningful, or
+
+       b) under the GNU GPL, with none of the additional permissions of
+       this License applicable to that copy.
+
+      3. Object Code Incorporating Material from Library Header Files.
+
+      The object code form of an Application may incorporate material from
+    a header file that is part of the Library.  You may convey such object
+    code under terms of your choice, provided that, if the incorporated
+    material is not limited to numerical parameters, data structure
+    layouts and accessors, or small macros, inline functions and templates
+    (ten or fewer lines in length), you do both of the following:
+
+       a) Give prominent notice with each copy of the object code that the
+       Library is used in it and that the Library and its use are
+       covered by this License.
+
+       b) Accompany the object code with a copy of the GNU GPL and this license
+       document.
+
+      4. Combined Works.
+
+      You may convey a Combined Work under terms of your choice that,
+    taken together, effectively do not restrict modification of the
+    portions of the Library contained in the Combined Work and reverse
+    engineering for debugging such modifications, if you also do each of
+    the following:
+
+       a) Give prominent notice with each copy of the Combined Work that
+       the Library is used in it and that the Library and its use are
+       covered by this License.
+
+       b) Accompany the Combined Work with a copy of the GNU GPL and this license
+       document.
+
+       c) For a Combined Work that displays copyright notices during
+       execution, include the copyright notice for the Library among
+       these notices, as well as a reference directing the user to the
+       copies of the GNU GPL and this license document.
+
+       d) Do one of the following:
+
+           0) Convey the Minimal Corresponding Source under the terms of this
+           License, and the Corresponding Application Code in a form
+           suitable for, and under terms that permit, the user to
+           recombine or relink the Application with a modified version of
+           the Linked Version to produce a modified Combined Work, in the
+           manner specified by section 6 of the GNU GPL for conveying
+           Corresponding Source.
+
+           1) Use a suitable shared library mechanism for linking with the
+           Library.  A suitable mechanism is one that (a) uses at run time
+           a copy of the Library already present on the user's computer
+           system, and (b) will operate properly with a modified version
+           of the Library that is interface-compatible with the Linked
+           Version.
+
+       e) Provide Installation Information, but only if you would otherwise
+       be required to provide such information under section 6 of the
+       GNU GPL, and only to the extent that such information is
+       necessary to install and execute a modified version of the
+       Combined Work produced by recombining or relinking the
+       Application with a modified version of the Linked Version. (If
+       you use option 4d0, the Installation Information must accompany
+       the Minimal Corresponding Source and Corresponding Application
+       Code. If you use option 4d1, you must provide the Installation
+       Information in the manner specified by section 6 of the GNU GPL
+       for conveying Corresponding Source.)
+
+      5. Combined Libraries.
+
+      You may place library facilities that are a work based on the
+    Library side by side in a single library together with other library
+    facilities that are not Applications and are not covered by this
+    License, and convey such a combined library under terms of your
+    choice, if you do both of the following:
+
+       a) Accompany the combined library with a copy of the same work based
+       on the Library, uncombined with any other library facilities,
+       conveyed under the terms of this License.
+
+       b) Give prominent notice with the combined library that part of it
+       is a work based on the Library, and explaining where to find the
+       accompanying uncombined form of the same work.
+
+      6. Revised Versions of the GNU Lesser General Public License.
+
+      The Free Software Foundation may publish revised and/or new versions
+    of the GNU Lesser General Public License from time to time. Such new
+    versions will be similar in spirit to the present version, but may
+    differ in detail to address new problems or concerns.
+
+      Each version is given a distinguishing version number. If the
+    Library as you received it specifies that a certain numbered version
+    of the GNU Lesser General Public License "or any later version"
+    applies to it, you have the option of following the terms and
+    conditions either of that published version or of any later version
+    published by the Free Software Foundation. If the Library as you
+    received it does not specify a version number of the GNU Lesser
+    General Public License, you may choose any version of the GNU Lesser
+    General Public License ever published by the Free Software Foundation.
+
+      If the Library as you received it specifies that a proxy can decide
+    whether future versions of the GNU Lesser General Public License shall
+    apply, that proxy's public statement of acceptance of any version is
+    permanent authorization for you to choose that version for the
+    Library.

dev/LICENSE-text.md

@@ -0,0 +1,26 @@
+# Authors and Citation
+
+## Authors
+
+- **Sebastian Fischer**. Maintainer, author.
+  [](https://orcid.org/0000-0002-9609-3197)
+
+- **Hannah Schulz-Kümpel**. Author.
+  [](https://orcid.org/0000-0003-3972-8392)
+
+## Citation
+
+Source:
+[`DESCRIPTION`](https://github.com/mlr-org/mlr3inferr/blob/main/DESCRIPTION)
+
+Fischer S, Schulz-Kümpel H (2025). *mlr3inferr: Inference on the
+Generalization Error*. R package version 0.2.0.9000,
+<https://mlr3inferr.mlr-org.com>.
+
+    @Manual{,
+      title = {mlr3inferr: Inference on the Generalization Error},
+      author = {Sebastian Fischer and Hannah Schulz-Kümpel},
+      year = {2025},
+      note = {R package version 0.2.0.9000},
+      url = {https://mlr3inferr.mlr-org.com},
+    }

dev/authors.html

@@ -0,0 +1,145 @@
+# mlr3inferr
+
+Methods for statistical **inf**erence on the generalization **err**or.
+
+Package website: [release](https://mlr3inferr.mlr-org.com/) \|
+[dev](https://mlr3inferr.mlr-org.com/dev/)
+
+## Installation
+
+``` r
+# Install release from CRAN
+install.packages("mlr3inferr")
+# Install development version from GitHub
+pak::pkg_install("mlr-org/mlr3inferr")
+```
+
+## What is `mlr3inferr`?
+
+The main purpose of the package is to allow to obtain confidence
+intervals for the generalization error for a number of resampling
+methods. Below, we evaluate a decision tree on the sonar task using a
+holdout resampling and obtain a confidence interval for the
+generalization error. This is achieved using the `msr("ci.holdout")`
+measure, to which we pass another
+[`mlr3::Measure`](https://mlr3.mlr-org.com/reference/Measure.html) that
+determines the loss function.
+
+``` r
+library(mlr3inferr)
+
+rr = resample(tsk("sonar"), lrn("classif.rpart"), rsmp("holdout"))
+# 0.05 is also the default
+ci = msr("ci.holdout", "classif.acc", alpha = 0.05)
+rr$aggregate(ci)
+#>       classif.acc classif.acc.lower classif.acc.upper 
+#>         0.7391304         0.6347628         0.8434981
+```
+
+It is also possible to select the default inference method for a certain
+`Resampling` method using `msr("ci")`
+
+``` r
+ci_default = msr("ci", "classif.acc")
+rr$aggregate(ci_default)
+#>       classif.acc classif.acc.lower classif.acc.upper 
+#>         0.7391304         0.6347628         0.8434981
+```
+
+With [`mlr3viz`](https://mlr3viz.mlr-org.com), it is also possible to
+visualize multiple confidence intervals. Below, we compare a random
+forest with a decision tree and a featureless learner:
+
+``` r
+library(mlr3learners)
+library(mlr3viz)
+
+bmr = benchmark(benchmark_grid(
+  tsks(c("sonar", "german_credit")),
+  lrns(c("classif.rpart", "classif.ranger", "classif.featureless")),
+  rsmp("subsampling")
+))
+
+autoplot(bmr, "ci", msr("ci", "classif.ce"))
+```
+
+![](reference/figures/README-unnamed-chunk-5-1.png)
+
+Note that:
+
+- Some methods require pointwise loss functions, i.e. have an
+  `$obs_loss` field.
+- Not for every resampling method exists an inference method.
+- There are combinations of datasets and learners, where inference
+  methods can fail.
+
+⚠️ **Different point estimates for the same measure**
+
+In general, the point estimate of `$aggregate(msr("ci", <key>))` will
+**not** always be exactly identical to the point estimate of
+`$aggregate(msr(<key>))`. This is because the point estimation for the
+former is defined by the inference method, and can, for example, as is
+the case for nested cross-validation, contain a bias correction term, or
+use a different aggregation method. This is demonstrated in the example
+below.
+
+``` r
+rr = resample(tsk("iris"), lrn("classif.rpart"), rsmp("ncv", folds = 5L, repeats = 20L))
+ce = msr("classif.ce")
+ci = msr("ci", ce)
+c(rr$aggregate(ce)[[1]], rr$aggregate(ci)[[1]])
+#> [1] 0.06466667 0.06646667
+```
+
+## Features
+
+- Additional Resampling Methods
+- Confidence Intervals for the Generalization Error for some resampling
+  methods
+
+## Inference Methods
+
+| Key        | Label                   | Resamplings       | Only Pointwise Loss |
+|:-----------|:------------------------|:------------------|:--------------------|
+| ci.con_z   | Conservative-Z Interval | PairedSubsampling | false               |
+| ci.cor_t   | Corrected-T Interval    | Subsampling       | false               |
+| ci.holdout | Holdout Interval        | Holdout           | yes                 |
+| ci.ncv     | Nested CV Interval      | NestedCV          | yes                 |
+| ci.wald_cv | Wald CV Interval        | CV, LOO           | yes                 |
+
+## Citing mlr3
+
+If you use mlr3inferr, please cite our paper:
+
+``` R
+@misc{kuempelfischer2024ciforge,
+      title={Constructing Confidence Intervals for 'the' Generalization Error -- a Comprehensive Benchmark Study},
+      author={Hannah Schulz-Kümpel and Sebastian Fischer and Thomas Nagler and Anne-Laure Boulesteix and Bernd Bischl and Roman Hornung},
+      year={2024},
+      eprint={2409.18836},
+      archivePrefix={arXiv},
+      primaryClass={stat.ML},
+      url={https://arxiv.org/abs/2409.18836},
+}
+```
+
+## Acknowledgements
+
+This R package is developed as part of the [Mathematical Research Data
+Initiative](https://www.mardi4nfdi.de/about/mission).
+
+## Bugs, Questions, Feedback
+
+*mlr3inferr* is a free and open source software project that encourages
+participation and feedback. If you have any issues, questions,
+suggestions or feedback, please do not hesitate to open an “issue” about
+it on the GitHub page!
+
+In case of problems / bugs, it is often helpful if you provide a
+“minimum working example” that showcases the behaviour (but don’t worry
+about this if the bug is obvious).
+
+Please understand that the resources of the project are limited:
+response may sometimes be delayed by a few days, and some feature
+suggestions may be rejected if they are deemed too tangential to the
+vision behind the project.

dev/authors.md

@@ -11,4 +11,6 @@ document.addEventListener("DOMContentLoaded", function () {
         macros: macros,
         fleqn: false
       });
-    }}});
+    }
+  }
+});

dev/deps/bootstrap-5.3.1/bootstrap.min.css

@@ -26,7 +26,7 @@ const setTheme = theme => {
   }
 }
 
-function bsSetupThemeToggle () {
+function bsSetupThemeToggle() {
   'use strict'
 
   const showActiveTheme = (theme, focus = false) => {
@@ -35,7 +35,7 @@ function bsSetupThemeToggle () {
     document.querySelectorAll('[data-bs-theme-value]').forEach(element => {
       const buttonTheme = element.getAttribute('data-bs-theme-value')
       const isActive = buttonTheme == theme
-      
+
       element.classList.toggle('active', isActive)
       element.setAttribute('aria-pressed', isActive)