Skip to content

Commit a4266f4

Browse files
elemetselemets
committed
Pushing doc changes
1 parent 771396a commit a4266f4

File tree

14 files changed

+303
-66
lines changed

14 files changed

+303
-66
lines changed

‎docs/_sources/usage_guide.rst.txt‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1960,6 +1960,128 @@ See :ref:`this section <model_calibration>` for more information on model calibr
19601960
<div style="height: 50px;"></div>
19611961

19621962

1963+
F1 Beta Threshold Tuning
1964+
------------------------
1965+
In binary classification, selecting an optimal classification threshold is crucial for
1966+
achieving the best balance between precision and recall. The default threshold of 0.5
1967+
may not always yield optimal performance, especially when dealing with imbalanced
1968+
datasets. F1 Beta threshold tuning helps adjust this threshold to maximize the F-beta
1969+
score, which balances precision and recall according to the importance assigned to each
1970+
through the beta parameter.
1971+
1972+
Understanding F1 Beta Score
1973+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1974+
1975+
The F-beta score is a generalization of the F1-score that allows you to weigh recall
1976+
more heavily than precision (or vice versa) based on the beta parameter:
1977+
1978+
- F1-Score (beta = 1): Equal importance to precision and recall.
1979+
1980+
- F-beta > 1: More emphasis on recall. Useful when false negatives are more critical (e.g., disease detection).
1981+
1982+
- F-beta < 1: More emphasis on precision. Suitable when false positives are costlier (e.g., spam detection).
1983+
1984+
Example usage: default (beta = 1)
1985+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1986+
1987+
Setting up the Model object ready for tuning.
1988+
1989+
.. code-block:: python
1990+
1991+
from xgboost import XGBClassifier
1992+
1993+
xgb_name = "xgb"
1994+
xgb = XGBClassifier(
1995+
objective="binary:logistic"
1996+
random_state=222,
1997+
)
1998+
1999+
tuned_parameters_xgb = {
2000+
f"{xgb_name}__max_depth": [3, 10, 20, 200, 500],
2001+
f"{xgb_name}__learning_rate": [1e-4],
2002+
f"{xgb_name}__n_estimators": [1000],
2003+
f"{xgb_name}__early_stopping_rounds": [100],
2004+
f"{xgb_name}__verbose": [0],
2005+
f"{xgb_name}__eval_metric": ["logloss"],
2006+
}
2007+
2008+
xgb_model = Model(
2009+
name=f"Threshold Example Model",
2010+
estimator_name=xgb_name,
2011+
calibrate=False,
2012+
model_type="classification",
2013+
estimator=xgb,
2014+
kfold=False,
2015+
stratify_y=True,
2016+
stratify_cols=False,
2017+
grid=tuned_parameters_xgb,
2018+
randomized_grid=False,
2019+
boost_early=False,
2020+
scoring=["roc_auc"],
2021+
random_state=222,
2022+
n_jobs=2,
2023+
)
2024+
2025+
2026+
In the grid_search_param_tuning use the f1_beta_tune variable when using
2027+
grid_search_param_tuning(). Set this to True to enable tuning.
2028+
2029+
.. code-block:: python
2030+
2031+
xgb_model.grid_search_param_tuning(X, y, f1_beta_tune=True)
2032+
2033+
This will find the best hyperparameters and then find the best threshold for these
2034+
balancing both precision and recall. The threshold is stored in the Model object.
2035+
To access this:
2036+
2037+
.. code-block:: python
2038+
2039+
xgb_model.threshold
2040+
2041+
This will give the best threshold found for each score specified in the Model object.
2042+
2043+
When using methods to return metrics or report metrics and an optimal threshold was used
2044+
make sure to remember to specify this in them for example:
2045+
2046+
.. code-block:: python
2047+
2048+
xgb_model.return_metrics(
2049+
X_valid,
2050+
y_valid,
2051+
optimal_threshold=True,
2052+
print_threshold=True,
2053+
model_metrics=True,
2054+
)
2055+
2056+
2057+
Example usage: custom betas (higher recall)
2058+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2059+
If we want to have a higher recall score and care less about precision then we increase
2060+
the beta value. This looks very similar to the previous example except that when we
2061+
use f1_beta_tune, we also set a beta value like so:
2062+
2063+
2064+
.. code-block:: python
2065+
2066+
xgb_model.grid_search_param_tuning(X, y, f1_beta_tune=True, betas=[2])
2067+
2068+
Setting the beta value to 2 will priortise increasing the recall over the precision.
2069+
2070+
2071+
2072+
Example usage: custom betas (higher precision)
2073+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2074+
If we want to have a higher precision score and care less about recall then we decrease
2075+
the beta value. This looks very similar to the previous example except that we set the
2076+
beta value to less than 1.
2077+
2078+
2079+
.. code-block:: python
2080+
2081+
xgb_model.grid_search_param_tuning(X, y, f1_beta_tune=True, betas=[0.5])
2082+
2083+
Setting the beta value to 0.5 will priortise increasing the precision over the recall.
2084+
19632085
Imbalanced Learning
19642086
------------------------
19652087

‎docs/_static/pygments.css‎

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@ span.linenos.special { color: #000000; background-color: #ffffc0; padding-left:
66
.highlight .hll { background-color: #ffffcc }
77
.highlight { background: #f8f8f8; }
88
.highlight .c { color: #3D7B7B; font-style: italic } /* Comment */
9-
.highlight .err { border: 1px solid #F00 } /* Error */
9+
.highlight .err { border: 1px solid #FF0000 } /* Error */
1010
.highlight .k { color: #008000; font-weight: bold } /* Keyword */
11-
.highlight .o { color: #666 } /* Operator */
11+
.highlight .o { color: #666666 } /* Operator */
1212
.highlight .ch { color: #3D7B7B; font-style: italic } /* Comment.Hashbang */
1313
.highlight .cm { color: #3D7B7B; font-style: italic } /* Comment.Multiline */
1414
.highlight .cp { color: #9C6500 } /* Comment.Preproc */
@@ -25,34 +25,34 @@ span.linenos.special { color: #000000; background-color: #ffffc0; padding-left:
2525
.highlight .gp { color: #000080; font-weight: bold } /* Generic.Prompt */
2626
.highlight .gs { font-weight: bold } /* Generic.Strong */
2727
.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */
28-
.highlight .gt { color: #04D } /* Generic.Traceback */
28+
.highlight .gt { color: #0044DD } /* Generic.Traceback */
2929
.highlight .kc { color: #008000; font-weight: bold } /* Keyword.Constant */
3030
.highlight .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */
3131
.highlight .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */
3232
.highlight .kp { color: #008000 } /* Keyword.Pseudo */
3333
.highlight .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */
3434
.highlight .kt { color: #B00040 } /* Keyword.Type */
35-
.highlight .m { color: #666 } /* Literal.Number */
35+
.highlight .m { color: #666666 } /* Literal.Number */
3636
.highlight .s { color: #BA2121 } /* Literal.String */
3737
.highlight .na { color: #687822 } /* Name.Attribute */
3838
.highlight .nb { color: #008000 } /* Name.Builtin */
39-
.highlight .nc { color: #00F; font-weight: bold } /* Name.Class */
40-
.highlight .no { color: #800 } /* Name.Constant */
41-
.highlight .nd { color: #A2F } /* Name.Decorator */
39+
.highlight .nc { color: #0000FF; font-weight: bold } /* Name.Class */
40+
.highlight .no { color: #880000 } /* Name.Constant */
41+
.highlight .nd { color: #AA22FF } /* Name.Decorator */
4242
.highlight .ni { color: #717171; font-weight: bold } /* Name.Entity */
4343
.highlight .ne { color: #CB3F38; font-weight: bold } /* Name.Exception */
44-
.highlight .nf { color: #00F } /* Name.Function */
44+
.highlight .nf { color: #0000FF } /* Name.Function */
4545
.highlight .nl { color: #767600 } /* Name.Label */
46-
.highlight .nn { color: #00F; font-weight: bold } /* Name.Namespace */
46+
.highlight .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */
4747
.highlight .nt { color: #008000; font-weight: bold } /* Name.Tag */
4848
.highlight .nv { color: #19177C } /* Name.Variable */
49-
.highlight .ow { color: #A2F; font-weight: bold } /* Operator.Word */
50-
.highlight .w { color: #BBB } /* Text.Whitespace */
51-
.highlight .mb { color: #666 } /* Literal.Number.Bin */
52-
.highlight .mf { color: #666 } /* Literal.Number.Float */
53-
.highlight .mh { color: #666 } /* Literal.Number.Hex */
54-
.highlight .mi { color: #666 } /* Literal.Number.Integer */
55-
.highlight .mo { color: #666 } /* Literal.Number.Oct */
49+
.highlight .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */
50+
.highlight .w { color: #bbbbbb } /* Text.Whitespace */
51+
.highlight .mb { color: #666666 } /* Literal.Number.Bin */
52+
.highlight .mf { color: #666666 } /* Literal.Number.Float */
53+
.highlight .mh { color: #666666 } /* Literal.Number.Hex */
54+
.highlight .mi { color: #666666 } /* Literal.Number.Integer */
55+
.highlight .mo { color: #666666 } /* Literal.Number.Oct */
5656
.highlight .sa { color: #BA2121 } /* Literal.String.Affix */
5757
.highlight .sb { color: #BA2121 } /* Literal.String.Backtick */
5858
.highlight .sc { color: #BA2121 } /* Literal.String.Char */
@@ -67,9 +67,9 @@ span.linenos.special { color: #000000; background-color: #ffffc0; padding-left:
6767
.highlight .s1 { color: #BA2121 } /* Literal.String.Single */
6868
.highlight .ss { color: #19177C } /* Literal.String.Symbol */
6969
.highlight .bp { color: #008000 } /* Name.Builtin.Pseudo */
70-
.highlight .fm { color: #00F } /* Name.Function.Magic */
70+
.highlight .fm { color: #0000FF } /* Name.Function.Magic */
7171
.highlight .vc { color: #19177C } /* Name.Variable.Class */
7272
.highlight .vg { color: #19177C } /* Name.Variable.Global */
7373
.highlight .vi { color: #19177C } /* Name.Variable.Instance */
7474
.highlight .vm { color: #19177C } /* Name.Variable.Magic */
75-
.highlight .il { color: #666 } /* Literal.Number.Integer.Long */
75+
.highlight .il { color: #666666 } /* Literal.Number.Integer.Long */

‎docs/about.html‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
99
<title>GitHub Repository &mdash; Model Tuner 0.0.23a0 documentation</title>
10-
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=b86133f3" />
10+
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
1111
<link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=e59714d7" />
1212
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
1313
<link rel="stylesheet" type="text/css" href="_static/custom.css?v=72ec8e44" />

‎docs/caveats.html‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
99
<title>Zero Variance Columns &mdash; Model Tuner 0.0.23a0 documentation</title>
10-
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=b86133f3" />
10+
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
1111
<link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=e59714d7" />
1212
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
1313
<link rel="stylesheet" type="text/css" href="_static/custom.css?v=72ec8e44" />

‎docs/changelog.html‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
99
<title>Changelog &mdash; Model Tuner 0.0.23a0 documentation</title>
10-
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=b86133f3" />
10+
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
1111
<link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=e59714d7" />
1212
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
1313
<link rel="stylesheet" type="text/css" href="_static/custom.css?v=72ec8e44" />

‎docs/genindex.html‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
<meta charset="utf-8" />
77
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
88
<title>Index &mdash; Model Tuner 0.0.23a0 documentation</title>
9-
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=b86133f3" />
9+
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
1010
<link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=e59714d7" />
1111
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
1212
<link rel="stylesheet" type="text/css" href="_static/custom.css?v=72ec8e44" />

‎docs/getting_started.html‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
99
<title>Welcome to Model Tuner’s Documentation! &mdash; Model Tuner 0.0.23a0 documentation</title>
10-
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=b86133f3" />
10+
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
1111
<link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=e59714d7" />
1212
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
1313
<link rel="stylesheet" type="text/css" href="_static/custom.css?v=72ec8e44" />

‎docs/index.html‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
99
<title>Model Tuner Documentation &mdash; Model Tuner 0.0.23a0 documentation</title>
10-
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=b86133f3" />
10+
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
1111
<link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=e59714d7" />
1212
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
1313
<link rel="stylesheet" type="text/css" href="_static/custom.css?v=72ec8e44" />
@@ -200,6 +200,13 @@ <h1>Model Tuner Documentation<a class="headerlink" href="#model-tuner-documentat
200200
<li class="toctree-l3"><a class="reference internal" href="usage_guide.html#step-10-calibrate-the-model-if-needed">Step 10: Calibrate the model (if needed)</a></li>
201201
</ul>
202202
</li>
203+
<li class="toctree-l2"><a class="reference internal" href="usage_guide.html#f1-beta-threshold-tuning">F1 Beta Threshold Tuning</a><ul>
204+
<li class="toctree-l3"><a class="reference internal" href="usage_guide.html#understanding-f1-beta-score">Understanding F1 Beta Score</a></li>
205+
<li class="toctree-l3"><a class="reference internal" href="usage_guide.html#example-usage-default-beta-1">Example usage: default (beta = 1)</a></li>
206+
<li class="toctree-l3"><a class="reference internal" href="usage_guide.html#example-usage-custom-betas-higher-recall">Example usage: custom betas (higher recall)</a></li>
207+
<li class="toctree-l3"><a class="reference internal" href="usage_guide.html#example-usage-custom-betas-higher-precision">Example usage: custom betas (higher precision)</a></li>
208+
</ul>
209+
</li>
203210
<li class="toctree-l2"><a class="reference internal" href="usage_guide.html#imbalanced-learning">Imbalanced Learning</a><ul>
204211
<li class="toctree-l3"><a class="reference internal" href="usage_guide.html#generating-an-imbalanced-dataset">Generating an imbalanced dataset</a></li>
205212
<li class="toctree-l3"><a class="reference internal" href="usage_guide.html#define-hyperparameters-for-xgboost">Define hyperparameters for XGBoost</a></li>

‎docs/objects.inv‎

1 Byte
Binary file not shown.

‎docs/references.html‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
99
<title>References &mdash; Model Tuner 0.0.23a0 documentation</title>
10-
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=b86133f3" />
10+
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=80d5e7a1" />
1111
<link rel="stylesheet" type="text/css" href="_static/css/theme.css?v=e59714d7" />
1212
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
1313
<link rel="stylesheet" type="text/css" href="_static/custom.css?v=72ec8e44" />

0 commit comments

Comments
 (0)