Skip to content

Commit 90353dc

Browse files
frazanesallen12nicholasloveday
authored
Extend documentation (#94)
* add theory documentation * add readthedocs config * add readthedocs files * delete obsolete files * update gitignore * update docs requirements * use docs requirements file * update docs requirements * update docs requirements * misc fixes and test cross-references * remove old file from index * install the local package code for the build * expand API reference documentation * delete mkdocs config file * wip on crps docstrings * more wip on crps docstrings * pass doctest for examples in _crps.py * more work on docstrings - math blocks, args, formatting, etc. * add examples and pass doctest for variogram scores * comment out latex math hook * update theory docs * add documentation page for ensemble forecasts * add documentation page for ensemble forecasts * add documentation page for weighted scoring rules * update docu index * update axis naming args (#95) * change ens member axis to m_axis * change categorical axis to k_axis * update mixture components axis name * fix m_axis bug * change test_kernels to use m_axis argument * fix m_axis bug in logs_ensemble * increase N in crps ensemble tests * add exception in crps ensemble tests for akr approximations * change energy score function name * change variogram score function name * add default threshold arguments to twcrps_ensemble * add default threshold arguments to owcrps_ensemble * add default threshold arguments to vrcrps_ensemble * reduce tolerance on owgksuv_ensemble test * add default threshold argument to weighted gksuv scores * update default w_func in vrcrps_ensemble * update test_wcrps * reduce tolerance of owgks tests * reduce tolerance of owgks tests * update documentation link and prepare release tag * add deprecated aliases * more minor fixes --------- Co-authored-by: sallen12 <[email protected]> Co-authored-by: Nicholas Loveday <[email protected]>
1 parent 966f802 commit 90353dc

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

67 files changed

+5073
-2134
lines changed

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,7 @@ instance/
7171
# Sphinx documentation
7272
docs/_build/
7373
docs/build/
74+
_build*
7475

7576
# PyBuilder
7677
.pybuilder/
@@ -151,3 +152,5 @@ scratch_nbs/
151152
_devlog/
152153
tests/output
153154
.devcontainer/
155+
docs/generated
156+
scoringrules/vendored

‎.pre-commit-config.yaml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,8 @@ repos:
2626
hooks:
2727
- id: codespell
2828

29-
- repo: https://github.com/frazane/check-latex-math
30-
rev: v0.2.2
31-
hooks:
32-
- id: latex-math-validation
33-
args: ["scoringrules/", "docs/", "*.py", "*.md"]
29+
#- repo: https://github.com/frazane/check-latex-math
30+
# rev: v0.2.2
31+
# hooks:
32+
# - id: latex-math-validation
33+
# args: ["scoringrules/", "docs/", "*.py", "*.md"]

‎.readthedocs.yaml‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,13 +10,13 @@ build:
1010
tools:
1111
python: "3.13"
1212

13-
# Build documentation with Mkdocs
14-
mkdocs:
15-
configuration: mkdocs.yaml
13+
# Build documentation in the "docs/" directory with Sphinx
14+
sphinx:
15+
configuration: docs/conf.py
1616

1717
# Optionally, but recommended,
1818
# declare the Python requirements required to build your documentation
1919
# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
20-
# python:
21-
# install:
22-
# - requirements: docs/requirements.txt
20+
python:
21+
install:
22+
- requirements: docs/requirements.txt

‎README.md‎

Lines changed: 33 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,38 @@
1+
<!-- centered images -->
12
<p align="center">
2-
<img src="docs/assets/images/banner_light.svg#gh-light-mode-only" />
3-
<img src="docs/assets/images/banner_dark.svg#gh-dark-mode-only" />
3+
<img src="docs/_static/banner_dark_long.svg#gh-dark-mode-only" alt="dark banner">
4+
<img src="docs/_static/banner_light_long.svg#gh-light-mode-only" alt="light banner">
45
</p>
56

6-
<br>
7+
# Probabilistic forecast evaluation
78

89
[![CI](https://github.com/frazane/scoringrules/actions/workflows/ci.yaml/badge.svg?branch=main)](https://github.com/frazane/scoringrules/actions/workflows/ci.yaml)
910
[![codecov](https://codecov.io/github/frazane/scoringrules/graph/badge.svg?token=J194X4HEBH)](https://codecov.io/github/frazane/scoringrules)
1011
[![pypi](https://img.shields.io/pypi/v/scoringrules.svg?colorB=<brightgreen>)](https://pypi.python.org/pypi/scoringrules/)
1112

12-
Scoringrules is a python library for evaluating probabilistic forecasts by computing
13-
scoring rules and other diagnostic quantities. It aims to assist forecasting practitioners by
14-
providing a set of tools based the scientific literature and via its didactic approach.
13+
`scoringrules` is a python library that provides scoring rules to evaluate probabilistic forecasts.
14+
It's original goal was to reproduce the functionality of the R package
15+
[`scoringRules`](https://cran.r-project.org/web/packages/scoringRules/index.html) in python,
16+
thereby allowing forecasting practitioners working in python to enjoy the same tools as those
17+
working in R. The methods implemented in `scoringrules` are therefore based around those
18+
available in `scoringRules`, which are rooted in the scientific literature on probabilistic forecasting.
19+
20+
The scoring rules available in `scoringrules` include, but are not limited to, the
21+
- Brier Score
22+
- Logarithmic Score
23+
- (Discrete) Ranked Probability Score
24+
- Continuous Ranked Probability Score (CRPS)
25+
- Dawid-Sebastiani Score
26+
- Energy Score
27+
- Variogram Score
28+
- Gaussian Kernel Score
29+
- Threshold-Weighted CRPS
30+
- Threshold-Weighted Energy Score
31+
1532

1633
## Features
1734

18-
- **Fast** computations of several probabilistic univariate and multivariate verification metrics
35+
- **Fast** computation of several probabilistic univariate and multivariate verification metrics
1936
- **Multiple backends**: support for numpy (accelerated with numba), jax, pytorch and tensorflow
2037
- **Didactic approach** to probabilistic forecast evaluation through clear code and documentation
2138

@@ -28,7 +45,7 @@ pip install scoringrules
2845

2946
## Documentation
3047

31-
Learn more about scoringrules in its official documentation at https://frazane.github.io/scoringrules/.
48+
Learn more about `scoringrules` in its official documentation at https://scoringrules.readthedocs.io/en/latest/.
3249

3350

3451
## Quick example
@@ -41,26 +58,22 @@ fct = obs[:,None] + np.random.randn(100, 21) * 0.1
4158
sr.crps_ensemble(obs, fct)
4259
```
4360

44-
## Metrics
45-
- Brier Score
46-
- Continuous Ranked Probability Score (CRPS)
47-
- Logarithmic score
48-
- Error Spread Score
49-
- Energy Score
50-
- Variogram Score
51-
52-
5361
## Citation
54-
If you found this library useful for your own research, consider citing:
62+
If you found this library useful, consider citing:
5563

5664
```
5765
@software{zanetta_scoringrules_2024,
5866
author = {Francesco Zanetta and Sam Allen},
59-
title = {Scoringrules: a python library for probabilistic forecast evaluation},
67+
title = {scoringrules: a python library for probabilistic forecast evaluation},
6068
year = {2024},
6169
url = {https://github.com/frazane/scoringrules}
6270
}
6371
```
6472

6573
## Acknowledgements
66-
[scoringRules](https://cran.r-project.org/web/packages/scoringRules/index.html) served as a reference for this library. The authors did an outstanding work which greatly facilitated ours. The implementation of the ensemble-based metrics as jit-compiled numpy generalized `ufuncs` was first proposed in [properscoring](https://github.com/properscoring/properscoring), released under Apache License, Version 2.0.
74+
- The widely-used R package [`scoringRules`](https://cran.r-project.org/web/packages/scoringRules/index.html)
75+
served as a reference for this library, which greatly facilitated our work. We are additionally
76+
grateful for fruitful discussions with the authors.
77+
- The quality of this library has also benefited a lot from discussions with (and contributions from)
78+
Nick Loveday and Tennessee Leeuwenburg, whose python library [`scores`](https://github.com/nci/scores)
79+
similarly provides a comprehensive collection of forecast evaluation methods.

‎docs/Makefile‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Minimal makefile for Sphinx documentation
2+
#
3+
4+
# You can set these variables from the command line, and also
5+
# from the environment for the first two.
6+
SPHINXOPTS ?=
7+
SPHINXBUILD ?= sphinx-build
8+
SOURCEDIR = .
9+
BUILDDIR = _build
10+
11+
# Put it first so that "make" without argument is like "make help".
12+
help:
13+
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
14+
15+
.PHONY: help Makefile
16+
17+
# Catch-all target: route all unknown targets to Sphinx using the new
18+
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
19+
%: Makefile
20+
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

‎docs/_static/banner_dark.svg‎

Lines changed: 20 additions & 0 deletions
Loading

‎docs/_static/banner_dark_long.svg‎

Lines changed: 19 additions & 0 deletions
Loading

‎docs/_static/banner_light.svg‎

Lines changed: 20 additions & 0 deletions
Loading

‎docs/_static/banner_light_long.svg‎

Lines changed: 19 additions & 0 deletions
Loading

‎docs/_static/favicon.ico‎

8.06 KB
Binary file not shown.

0 commit comments

Comments
 (0)