-
Notifications
You must be signed in to change notification settings - Fork 8
Expand file tree
/
Copy pathREADME.Rmd
More file actions
112 lines (76 loc) · 4.68 KB
/
Copy pathREADME.Rmd
File metadata and controls
112 lines (76 loc) · 4.68 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
<!-- rmarkdown v1 -->
```{r, echo = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-"
)
```
# oddstream <img src="man/figures/logo.png" align="right" height="150"/>
[](https://www.repostatus.org/#wip)
[](https://www.gnu.org/licenses/gpl-3.0.en.html)
[](https://travis-ci.org/pridiltal/oddstream)
---
[](https://cran.r-project.org/)
[](https://cran.r-project.org/package=oddstream)
[](commits/master)
---
[)`-yellowgreen.svg)](/commits/master)
---
The goal of oddstream (Outlier Detection in Data Streams) is to propose a framework that provides real time support for early detection of anomalous series within a large collection of streaming time series data. By definition, anomalies are rare in comparison to a system's typical behaviour. We define an anomaly as an observation that is very unlikely given the forecast distribution. The algorithm first forecasts a boundary for the system's typical behaviour using a representative sample of the typical behaviour of the system. An approach based on extreme value theory is used for this boundary prediction process. Then a sliding window is used to test for anomalous series within the newly arrived collection of series. Feature based representation of time series is used as the input to the model. To cope with concept drift, the forecast boundary for the system's typical behaviour is updated periodically.
More details regarding the algorithm can be found in our paper [Anomaly Detection in Streaming Nonstationary Temporal Data](https://www.tandfonline.com/doi/abs/10.1080/10618600.2019.1617160)
## Installation
You can install oddstream from github with:
```{r gh-installation, eval = FALSE}
# install.packages("devtools")
devtools::install_github("pridiltal/oddstream")
```
## Example
### Multivariate time series plot of `anomalous_stream` dataset available in `oddstream` package
```{r dataset, echo=TRUE, eval=TRUE, warning=FALSE, message=FALSE, out.width= "100%", fig.height=3}
library(oddstream)
library(tidyverse)
t <- nrow(anomalous_stream)
f <- ncol(anomalous_stream)
data <- as_tibble(anomalous_stream) %>%
gather() %>%
mutate(key = rep((1:f), each = t), Time = rep(1:t, f)) %>%
setNames(c("Cable", "Value", "Time"))
p <- data %>%
ggplot(aes(x = Time, y = Cable, fill = Value)) +
geom_tile() +
scale_fill_gradientn(colours = c("#F0E442", "#000000", "#000000"), values = c(0, .1, max(anomalous_stream))) +
scale_x_continuous(breaks = seq(0, 1400, 200)) +
scale_y_continuous(breaks = seq(0, 600, 100)) +
labs(x= "Time", y= "Time Series ID")
p
```
### Two dimensional feature space
```{r 2dpcspace}
library(oddstream)
features <- extract_tsfeatures(anomalous_stream[1:100, 1:100])
pc <- get_pc_space(features)
p <- gg_featurespace(pc)
p
# Perform a 2D kernel density estimation using MASS::kde2d() and display the results with contours.
p + ggplot2::geom_density_2d()
```
```{r example, eval = FALSE, echo = TRUE}
library(oddstream)
# Considers the first window of the data set as the training set and the remaining as the test stream
train_data <- anomalous_stream[1:100, ]
test_stream <- anomalous_stream[101:1456, ]
output <- find_odd_streams(train_data, test_stream, trials = 100)
```
### Further Details
```{r help, echo = TRUE, eval= FALSE}
?oddstream
```
## References
- Clifton, D. A., Hugueny, S., & Tarassenko, L. (2011). Novelty detection with multivariate extreme value statistics. Journal of signal processing systems, 65 (3), (pp. 371-389).
- Hyndman, R. J., Wang, E., & Laptev, N. (2015). Large-scale unusual time series detection. In 2015 IEEE International Conference on Data Mining Workshop (ICDMW), (pp. 1616-1619). IEEE.
- Talagala, P. D., Hyndman, R. J., Smith-Miles, K., et al. (2018). Anomaly Detection in Streaming Nonstationary Temporal Data. Anomaly detection in streaming nonstationary temporal data. Journal of Computational and Graphical Statistics, (just-accepted), 1-28. DOI:10.1080/10618600.2019.1617160.