|
| 1 | + |
| 2 | +# ANN2 |
| 3 | + |
1 | 4 | [](https://www.gnu.org/licenses/gpl-3.0.en.html) |
2 | 5 | [](https://cran.r-project.org/package=ANN2) |
3 | 6 |  |
4 | 7 |  |
5 | 8 | [](https://codecov.io/gh/bflammers/ANN2) |
6 | 9 |
|
7 | | -# ANN2 |
8 | 10 | Artificial Neural Networks package for R |
9 | 11 |
|
10 | | -Training of neural networks for classification and regression tasks using mini-batch gradient descent. Special features include a function for training autoencoders, which can be used to detect anomalies, and some related plotting functions. Multiple activation functions are supported, including tanh, relu, step and ramp. For the use of the step and ramp activation functions in detecting anomalies using autoencoders, see Hawkins et al. (2002). Furthermore, several loss functions are supporterd, including robust ones such as Huber and pseudo-Huber loss, as well as L1 and L2 regularization. The possible options for optimization algorithms are RMSprop, Adam and SGD with momentum. The package contains a vectorized C++ implementation that facilitates fast training through mini-batch learning. |
| 12 | +This package allows to train neural networks for classification and regression tasks, as well as autoencoders for anomaly detection. Several helper and plotting functions are included for improved usability and understanding what the model does. ANN2 contains a vectorized neural net implementation in C++ that facilitates fast training through mini-batch gradient descent. |
| 13 | + |
| 14 | +ANN2 contains the following features: |
| 15 | +* Easy to use interface - defining and training neural nets with a single function call! |
| 16 | +* Activation functions: tanh, sigmoid, relu, linear, ramp, step |
| 17 | +* Loss functions: log, squared, absolute, huber, pseudo-huber |
| 18 | +* Regularization: L1, L2 |
| 19 | +* Optimizers: sgd, sgd w/ momentum, RMSprop, ADAM |
| 20 | +* Ploting functions for visualizing encodings, reconstructions and loss (training and validation) |
| 21 | +* Helper functions for predicting, reconstructing, encoding and decoding |
| 22 | +* Reading and writing the trained model from / to disk |
| 23 | +* Access to model parameters and low-level Rcpp module methods |
| 24 | + |
| 25 | +## Usage |
| 26 | + |
| 27 | +Defining and training a multilayer neural network with ANN2 is done using a single function call to either: |
| 28 | +* `neuralnetwork()` - for a multilayer neural net for classification or regression, |
| 29 | +* `autoencoder()` - for an autoencoding neural network that is trained to reconstruct its inputs. |
| 30 | + |
| 31 | +Below are two examples with minimal code snippets that show how to use these functions. |
| 32 | + |
| 33 | +### `neuralnetwork()` |
| 34 | + |
| 35 | +We'll train a neural network with dimensions 4 x 5 x 5 x 3 on the Iris data set that classifies whether each observation (sepal length and width and petal length and width measurements for three species of floweres) belongs to class setosa, versicolor or virginica. The dimensions of the input and output layers are inferred from the data, the hidden layer dimensions are defined by providing a single vector that specifies the number of nodes for each hidden layer as argument `hidden.layers`. |
| 36 | + |
| 37 | +``` r |
| 38 | +library(ANN2) |
| 39 | + |
| 40 | +# Prepare test and train sets |
| 41 | +random_idx <- sample(1:nrow(iris), size = 145) |
| 42 | +X_train <- iris[random_idx, 1:4] |
| 43 | +y_train <- iris[random_idx, 5] |
| 44 | +X_test <- iris[setdiff(1:nrow(iris), random_idx), 1:4] |
| 45 | +y_test <- iris[setdiff(1:nrow(iris), random_idx), 5] |
| 46 | + |
| 47 | +# Train neural network on classification task |
| 48 | +NN <- neuralnetwork(X = X_train, |
| 49 | + y = y_train, |
| 50 | + hidden.layers = c(5, 5), |
| 51 | + optim.type = 'adam', |
| 52 | + n.epochs = 5000) |
| 53 | + |
| 54 | +# Predict the class for new data points |
| 55 | +predict(NN, X_test) |
| 56 | + |
| 57 | +# $predictions |
| 58 | +# [1] "setosa" "setosa" "setosa" "versicolor" "versicolor" |
| 59 | +# |
| 60 | +# $probabilities |
| 61 | +# class_setosa class_versicolor class_virginica |
| 62 | +# [1,] 0.9998184126 0.0001814204 1.670401e-07 |
| 63 | +# [2,] 0.9998311154 0.0001687264 1.582390e-07 |
| 64 | +# [3,] 0.9998280223 0.0001718171 1.605735e-07 |
| 65 | +# [4,] 0.0001074780 0.9997372337 1.552883e-04 |
| 66 | +# [5,] 0.0001077757 0.9996626441 2.295802e-04 |
| 67 | + |
| 68 | +# Plot the training and validation loss |
| 69 | +plot(NN) |
| 70 | +``` |
| 71 | + |
| 72 | + |
| 73 | + |
| 74 | +You can interact with the resulting `ANN` object using methods `plot()`, `print()` and `predict()`. Storing and loading the model to/from disk can be done using `write_ANN()` and `read_ANN()`, respectively. Other, more low-level, methods of the C++ module can be accessed through the `$` operator as members of the object, eg. `NN$Rcpp_ANN$getParams()` for getting the parameters (weight matrices and bias vectors) from the trained model. |
| 75 | + |
| 76 | +### `autoencoder()` |
| 77 | + |
| 78 | +The `autoencoder()` function allows to train an autoencoding neural network. In the next example we'll train an autoencoder of dimension 4 x 10 x 2 x 10 x 4 on the USArrests data set. The middle hidden layer acts as a bottleneck that forces the autoencoder to only retain structural variation and discard random variation. By denoting data points that are poorly reconstructed (high reconstruction error) as aberant, we exploit this denoising property for anomaly detection. |
| 79 | + |
| 80 | +``` r |
| 81 | +# Prepare test and train sets |
| 82 | +random_idx <- sample(1:nrow(USArrests), size = 45) |
| 83 | +X_train <- USArrests[random_idx,] |
| 84 | +X_test <- USArrests[setdiff(1:nrow(USArrests), random_idx),] |
| 85 | + |
| 86 | +# Define and train autoencoder |
| 87 | +AE <- autoencoder(X = X_train, |
| 88 | + hidden.layers = c(10,3,10), |
| 89 | + loss.type = 'pseudo-huber', |
| 90 | + optim.type = 'adam', |
| 91 | + n.epochs = 5000) |
| 92 | + |
| 93 | +# Reconstruct test data |
| 94 | +reconstruct(AE, X_test) |
| 95 | + |
| 96 | +# $reconstructed |
| 97 | +# Murder Assault UrbanPop Rape |
| 98 | +# [1,] 8.547431 243.85898 75.60763 37.791746 |
| 99 | +# [2,] 12.972505 268.68226 65.40411 29.475545 |
| 100 | +# [3,] 2.107441 78.55883 67.75074 15.040075 |
| 101 | +# [4,] 2.085750 56.76030 55.32376 9.346483 |
| 102 | +# [5,] 12.936357 252.09209 56.24075 24.964715 |
| 103 | +# |
| 104 | +# $anomaly_scores |
| 105 | +# [1] 398.926431 247.238111 11.613522 0.134633 1029.806121 |
| 106 | + |
| 107 | +# Plot original points (grey) and reconstructions (colored) for training data |
| 108 | +reconstruction_plot(AE, X_train) |
| 109 | +``` |
| 110 | + |
| 111 | + |
| 112 | + |
| 113 | +In the reconstruction plot we see the original points (grey) along with their reconstructions (color scale based on reconstruction error), connected to each other by a grey line. The length of the line denotes the reconstruction error. |
| 114 | + |
| 115 | +You can interact with the `ANN` object that results from the `autoencoder()` function call using various methods, including `plot()`, `encode()`, `decode()` and `reconstruct()`. |
| 116 | + |
| 117 | +More details on supported arguments to `neuralnetwork()` and `autoencoder()`, as well as examples and explanations on using the helper and plotting functions can be found in the manual. |
11 | 118 |
|
12 | 119 | *** |
13 | 120 |
|
0 commit comments