Merge pull request #393 from apphp/392-convert-layers-to-numpower

392 convert layers to numpower

Author: apphp

docs/neural-network/hidden-layers/activation.md

@@ -1,4 +1,4 @@
-<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/Activation.php">[source]</a></span>
+<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/Activation/Activation.php">[source]</a></span>
 
 # Activation
 Activation layers apply a user-defined non-linear activation function to their inputs. They often work in conjunction with [Dense](dense.md) layers as a way to transform their output.
@@ -10,8 +10,8 @@ Activation layers apply a user-defined non-linear activation function to their i
 
 ## Example
 ```php
-use Rubix\ML\NeuralNet\Layers\Activation;
-use Rubix\ML\NeuralNet\ActivationFunctions\ReLU;
+use Rubix\ML\NeuralNet\Layers\Activation\Activation;
+use Rubix\ML\NeuralNet\ActivationFunctions\ReLU\ReLU;
 
 $layer = new Activation(new ReLU());
-```
+```

docs/neural-network/hidden-layers/batch-norm.md

@@ -1,4 +1,4 @@
-<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/BatchNorm.php">[source]</a></span>
+<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/BatchNorm/BatchNorm.php">[source]</a></span>
 
 # Batch Norm
 Batch Norm layers normalize the activations of the previous layer such that the mean activation is *close* to 0 and the standard deviation is *close* to 1. Adding Batch Norm reduces the amount of covariate shift within the network which makes it possible to use higher learning rates and thus converge faster under some circumstances.
@@ -12,12 +12,12 @@ Batch Norm layers normalize the activations of the previous layer such that the
 
 ## Example
 ```php
-use Rubix\ML\NeuralNet\Layers\BatchNorm;
-use Rubix\ML\NeuralNet\Initializers\Constant;
-use Rubix\ML\NeuralNet\Initializers\Normal;
+use Rubix\ML\NeuralNet\Layers\BatchNorm\BatchNorm;
+use Rubix\ML\NeuralNet\Initializers\Constant\Constant;
+use Rubix\ML\NeuralNet\Initializers\Normal\Normal;
 
 $layer = new BatchNorm(0.7, new Constant(0.), new Normal(1.));
 ```
 
 ## References
-[^1]: S. Ioffe et al. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.
+[^1]: S. Ioffe et al. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.

docs/neural-network/hidden-layers/dense.md

@@ -1,4 +1,4 @@
-<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/Dense.php">[source]</a></span>
+<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/Dense/Dense.php">[source]</a></span>
 
 # Dense
 Dense (or *fully connected*) hidden layers are layers of neurons that connect to each node in the previous layer by a parameterized synapse. They perform a linear transformation on their input and are usually followed by an [Activation](activation.md) layer. The majority of the trainable parameters in a standard feed forward neural network are contained within Dense hidden layers.
@@ -14,9 +14,9 @@ Dense (or *fully connected*) hidden layers are layers of neurons that connect to
 
 ## Example
 ```php
-use Rubix\ML\NeuralNet\Layers\Dense;
+use Rubix\ML\NeuralNet\Layers\Dense\Dense;
 use Rubix\ML\NeuralNet\Initializers\He;
 use Rubix\ML\NeuralNet\Initializers\Constant;
 
 $layer = new Dense(100, 1e-4, true, new He(), new Constant(0.0));
-```
+```

docs/neural-network/hidden-layers/dropout.md

@@ -1,4 +1,4 @@
-<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/Dropout.php">[source]</a></span>
+<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/Dropout/Dropout.php">[source]</a></span>
 
 # Dropout
 Dropout is a regularization technique to reduce overfitting in neural networks by preventing complex co-adaptations on training data. It works by temporarily disabling output nodes during each training pass. It also acts as an efficient way of performing model averaging with the parameters of neural networks.
@@ -10,10 +10,10 @@ Dropout is a regularization technique to reduce overfitting in neural networks b
 
 ## Example
 ```php
-use Rubix\ML\NeuralNet\Layers\Dropout;
+use Rubix\ML\NeuralNet\Layers\Dropout\Dropout;
 
 $layer = new Dropout(0.2);
 ```
 
 ## References
-[^1]: N. Srivastava et al. (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting.
+[^1]: N. Srivastava et al. (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting.

docs/neural-network/hidden-layers/noise.md

@@ -1,4 +1,4 @@
-<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/Noise.php">[source]</a></span>
+<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/Noise/Noise.php">[source]</a></span>
 
 # Noise
 This layer adds random Gaussian noise to the inputs with a user-defined standard deviation. Noise added to neural network activations acts as a regularizer by indirectly adding a penalty to the weights through the cost function in the output layer.
@@ -10,10 +10,10 @@ This layer adds random Gaussian noise to the inputs with a user-defined standard
 
 ## Example
 ```php
-use Rubix\ML\NeuralNet\Layers\Noise;
+use Rubix\ML\NeuralNet\Layers\Noise\Noise;
 
 $layer = new Noise(1e-3);
 ```
 
 ## References
-[^1]: C. Gulcehre et al. (2016). Noisy Activation Functions.
+[^1]: C. Gulcehre et al. (2016). Noisy Activation Functions.

docs/neural-network/hidden-layers/placeholder1d.md

@@ -0,0 +1,17 @@
+<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/Placeholder1D/Placeholder1D.php">[source]</a></span>
+
+# Placeholder 1D
+
+The Placeholder 1D input layer represents the future input values of a mini batch (matrix) of single dimensional tensors (vectors) to the neural network. It performs shape validation on the input and then forwards it unchanged to the next layer.
+
+## Parameters
+| # | Name | Default | Type | Description |
+|---|---|---|---|---|
+| 1 | inputs | | int | The number of input nodes (features). |
+
+## Example
+```php
+use Rubix\ML\NeuralNet\Layers\Placeholder1D\Placeholder1D;
+
+$layer = new Placeholder1D(10);
+```

docs/neural-network/hidden-layers/prelu.md

@@ -1,4 +1,4 @@
-<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/PReLU.php">[source]</a></span>
+<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/PReLU/PReLU.php">[source]</a></span>
 
 # PReLU
 Parametric Rectified Linear Units are leaky rectifiers whose *leakage* coefficient is learned during training. Unlike standard [Leaky ReLUs](../activation-functions/leaky-relu.md) whose leakage remains constant, PReLU layers can adjust the leakage to better suite the model on a per node basis.
@@ -14,8 +14,8 @@ $$
 
 ## Example
 ```php
-use Rubix\ML\NeuralNet\Layers\PReLU;
-use Rubix\ML\NeuralNet\Initializers\Normal;
+use Rubix\ML\NeuralNet\Layers\PReLU\PReLU;
+use Rubix\ML\NeuralNet\Initializers\Normal\Normal;
 
 $layer = new PReLU(new Normal(0.5));
 ```

docs/neural-network/hidden-layers/swish.md

@@ -1,4 +1,4 @@
-<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/Swish.php">[source]</a></span>
+<span style="float:right;"><a href="https://github.com/RubixML/ML/blob/master/src/NeuralNet/Layers/Swish/Swish.php">[source]</a></span>
 
 # Swish
 Swish is a parametric activation layer that utilizes smooth rectified activation functions. The trainable *beta* parameter allows each activation function in the layer to tailor its output to the training set by interpolating between the linear function and ReLU.
@@ -10,8 +10,8 @@ Swish is a parametric activation layer that utilizes smooth rectified activation
 
 ## Example
 ```php
-use Rubix\ML\NeuralNet\Layers\Swish;
-use Rubix\ML\NeuralNet\Initializers\Constant;
+use Rubix\ML\NeuralNet\Layers\Swish\Swish;
+use Rubix\ML\NeuralNet\Initializers\Constant\Constant;
 
 $layer = new Swish(new Constant(1.0));
 ```

phpunit.xml

@@ -83,5 +83,6 @@
   </testsuites>
   <php>
     <env name="ENV" value="testing"/>
+    <ini name="memory_limit" value="256M"/>
   </php>
 </phpunit>

src/NeuralNet/Initializers/He/HeNormal.php

@@ -35,7 +35,7 @@ public function initialize(int $fanIn, int $fanOut) : NDArray
 
         $stdDev = sqrt(2 / $fanOut);
 
-        return NumPower::truncatedNormal(size: [$fanOut, $fanIn], scale: $stdDev);
+        return NumPower::truncatedNormal(size: [$fanOut, $fanIn], loc: 0.0, scale: $stdDev);
     }
 
     /**