You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Autencoders are an approach to use nearual networks to distill data into it's most important features, thereby compressing the data. We will be following the [Keras tutorial](https://blog.keras.io/building-autoencoders-in-keras.html) on the topic, which goes much more in depth and breadth than we will here. You are highly encouraged to check out that tutorial if you want to learn about autoencoders in the general sense.
22
+
Autencoders are an approach to use nearual networks to distill data into it's most important features, thereby compressing the data.
23
+
We will be following the [Keras tutorial](https://blog.keras.io/building-autoencoders-in-keras.html) on the topic, which goes much more in depth and breadth than we will here.
24
+
You are highly encouraged to check out that tutorial if you want to learn about autoencoders in the general sense.
23
25
24
26
## Table of contents
25
27
@@ -28,6 +30,7 @@ Autencoders are an approach to use nearual networks to distill data into it's mo
28
30
*[3. Define Keras Model](#3.-Define-Keras-Model)
29
31
*[4. Training](#4.-Training)
30
32
*[5. Explore Results](#5.-Explore-Results)
33
+
*[6. Deep AutoEncoder](#6.-Deep-AutoEncoder)
31
34
32
35
## 1. Setup
33
36
@@ -73,99 +76,110 @@ print(x_test.shape)
73
76
74
77
## 3. Define Keras Model
75
78
76
-
We will be defining a very simple autencoder. We define _three_ model building methods:
79
+
We will be defining a very simple autencoder. We define _three_ model architectures:
77
80
78
-
1.One to build a full end-to-end autoencoder.
79
-
2.One to create a model that includes only the encoder portion.
80
-
3.One that creates a model that includes only the decoder portion.
81
+
1.An encoder: a series of densly connected layers culminating in an "output" layer that determines the encoding dimensions.
82
+
2.A decoder: takes the output of the encoder as it's input and reconstructs the original data.
83
+
3.An autoencoder: a chain of the encoder and decoder that directly connects them for training purposes.
81
84
82
85
The only variable we give our model is the encoding dimensions, which will be a hyperparemter of our final transformer.
83
86
84
-
```python
85
-
from tensorflow import keras
87
+
The encoder and decoder are views to the first/last layers of the autoencoder model.
88
+
They'll be directly used in `transform` and `inverse_transform`, so we'll create some SciKeras models with those layers
89
+
and save them as in `encoder_model_` and `decoder_model_`. All three models are created within `_keras_build_fn`.
86
90
91
+
For a background on chaining Functional Models like this, see [All models are callable](https://keras.io/guides/functional_api/#all-models-are-callable-just-like-layers) in the Keras docs.
Next we create a class that that will enable the `transform` and `fit_transform` methods, as well as integrating all three of our models into a single estimator.
# at this point, encoder_model_ and decoder_model_
145
+
# are both "fitted" because they share layers w/ model_
146
+
# which is fit in the above call
145
147
returnself
146
148
147
-
deftransform(self, X):
148
-
X =self.feature_encoder_.transform(X)
149
-
X_tf =self.tf_est_.predict(X)
150
-
return X_tf
151
-
152
-
definverse_transform(self, X_tf):
153
-
X =self.inv_tf_est_.predict(X_tf)
154
-
X =self.feature_encoder_.inverse_transform(X)
155
-
return X
149
+
defscore(self, X) -> float:
150
+
# Note: we use 1-MSE as the score
151
+
# With MSE, "larger is better", but Scikit-Learn
152
+
# always maximizes the score (e.g. in GridSearch)
153
+
return1- mean_squared_error(self.predict(X), X)
154
+
155
+
deftransform(self, X) -> np.ndarray:
156
+
X: np.ndarray =self.feature_encoder_.transform(X)
157
+
returnself.encoder_model_.predict(X)
158
+
159
+
definverse_transform(self, X_tf: np.ndarray):
160
+
X: np.ndarray =self.decoder_model_.predict(X_tf)
161
+
returnself.feature_encoder_.inverse_transform(X)
156
162
```
157
163
158
-
Next, we wrap the Keras Model with Scikeras. Note that for our encoder/decoder estimators, we do not need to provide a loss function since no training will be done. We do however need to have the `fit_model` and `encoding_dim` so that these will be settable by `BaseWrapper.set_params`.
164
+
Next, we wrap the Keras Model with Scikeras. Note that for our encoder/decoder estimators, we do not need to provide a loss function since no training will be done.
165
+
We do however need to have the `fit_model` and `encoding_dim` so that these will be settable by `BaseWrapper.set_params`.
To train the model, we pass the input images as both the features and the target. This will train the layers to compress the data as accurately as possible between the encoder and decoder. Note that we only pass the `X` parameter, since we defined the mapping `y=X` in `KerasTransformer.fit` above.
180
+
To train the model, we pass the input images as both the features and the target.
181
+
This will train the layers to compress the data as accurately as possible between the encoder and decoder.
182
+
Note that we only pass the `X` parameter, since we defined the mapping `y=X` in `KerasTransformer.fit` above.
169
183
170
184
```python
171
185
_ = autoencoder.fit(X=x_train)
@@ -208,8 +222,77 @@ What about the compression? Let's check the sizes of the arrays.
print(f"x_test size (in MB): {x_test.nbytes/1024**2:.2f}")
226
+
print(f"encoded_imgs size (in MB): {encoded_imgs.nbytes/1024**2:.2f}")
213
227
cr =round((encoded_imgs.nbytes/x_test.nbytes), 2)
214
228
print(f"Compression ratio: 1/{1/cr:.0f}")
215
229
```
230
+
231
+
## 6. Deep AutoEncoder
232
+
233
+
234
+
We can easily expand our model to be a deep autoencoder by adding some hidden layers. All we have to do is add a parameter `hidden_layer_sizes` and use it in `_keras_build_fn` to build hidden layers.
235
+
For simplicity, we use a single `hidden_layer_sizes` parameter and mirror it across the encoding layers and decoding layers, but there is nothing forcing us to build symetrical models.
236
+
237
+
```python
238
+
from typing import List
239
+
240
+
241
+
classDeepAutoEncoder(AutoEncoder):
242
+
"""A class that enables transform and fit_transform.
print("1-MSE for training set (higher is better)\n")
289
+
score = autoencoder.score(X=x_test)
290
+
print(f"AutoEncoder: {score:.4f}")
291
+
292
+
score = deep.score(X=x_test)
293
+
print(f"Deep AutoEncoder: {score:.4f}")
294
+
```
295
+
296
+
Suprisingly, our score got worse. It's possible that that because of the extra trainable variables, our deep model trains slower than our simple model.
297
+
298
+
Check out the [Keras tutorial](https://blog.keras.io/building-autoencoders-in-keras.html) to see the difference after 100 epochs of training, as well as more architectures and applications for AutoEncoders!
0 commit comments