Skip to content

mnist convolutional example

Jump to bottom
Tobias Kind edited this page Nov 21, 2015 · 30 revisions

The mnist/convolutional.py example is directly taken from the tensorflow/models/image example folder. It will train a neural net based on the NIST handwriting data set. More specifically a LeNet-5 like convolutional neural network. The Google TensorFlow tutorials contain beginners and expert code for better understanding of the machinery behind.

Before we can start we need to download the python file from the repository or best clone the repository locally.

git clone https://tensorflow.googlesource.com/release

After the installation or download of the tensorflow repository a new subdirectory with the name data will be created. This is were four files will be installed. The data is taken from http://yann.lecun.com/exdb/mnist/ which also nicely explains the details behind this data set.

-rw-r--r-- 1 vm vm 1648877 Nov 20 18:59 t10k-images-idx3-ubyte.gz
-rw-r--r-- 1 vm vm    4542 Nov 20 18:59 t10k-labels-idx1-ubyte.gz
-rw-r--r-- 1 vm vm 9912422 Nov 20 18:59 train-images-idx3-ubyte.gz
-rw-r--r-- 1 vm vm   28881 Nov 20 18:59 train-labels-idx1-ubyte.gz

Then we can change to the tensorflow directory and call
python tensorflow/models/image/mnist/convolutional.py

vm@ubuntu:~/tensorflow$python tensorflow/models/image/mnist/convolutional.py
Extracting data/train-images-idx3-ubyte.gz
Extracting data/train-labels-idx1-ubyte.gz
Extracting data/t10k-images-idx3-ubyte.gz
Extracting data/t10k-labels-idx1-ubyte.gz
I tensorflow/core/common_runtime/local_device.cc:25] Local device intra op parallelism threads: 8
I tensorflow/core/common_runtime/local_session.cc:45] Local session inter op parallelism threads: 8
Initialized!
Epoch 0.00
Minibatch loss: 12.054, learning rate: 0.010000
Minibatch error: 90.6%
Validation error: 84.6%

We can see now that tensorflow starts training and that the CPU use is quite interesting with 50% use of the 8 threads (4 CPU cores) and a sudden up-shoot to maximum utilization. Because the program is running in a Oracle VM VirtualBox using Ubuntu 13 this could be an artifact and can only be confirmed when running natively without a virtual machine.

tensorflow-mnist-memory-footprint-cpu-only

Epoch 9.77
Minibatch loss: 1.596, learning rate: 0.006302
Minibatch error: 0.0%
Validation error: 0.9%
Epoch 9.89
Minibatch loss: 1.602, learning rate: 0.006302
Minibatch error: 0.0%
Validation error: 0.9%
Test error: 0.8%

real	34m16.645s
user	104m20.376s
sys     33m22.867s
vm@ubuntu:~/tensorflow$ 

```

We can see that benchmarking the **convolutional.py** using a three year old Core i7-2600K CPU (4.2 Ghz) and some random DDR memory with tensorflow in CPU-only mode (no CUDA or OPENCL) will take around **34 minutes** to finalize the example. 


----

**Links**

[MNIST data set](http://yann.lecun.com/exdb/mnist/) - THE MNIST DATABASE of handwritten digits

[LeNet 5 NN](http://yann.lecun.com/exdb/lenet/) - Short explanation of a LeNet 5 neural network

[convolutional.py](https://tensorflow.googlesource.com/tensorflow/+/master/tensorflow/models/image/mnist/convolutional.py) - The above discussed example hosted at Google Git

[MNIST For ML Beginners ](http://tensorflow.org/tutorials/mnist/beginners/index.md) - Introduction to the MNIST hand writing problem

[Deep MNIST for Experts ](http://tensorflow.org/tutorials/mnist/pros/index.md) - TensorFlow tutorial for the MNIST expert

Clone this wiki locally