Skip to content

Commit 54bb19e

Browse files
author
M.Notter
committed
Updates ML/DL blog posts
1 parent 17696fa commit 54bb19e

35 files changed

+1061
-78
lines changed

.gitignore

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,4 +8,6 @@
88
.tweet-cache
99
Gemfile.lock
1010
_site
11-
vendor
11+
vendor
12+
model_backup*
13+
history_log*

_posts/2023-10-23-01_scikit_simple.md

Lines changed: 16 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -9,26 +9,30 @@ This post is part of a comprehensive machine learning series that takes you from
99

1010
1. **Getting Started with Machine Learning** (Current Post)
1111
Basic classification using Scikit-learn with the MNIST dataset
12+
([View code]({{ site.baseurl }}/scripts/01_scikit_simple.py))
1213

1314
2. **Deep Learning Fundamentals**
1415
Introduction to neural networks using TensorFlow
16+
([View code]({{ site.baseurl }}/scripts/02_tensorflow_simple.py))
1517

1618
3. **Advanced Machine Learning**
1719
Complex regression pipelines with Scikit-learn
20+
([View code]({{ site.baseurl }}/scripts/03_scikit_advanced.py))
1821

1922
4. **Advanced Deep Learning**
2023
Sophisticated neural network architectures in TensorFlow
24+
([View code]({{ site.baseurl }}/scripts/04_tensorflow_advanced.py))
2125

2226
Each tutorial builds upon concepts from previous posts while introducing new techniques and best practices. Whether you're new to machine learning or looking to expand your skills, this series provides hands-on experience with real-world datasets and modern ML tools.
2327

2428
Have you ever wondered how to get started with machine learning? This series of posts will guide you through practical implementations using two of Python's most popular frameworks: Scikit-learn and TensorFlow. Whether you're a beginner looking to understand the basics or an experienced developer wanting to refresh your knowledge, we'll progress from basic classification tasks to more advanced regression problems.
2529

2630
The series consists of four parts:
2731

28-
1. **[Getting Started with Classification using Scikit-learn](../blog/2023/01_scikit_simple)** (You are here)<br>Introduction to machine learning basics using the MNIST dataset
29-
2. **[Basic Neural Networks with TensorFlow](../blog/2023/02_tensorflow_simple)** (Part 2)<br>Building your first neural network for image classification
30-
3. **[Advanced Machine Learning with Scikit-learn](../blog/2023/03_scikit_advanced)** (Part 3)<br>Exploring complex regression problems and model optimization
31-
4. **[Advanced Neural Networks with TensorFlow](../blog/2023/04_tensorflow_advanced)** (Part 4)<br>Implementing sophisticated neural network architectures
32+
1. **[Getting Started with Classification using Scikit-learn]({{ site.baseurl }}/blog/2023/01_scikit_simple)** (You are here)<br>Introduction to machine learning basics using the MNIST dataset
33+
2. **[Basic Neural Networks with TensorFlow]({{ site.baseurl }}/blog/2023/02_tensorflow_simple)** (Part 2)<br>Building your first neural network for image classification
34+
3. **[Advanced Machine Learning with Scikit-learn]({{ site.baseurl }}/blog/2023/03_scikit_advanced)** (Part 3)<br>Exploring complex regression problems and model optimization
35+
4. **[Advanced Neural Networks with TensorFlow]({{ site.baseurl }}/blog/2023/04_tensorflow_advanced)** (Part 4)<br>Implementing sophisticated neural network architectures
3236

3337
### Why These Tools?
3438

@@ -89,7 +93,7 @@ for ax, image, label in zip(axes.ravel(), digits.images, digits.target):
8993
```
9094

9195
<div style="text-align: center">
92-
<img class="img-fluid rounded z-depth-1" src="../assets/ex_plots/01_scikit_digits_sample.png" data-zoomable width=600px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
96+
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/01_scikit_digits_sample.png" data-zoomable width=600px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
9397
<div class="caption">
9498
Figure 1: Sample of MNIST digits showing different handwritten numbers from 0-9. Each image is an 8x8 pixel grayscale representation.
9599
</div>
@@ -228,7 +232,7 @@ plt.close()
228232
```
229233

230234
<div style="text-align: center">
231-
<img class="img-fluid rounded z-depth-1" src="../assets/ex_plots/01_scikit_rf_heatmap.png" data-zoomable width=500px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
235+
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/01_scikit_rf_heatmap.png" data-zoomable width=500px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
232236
<div class="caption">
233237
Figure 2: Heatmap showing model accuracy (%) for different combinations of SVM hyperparameters gamma and C. Darker colors indicate better performance.
234238
</div>
@@ -316,7 +320,7 @@ plt.close()
316320
```
317321

318322
<div style="text-align: center">
319-
<img class="img-fluid rounded z-depth-1" src="../assets/ex_plots/01_scikit_svm_heatmap.png" data-zoomable width=500px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
323+
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/01_scikit_svm_heatmap.png" data-zoomable width=500px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
320324
<div class="caption">
321325
Figure 3: Confusion matrix showing the model's prediction performance across all digit classes. Diagonal elements represent correct predictions.
322326
</div>
@@ -375,7 +379,7 @@ plt.close()
375379
```
376380

377381
<div style="text-align: center">
378-
<img class="img-fluid rounded z-depth-1" src="../assets/ex_plots/01_scikit_confusion_matrix.png" data-zoomable width=500px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
382+
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/01_scikit_confusion_matrix.png" data-zoomable width=500px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
379383
<div class="caption">
380384
Figure 4: Feature importance heatmap showing which pixels in the 8x8 grid contribute most to the Random Forest's classification decisions.
381385
</div>
@@ -405,7 +409,7 @@ plt.close()
405409
```
406410

407411
<div style="text-align: center">
408-
<img class="img-fluid rounded z-depth-1" src="../assets/ex_plots/01_scikit_feature_importance.png" data-zoomable width=500px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
412+
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/01_scikit_feature_importance.png" data-zoomable width=500px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
409413
<div class="caption">
410414
Figure 5: Most confidently predicted digits from the test set, showing examples where the model has highest prediction probabilities.
411415
</div>
@@ -432,7 +436,7 @@ plt.savefig('../assets/ex_plots/01_scikit_confident_predictions.png', bbox_inche
432436
plt.close()
433437
```
434438

435-
<img class="img-fluid rounded z-depth-1" src="../assets/ex_plots/01_scikit_confident_predictions.png" data-zoomable width=800px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
439+
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/01_scikit_confident_predictions.png" data-zoomable width=800px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
436440
<div class="caption">
437441
Figure 6: Most confidently predicted digits from the test set, showing examples where the model has highest prediction probabilities.
438442
</div>
@@ -449,7 +453,7 @@ plt.savefig('../assets/ex_plots/01_scikit_uncertain_predictions.png', bbox_inche
449453
plt.close()
450454
```
451455

452-
<img class="img-fluid rounded z-depth-1" src="../assets/ex_plots/01_scikit_uncertain_predictions.png" data-zoomable width=800px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
456+
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/01_scikit_uncertain_predictions.png" data-zoomable width=800px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
453457
<div class="caption">
454458
Figure 7: Most challenging digits for the model to predict, showing examples where the model has lowest prediction confidence.
455459
</div>
@@ -580,4 +584,4 @@ In the next post, we'll tackle the same MNIST classification problem using Tenso
580584

581585
In Part 2, we'll explore how neural networks approach the same problem using TensorFlow, introducing deep learning concepts and comparing the two approaches.
582586

583-
[Continue to Part 2 →](../blog/2023/02_tensorflow_simple)
587+
[Continue to Part 2 →]({{ site.baseurl }}/blog/2023/02_tensorflow_simple)

_posts/2023-10-23-02_tensorflow_simple.md

Lines changed: 23 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ description: Building your first neural network for image classification
88

99
In this second part of our machine learning series, we'll implement the same MNIST classification task using [TensorFlow](https://www.tensorflow.org/). While Scikit-learn excels at classical machine learning, TensorFlow shines when building neural networks. We'll see how deep learning approaches differ from traditional methods and learn the basic concepts of neural network architecture.
1010

11+
The complete code for this tutorial can be found in the [02_tensorflow_simple.py]({{ site.baseurl }}/scripts/02_tensorflow_simple.py) script.
12+
1113
### Why Neural Networks?
1214

1315
While our Scikit-learn models performed well in Part 1, neural networks offer several key advantages for image classification:
@@ -155,29 +157,27 @@ and the number of trainable and non-trainable parameters.
155157
model.summary()
156158
```
157159

158-
Model: "sequential_1"
160+
Model: "sequential"
159161
_________________________________________________________________
160-
Layer (type) Output Shape Param #
162+
Layer (type) Output Shape Param #
161163
=================================================================
162-
conv2d_2 (Conv2D) (None, 26, 26, 32) 320
163-
re_lu (ReLU) (None, 26, 26, 32) 0
164-
max_pooling2d_2 (MaxPooling (None, 13, 13, 32) 0
165-
2D)
166-
conv2d_3 (Conv2D) (None, 11, 11, 64) 18496
167-
re_lu_1 (ReLU) (None, 11, 11, 64) 0
168-
max_pooling2d_3 (MaxPooling (None, 5, 5, 64) 0
169-
2D)
170-
flatten_1 (Flatten) (None, 1600) 0
171-
dropout_2 (Dropout) (None, 1600) 0
172-
dense_2 (Dense) (None, 32) 51232
173-
re_lu_2 (ReLU) (None, 32) 0
174-
dropout_3 (Dropout) (None, 32) 0
175-
dense_3 (Dense) (None, 10) 330
176-
softmax (Softmax) (None, 10) 0
164+
conv2d (Conv2D) (None, 26, 26, 32) 320
165+
re_lu (ReLU) (None, 26, 26, 32) 0
166+
max_pooling2d (MaxPooling2D) (None, 13, 13, 32) 0
167+
conv2d_1 (Conv2D) (None, 11, 11, 64) 18496
168+
re_lu_1 (ReLU) (None, 11, 11, 64) 0
169+
max_pooling2d_1 (MaxPooling2D) (None, 5, 5, 64) 0
170+
flatten (Flatten) (None, 1600) 0
171+
dropout (Dropout) (None, 1600) 0
172+
dense (Dense) (None, 32) 51232
173+
re_lu_2 (ReLU) (None, 32) 0
174+
dropout_1 (Dropout) (None, 32) 0
175+
dense_1 (Dense) (None, 10) 330
176+
softmax (Softmax) (None, 10) 0
177177
=================================================================
178-
Total params: 70,378
179-
Trainable params: 70,378
180-
Non-trainable params: 0
178+
Total params: 70,378 (274.91 KB)
179+
Trainable params: 70,378 (274.91 KB)
180+
Non-trainable params: 0 (0.00 Byte)
181181
_________________________________________________________________
182182

183183
This summary tells us several important things:
@@ -286,7 +286,7 @@ axs[1].set_ylabel("Accuracy")
286286
plt.show()
287287
```
288288

289-
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/ex_03_tensorflow_simple_output_16_0.png" data-zoomable width=800px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
289+
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/02_tensorflow_training_history.png" data-zoomable width=800px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
290290
<div class="caption">
291291
Figure 1: Training metrics over time showing model loss (left) and Mean Absolute Error (right) for both training and validation sets. The logarithmic scale helps visualize improvement across different magnitudes.
292292
</div>
@@ -330,7 +330,7 @@ plt.show()
330330
```
331331

332332
<div style="text-align: center">
333-
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/ex_03_tensorflow_simple_output_22_0.png" data-zoomable width=500px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
333+
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/02_tensorflow_confusion_matrix.png" data-zoomable width=500px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
334334
</div><br>
335335

336336
## 5. Model parameters
@@ -357,7 +357,7 @@ plt.tight_layout()
357357
plt.show()
358358
```
359359

360-
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/ex_03_tensorflow_simple_output_24_0.png" data-zoomable width=800px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
360+
<img class="img-fluid rounded z-depth-1" src="{{ site.baseurl }}/assets/ex_plots/02_tensorflow_conv_kernels.png" data-zoomable width=800px style="padding-top: 20px; padding-right: 20px; padding-bottom: 20px; padding-left: 20px">
361361

362362
### Common Deep Learning Pitfalls
363363
When starting with TensorFlow and neural networks, watch out for these common issues:

_posts/2023-10-23-03_scikit_advanced.md

Lines changed: 13 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ description: Exploring complex regression problems and preprocessing pipelines
88

99
In this third part of our series, we'll explore more sophisticated machine learning techniques using [Scikit-learn](https://scikit-learn.org/stable/). While Parts 1 and 2 focused on classification, we'll now tackle regression problems and learn how to build complex preprocessing pipelines. We'll use the California Housing dataset to demonstrate these concepts.
1010

11+
The complete code for this tutorial can be found in the [03_scikit_advanced.py]({{ site.baseurl }}/scripts/03_scikit_advanced.py) script.
12+
1113
**Note**: The purpose of this post is to highlight the flexibility and capabilities of scikit-learn's advanced features. Therefore, this tutorial focuses on introducing you to those advanced routines rather than creating the optimal regression model.
1214

1315
### Why Advanced Preprocessing?
@@ -437,16 +439,13 @@ import pandas as pd
437439

438440
df_res = pd.DataFrame(res.cv_results_)
439441
df_res = df_res.iloc[:, ~df_res.columns.str.contains('time|split[0-9]*|rank|params')]
440-
new_columns = [
441-
c.split('param_regressor__')[1] if 'param_regressor' in c else c
442-
for c in df_res.columns
443-
]
444-
new_columns = [
445-
c.split('preprocessor__')[1] if 'preprocessor__' in c else c for c in new_columns
446-
]
442+
new_columns = [c.split('param_regressor__')[1] if 'param_regressor' in c else c for c in df_res.columns]
443+
new_columns = [c.split('preprocessor__')[1] if 'preprocessor__' in c else c for c in new_columns]
447444
df_res.columns = new_columns
448445
df_res = df_res.sort_values('mean_test_score', ascending=False)
449-
df_res.head(10)
446+
447+
print("\nTop 10 parameter combinations:")
448+
print(df_res.head(10))
450449
```
451450

452451
| :-------------: | :------------------: | :-----------------------------------------: | :-----------------------------: | :-------------------------------------: | :-----------------------------------------: | :--------------------------------: | :-------------------------------------------: | :-------------------------------------: | :----------------------------------------: | :----------------: | :---------------: | :-----------------: | :----------------: |
@@ -480,17 +479,13 @@ Prediction accuracy on test data: {score_te*100:.2f}%"
480479
)
481480
```
482481

483-
Prediction accuracy on train data: 7.10%
484-
Prediction accuracy on test data: 8.38%
485-
486-
# Add this interpretation
487482
Let's interpret these regression metrics in practical terms:
488-
- **Train Error (7.10%)**: On average, predictions deviate by 7.10% from true house prices
489-
- For a $300,000 house, this means predictions are typically within ±$21,300
490-
- **Test Error (8.38%)**: Slightly higher error on unseen data
491-
- For a $300,000 house, predictions are typically within ±$25,140
492-
- **Error Difference (1.28%)**: Small gap indicates good generalization
493-
- **Context**: For house price prediction, ~8% error is relatively good considering market volatility
483+
- **Train Error**: On average, predictions deviate by about 7-8% from true house prices
484+
- For a $300,000 house, this means predictions are typically within ±$21,000-24,000
485+
- **Test Error**: Slightly higher error on unseen data
486+
- For a $300,000 house, predictions are typically within ±$24,000-27,000
487+
- **Error Difference**: Small gap indicates good generalization
488+
- **Context**: For house price prediction, ~8-9% error is relatively good considering market volatility
494489

495490
Great, the score seems reasonably good! But now that we know better which preprocessing routine seems to be the
496491
best (thanks to `RandomizedSearchCV`), let's go ahead and further fine-tune the ridge model.

0 commit comments

Comments
 (0)