Fix header hierarchy in Supervised_Noanswers.ipynb

- Changed top-level header from H2 to H1
- Adjusted section headers from H3 to H2
- Adjusted subsection headers from H4 to H3

Resolves myst.header warnings about non-consecutive header levels.

Author: bernalde

13-Classification/Supervised_Noanswers.ipynb

@@ -14,7 +14,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Supervised Learning ##\n",
+    "# Supervised Learning\n",
     "Supervised learning is one of the four common branches of machine learning which is concerned with predicting future observations based on labeled data. In a typical application, we will need to choose the relevant descriptors to provide as input to our model (feature selection), select a model, and decide on an optimal training strategy.\n",
     "\n",
     "<b>feature selection</b>: this is the process of deciding on the inputs that will be used for training our model. In the most general case, we are seeking a model $f(\\mathbf{x})=\\mathbf{y}$, where $\\mathbf{x}$ is the set of features, and $\\mathbf{y}$ is the set of labels we are trying to predict and $f()$ is the model. The choice of x may seem obvious in many cases if we have relevant domain information (e.g., physical laws that underly target phenomena), but in many cases it will be guided by data availability and trial and error. \n",
@@ -27,7 +27,7 @@
     "\n",
     "Supervised learning is the most common branch of machine learning and it underlies many commercial uses of ML. Today we will cover two examples <b>support vector machines</b> and <b>neural networks</b>.\n",
     "\n",
-    "### Support Vector Machines ###\n",
+    "## Support Vector Machines\n",
     "<b>Support vector machines</b> are a common model for supervised classification tasks. After covering the details you will also see on a variation of SVM could be used for unsupervised classification, which provides a nice bridge between the supervised and unsupervised learning lectures. \n",
     "\n",
     "The object SVMs is to identify the optimal boundary that separates two or more classes of objects with respect to the descrptors. That is, an SVM, like all supervised classification models attempts to solve the problem $\\mathbf{y} = f(\\mathbf{x})$ such that given a future x, we predict the correct class y to apply to the observation. In this case, SVM draws a <i>linear</i> boundary between the two classes. That is, if you plot the data classes with respect to <b>x</b>, the SVM solution is represented by the \"best\" straight line that divides them. The ability to draw such a boundary will clearly depend on your choice of <b>x</b>! \n",
@@ -42,7 +42,7 @@
     "\n",
     "4. <b>What is a support vector?</b> Since all a SVM is trying to do is find a boundary, it turns out the optimal solution only depends on the samples that are at the edge of the domain. The vectorial positions of these boundary data are the \"support vectors\" and hence the name of the model. \n",
     "\n",
-    "#### Sample Datasets ####\n",
+    "### Sample Datasets\n",
     "We'll use two of the datasets from the unsupervised learning lecture for illustrating the basics of SVMs. Critically we will keep the labels here, instead of hiding them like we did for the unsupervised demonstrations. `data1` will serve as our linearly separable example, `data2` will be the multiclass example, and `data3` will serve as our non-linear example. \n"
    ]
   },
@@ -142,7 +142,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### scikit-learn example ###\n",
+    "### scikit-learn example\n",
     "The cell below demonstrates how to use the scikit-learn `SVC` object to perform linear classification:"
    ]
   },
@@ -198,7 +198,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Multitask Example ####\n",
+    "### Multitask Example\n",
     "In scikit learn training SVMs for multitask assignment is almost identical to the two class case:"
    ]
   },
@@ -256,10 +256,10 @@
    "source": [
     "<b>Note:</b> It is messy to try and draw the boundaries so that they don't overlap. \n",
     "\n",
-    "#### Kernel Trick ####\n",
+    "### Kernel Trick\n",
     "The so-called \"kernel trick\" is used for linearizing non-linear problems. In simple terms this let's you determine curved boundaries with SVMs. In principle these are very expensive to calculate, but by applying the transform virtually (via the kernel) these curved boundaries can be calculated very effectively. The results will depend on the specific Kernel that you use. \n",
     "\n",
-    "#### Activity 1 - Kernel Trick ####\n",
+    "### Activity 1 - Kernel Trick\n",
     "In the cell below I have written the code for performing a linear prediction of the classification task on our \"bullseye\" dataset. It doesn't work very well (as we expect). \n",
     "\n",
     "1. Look up the documentation for the ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’ kernel options. \n",
@@ -302,7 +302,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Artificial Neural Networks ###\n",
+    "## Artificial Neural Networks\n",
     "The fundamental unit in a neural network is the \"artificial neuron\". These artificial neurons are the mathematical analog of their biological counterparts. These neurons link to other neurons, and \"fire\" depending on if their inputs rise above a certain threshold. A neural network can be made up of thousands of thes neurons connected in different ways to one another. The basic parameters in these models are associated with the weights for each neural connection (i.e., a scalar that scales the connection between a given pair of neurons), and the bias of the activation function (i.e., the threshhold necessary to switch the neuron). \n",
     "\n",
     "In neural network models there is a direct mapping between features via the input layer and labels via the output layer. Contrast this with the case of  support vector machines, where the model only fit a boundary, then predictions were based on which side of the boundary they resided. Any additional layers that are included between the input and output layers are referred to as \"hidden\" (partly because they are not easily interpretable). Models with hidden layers are called \"deep\", and their unique proficiency in mapping complex relationships has created the special term \"deep learning\". Although neural networks can in principle be arranged in any network structure, implementation is aided by some common architectures that have emerged (e.g., feed-forward and convolutional networks).\n",
@@ -311,7 +311,7 @@
     "\n",
     "Several specialized libraries have emerged to support the development of neural network based models. Here we will be using the `Keras` python API with a `tensorflow` backend.\n",
     "\n",
-    "#### Accuracy vs Loss ####\n",
+    "### Accuracy vs Loss\n",
     "As objective functions become more complex, we will want to distinguish between the value of the objective function and the performance on the desired prediction task. Specifically we have the following definitions:\n",
     "\n",
     "<b>loss</b>: this is the value of the objective function that our training algorithm is trying to minimize.\n",
@@ -320,7 +320,7 @@
     "\n",
     "It can be counterintuitive at first, but these two aren't as tightly coupled as you might expect. For instance, even in regression we started adding regularization terms to the objective function which made it difficult to interpret. \n",
     "\n",
-    "#### Problem Context ####\n",
+    "### Problem Context\n",
     "You work as a quality engineer for a company that builds pumps for industrial applications. As part of your job, you perform failure testing in order to determine if the pumps that are produced are in or out of specifications. However, this testing is a destructive process, and in an effort to increase the cost savings you wonder if there is a better method for quality control.\n",
     "\n",
     "At your disposal is a dataset consisting of measurements taken on 150 different pumps as part of quality control testing, as well as the result of the testing process. Is it possible to utilize machine learning to leverage this data and provide accurate failure predictions?\n",
@@ -507,7 +507,7 @@
    "source": [
     "We expect our pump to operate within a certain set of tolerances. If the inlet diameter is too small or too large, it can adversely affect the pump capacity. Within the acceptable inlet diameter range, we can accept some fluctuation in capacity. If the maximum head (analagous to maximum pressure) is very high but the minimum thickness of the pump is too small, the pump will fail during testing.\n",
     "\n",
-    "#### Activity 2 - Calculate Statistics ####\n",
+    "### Activity 2 - Calculate Statistics\n",
     "1. Use the cell below to calculate the mean and standard deviation of each feature. \n",
     "2. Make plots of the diameter vs capacity and head vs thickness."
    ]
@@ -529,7 +529,7 @@
     "\n",
     "<b>Q2</b>: Do the features have the same scale? If not, what should we do before building our model?\n",
     "\n",
-    "#### Activity 3 - Standardize Data ####\n",
+    "### Activity 3 - Standardize Data\n",
     "In the cell below standardize the features and replot the results"
    ]
   },
@@ -546,7 +546,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Training and Testing Split ####\n",
+    "### Training and Testing Split\n",
     "\n",
     "When training a neural network, we would like to use as much data as possible so that our model can learn on as representative a dataset as possible. However, it is also vital to ensure that the model is capable of accurate predictions on data that it has not seen before, as this matches how the model will be applied in practice. \n",
     "\n",
@@ -567,7 +567,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Single Layer Neural Network ####\n",
+    "### Single Layer Neural Network\n",
     "For this model, we will utilize a single, fully connected layer of 3 units. That is, each input parameter is fed to each of the 3 units. The units do no commnicate amongst each other. Each will use some linear combination of the inputs fed to them multiplied by a weighting factor, and will mediate their output through a nonlinear activation function. Here, because we are considering a classification task, we utilize the 'softmax' activation function. This will output a set of normalized probability values ( i.e. 0.3,0.5,0.2 ) corresponding to the probability that a given class has been observed.\n",
     "\n",
     "Such models are trained through a process called backpropagation, whereby the error in the network output is used to calculate derivatives of the error with respect to the layer weights. These derivatives are used to update the weights using stochastic gradient descent (SGD) in an iterative fashion to reduce the error. Because we are dealing with a classification task with three possible outputs (Pass,Rework,Fail) we utilize the categorical crossentropy loss function, which penalizes output distributions that diverge from the input distribution. We use an 'accuracy' metric to see how accurate the model is in predicting the input class. \n",
@@ -640,10 +640,10 @@
     "<b>Q1</b>: We see the validation loss start to increase in this example. What does this mean?\n",
     "<b>Q2</b>: What is the difference between loss and accuracy?\n",
     "\n",
-    "#### Multi-Layer Perceptron Model ####\n",
+    "### Multi-Layer Perceptron Model\n",
     "What if we add in an additional layer between input and output? This \"hidden\" layer allows a newtork to, in principle, approximate any continuous function. By adding in more than one hidden layer, we create a 'deep' neural network, one capable of learning latent features within a dataset an performing highly nonlinear classifications/regression tasks. \n",
     "\n",
-    "#### Activity 4 - Compile a Multilayer Perceptron Model ####\n",
+    "### Activity 4 - Compile a Multilayer Perceptron Model\n",
     "Based on the earlier example, try and compile a multilayer perceptron model in the cell below with the same input and output layers as below, but adding two dense hidden layers with 8 units each and a 'relu' activation function. \n",
     "\n",
     "I have copied the earlier example into the cell as a starting point:"