Skip to content

Commit c2ac6df

Browse files
akshay-rajanakshay-rajan
committed
Add SVM
1 parent 07b8073 commit c2ac6df

File tree

2 files changed

+164
-0
lines changed

2 files changed

+164
-0
lines changed

‎learn/templates/learn/svm.html‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
{% extends "learn/layout.html" %}
2+
{% load static %}
3+
4+
{% block title %}Support Vector Machines{% endblock %}
5+
6+
{% block body %}
7+
<div>
8+
<h1 class="chapter-title">Support Vector Machines (SVM)</h1>
9+
<div class="chapter-content">
10+
<p>
11+
<span class="dark">Support Vector Machines (SVM)</span> are supervised machine learning models used for classification and regression tasks. SVM is popular for classification problems and works by finding the hyperplane that best separates different classes of data. The hyperplane is chosen to maximize the margin between the two classes.
12+
</p>
13+
14+
<h2 class="chapter-subheading">Key Concepts</h2>
15+
<ul>
16+
<li>
17+
<span class="dark">Support Vectors:</span> Data points closest to the hyperplane. These influence the hyperplane's position and orientation.
18+
</li>
19+
<li>
20+
<span class="dark">Margin:</span> The distance between the hyperplane and the support vectors. SVM maximizes this margin.
21+
</li>
22+
<li>
23+
<span class="dark">Kernel Trick:</span> Enables SVM to handle non-linear boundaries by transforming data into a higher-dimensional space using functions like RBF or Polynomial kernels.
24+
</li>
25+
</ul>
26+
27+
<h2 class="chapter-subheading">Mathematical Intuition</h2>
28+
<p>
29+
The objective of SVM is to find a hyperplane in an \( N \)-dimensional space (where \( N \) is the number of features) that distinctly classifies the data points. The equation for the hyperplane is:
30+
</p>
31+
$$
32+
w^T x + b = 0
33+
$$
34+
<ul>
35+
<li>\( w \): Weight vector</li>
36+
<li>\( x \): Feature vector</li>
37+
<li>\( b \): Bias term</li>
38+
</ul>
39+
<p>
40+
For a point on the margin:
41+
</p>
42+
<ul>
43+
<li>\( w^T x + b = 1 \) for the positive class</li>
44+
<li>\( w^T x + b = -1 \) for the negative class</li>
45+
</ul>
46+
<p>
47+
The margin is computed as the perpendicular distance between these margins and the hyperplane. SVM maximizes this margin for better generalization.
48+
</p>
49+
50+
<h2 class="chapter-subheading">Implementation</h2>
51+
<div class="code-block">
52+
<pre><code class="language-python">from sklearn.datasets import make_classification
53+
from sklearn.model_selection import train_test_split
54+
from sklearn.svm import SVC
55+
from sklearn.metrics import accuracy_score, classification_report
56+
57+
# Generate synthetic data
58+
X, y = make_classification(random_state=42, n_samples=1000, n_features=2, n_informative=2, n_redundant=0)
59+
60+
# Train-test split
61+
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
62+
63+
# Train the SVM model
64+
model = SVC(kernel='linear')
65+
model.fit(X_train, y_train)
66+
67+
# Test the model
68+
y_pred = model.predict(X_test)
69+
accuracy = accuracy_score(y_test, y_pred)
70+
print("Accuracy:", accuracy)
71+
72+
# Classification report
73+
print(classification_report(y_test, y_pred))</code></pre>
74+
</div>
75+
<div class="output-block">
76+
<pre><code>Accuracy: 0.88
77+
Classification Report:
78+
precision recall f1-score support
79+
0 0.88 0.88 0.88 101
80+
1 0.88 0.88 0.88 99
81+
82+
accuracy 0.88 200
83+
macro avg 0.88 0.88 0.88 200
84+
weighted avg 0.88 0.88 0.88 200</code></pre>
85+
</div>
86+
87+
<h2 class="chapter-subheading">Hyperparameter Tuning</h2>
88+
<ul>
89+
<li>
90+
<span class="dark">C:</span> Regularization parameter controlling the trade-off between maximizing the margin and correctly classifying training points. Smaller \( C \) widens the margin but allows more misclassifications.
91+
</li>
92+
<li>
93+
<span class="dark">Kernel:</span> Determines the decision boundary shape. Common kernels include:
94+
<ul>
95+
<li><span class="dark">linear:</span> Linear hyperplane</li>
96+
<li><span class="dark">poly:</span> Polynomial kernel</li>
97+
<li><span class="dark">rbf:</span> Radial Basis Function (Gaussian)</li>
98+
</ul>
99+
</li>
100+
</ul>
101+
102+
<h2 class="chapter-subheading">Visualization</h2>
103+
<p>
104+
Visualization using two features:
105+
</p>
106+
<div class="code-block">
107+
<pre><code class="language-python">import numpy as np
108+
import matplotlib.pyplot as plt
109+
110+
# Create a mesh grid for visualization
111+
h = 0.02
112+
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
113+
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
114+
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
115+
116+
# Plot the decision boundary
117+
Z = model.decision_function(np.c_[xx.ravel(), yy.ravel()])
118+
Z = Z.reshape(xx.shape)
119+
120+
plt.scatter(X[:, 0], X[:, 1], c=y, cmap='viridis', alpha=0.6)
121+
plt.contourf(xx, yy, Z, alpha=0.5, levels=[-1, 0, 1], colors='k')
122+
plt.xlabel('Feature 1')
123+
plt.ylabel('Feature 2')
124+
plt.title('SVM Decision Boundary')
125+
plt.show()</code></pre>
126+
</div>
127+
<div class="img-div">
128+
<img src="{% static 'learn/img/svm_decision_boundary.png' %}" alt="SVM Decision Boundary" style="width:500px;">
129+
<span class="caption">SVM Decision Boundary</span>
130+
</div>
131+
132+
<h2 class="chapter-subheading">Soft Margin Formulation</h2>
133+
<p>
134+
Soft margin SVM allows certain misclassifications to ensure a wider margin, helping the model generalize better to unseen data. The \( C \) parameter controls the trade-off between margin width and misclassification.
135+
</p>
136+
137+
<h2 class="chapter-subheading">Errors in SVM</h2>
138+
<ul>
139+
<li><span class="dark">Classification Error:</span> When a data point is misclassified.</li>
140+
<li><span class="dark">Margin Error:</span> When a data point falls within the margin.</li>
141+
</ul>
142+
</div>
143+
144+
<div class="content-links d-flex justify-content-between">
145+
<a href="{% url 'chapter' 'kmeans' %}" class="content-link">
146+
<button class="btn btn-dark">
147+
Previous
148+
</button>
149+
</a>
150+
</div>
151+
<div class="other-links">
152+
<div class="hline"></div>
153+
<ul>
154+
<li><a href="{% url 'chapter' 'naive_bayes' %}">Naive Bayes</a></li>
155+
<li><a href="{% url 'chapter' 'decision_tree' %}">Decision Trees</a></li>
156+
</ul>
157+
</div>
158+
</div>
159+
{% endblock %}
160+
161+
{% block script %}
162+
<script src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js" async></script>
163+
{% endblock %}

‎learn/views.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
'linear_regression': 'learn/linear_regression.html',
1515
'regression_metrics': 'learn/regression_metrics.html',
1616
'kmeans': 'learn/kmeans.html',
17+
'svm': 'learn/svm.html',
1718
}
1819

1920
def index(request):

0 commit comments

Comments
 (0)