> For the complete documentation index, see [llms.txt](https://alham-rizvi.gitbook.io/alhamrizvi/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://alham-rizvi.gitbook.io/alhamrizvi/ai-college-notes/practical-2-a-support-vector-machine-svm-on-the-iris-dataset.md). # Practical 2(A) - Support Vector Machine (SVM) on the Iris Dataset **Language used:** Python 3, with the **scikit-learn** (`sklearn`) machine learning library. **Aim:** Build a Support Vector Machine classifier that identifies which species of Iris flower a sample belongs to, based on its measurements. *** ### 5.1 Before the code: what is an SVM, in plain English Imagine you have red dots and blue dots scattered on a piece of paper, and you want to draw a single straight line that separates the reds from the blues as cleanly as possible. A **Support Vector Machine** is an algorithm that finds the *best possible* line (or curve) to do this. "Best" here specifically means: the line that leaves the **widest possible gap** (called the margin) between itself and the closest points of each colour. Those closest points, the ones that "support" or define where the line goes, are called the **support vectors**, which is where the algorithm gets its name. ```mermaid flowchart LR subgraph Simple 2D example A((Class A points)) --- M[Widest gap / margin] M --- B((Class B points)) end ``` #### What if the data cannot be separated by a straight line? Real data is often messy and tangled together, not neatly separable by a straight line. SVM solves this using something called the **kernel trick**. A kernel is a mathematical function that reshapes the data into a higher dimension, where a straight separating line becomes possible again, even though the original data looked tangled in 2D or 3D. The kernel used in this practical is `'rbf'` (Radial Basis Function), one of the most popular kernels. It is very good at drawing curved, flexible boundaries instead of just straight lines. ```mermaid flowchart TD Data[Tangled data in original space] -->|apply RBF kernel| Reshaped[Reshaped into higher dimension] Reshaped -->|now separable| Line[A clean separating boundary can be drawn] ``` #### Multi-class classification SVM was originally designed to separate only two classes (red vs blue). The Iris dataset has **three** species, not two. Scikit-learn handles this automatically behind the scenes, by internally training multiple two-class SVMs and combining their votes, so we do not need to write any extra code for this ourselves. *** ### 5.2 The dataset: Iris The Iris dataset is one of the most famous beginner datasets in all of machine learning. It contains **150 flower samples**, each with **4 measurements**: | Feature | Meaning | | ------------ | ------------------------------------------------ | | Sepal length | Length of the outer leaf-like part of the flower | | Sepal width | Width of that same part | | Petal length | Length of the flower's petal | | Petal width | Width of the petal | Each sample belongs to one of **3 species** (encoded as numbers, not text): | Label | Species | | ----- | ---------- | | 0 | Setosa | | 1 | Versicolor | | 2 | Virginica | Our job is to train a model that looks at the 4 measurements of a new, unseen flower, and correctly predicts which of the 3 species it belongs to. *** ### 5.3 The full code ```python from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler from sklearn.svm import SVC from sklearn.metrics import accuracy_score, classification_report, confusion_matrix iris = load_iris() x = iris.data y = iris.target x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=42) scaler = StandardScaler() x_train_scaled = scaler.fit_transform(x_train) x_test_scaled = scaler.transform(x_test) svm = SVC(kernel='rbf', C=1) svm.fit(x_train, y_train) y_pred = svm.predict(x_test) accuracy = accuracy_score(y_test, y_pred) print(f'Accuracy: {accuracy:.3f}') print('Classification Report:') print(classification_report(y_test, y_pred)) print('Confusion Matrix:') print(confusion_matrix(y_test, y_pred)) ``` *** ### 5.4 Line by line explanation #### Imports ```python from sklearn.datasets import load_iris ``` Scikit-learn ships with a handful of famous toy datasets built in, so you don't have to download a CSV file yourself. This loads the Iris dataset directly. ```python from sklearn.model_selection import train_test_split ``` The standard tool for splitting data into a training portion and a testing portion. ```python from sklearn.preprocessing import StandardScaler ``` A tool that rescales numeric features so they are on a comparable scale. Explained in detail in section 5.5. ```python from sklearn.svm import SVC ``` `SVC` stands for Support Vector Classifier, scikit-learn's implementation of SVM for classification tasks. ```python from sklearn.metrics import accuracy_score, classification_report, confusion_matrix ``` Three different tools for judging how good the model is, explained in section 5.7. #### Loading the data ```python iris = load_iris() x = iris.data y = iris.target ``` * `iris.data` is a table of 150 rows by 4 columns, the measurements described in section 5.2. This becomes `x`, our features. * `iris.target` is a list of 150 numbers (0, 1, or 2), the correct species label for each row. This becomes `y`, our labels. #### Splitting the data ```python x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=42) ``` * `test_size=0.2` sets aside **20%** of the 150 samples (30 flowers) for testing, keeping **80%** (120 flowers) for training. * `random_state=42` fixes the randomness, so anyone running this exact code gets the exact same split, and therefore the exact same results shown in section 5.8. #### Scaling the features ```python scaler = StandardScaler() x_train_scaled = scaler.fit_transform(x_train) x_test_scaled = scaler.transform(x_test) ``` See section 5.5 for why this step exists. In short, it creates rescaled copies of the training and test features, stored in new variables `x_train_scaled` and `x_test_scaled`. #### Building and training the model ```python svm = SVC(kernel='rbf', C=1) ``` Creates an untrained SVM classifier. * `kernel='rbf'` tells it to use the Radial Basis Function kernel described in section 5.1, allowing curved decision boundaries. * `C=1` is the **regularization strength**. It controls the trade off between having a wide, simple margin versus classifying every single training point perfectly. A smaller `C` allows more misclassifications in training but usually generalizes better to new data. A larger `C` tries harder to classify every training point correctly, which can lead to overfitting. ```python svm.fit(x_train, y_train) ``` This is where the actual learning happens. The model looks at the training features and their correct labels together, and works out the best boundaries to separate the three species. #### Predicting and evaluating ```python y_pred = svm.predict(x_test) ``` Feeds the test features (which the model has never seen labels for) through the trained model, and gets back predicted species for each of the 30 test flowers. ```python accuracy = accuracy_score(y_test, y_pred) print(f'Accuracy: {accuracy:.3f}') ``` Compares the predicted labels against the true labels and calculates what fraction were correct. The `:.3f` in the f-string formats the number to 3 decimal places. ```python print('Classification Report:') print(classification_report(y_test, y_pred)) ``` Prints a detailed breakdown of performance for each class separately, explained in section 5.7. ```python print('Confusion Matrix:') print(confusion_matrix(y_test, y_pred)) ``` Prints a grid comparing predicted labels against actual labels, explained in section 5.7. *** ### 5.5 Why StandardScaler is created here, and an important catch `StandardScaler` rescales each feature so it has a mean of 0 and a standard deviation of 1. This matters for many ML algorithms, including SVM, because features measured on very different scales (for example, one column ranging from 0 to 1 and another from 0 to 10,000) can unfairly dominate the distance calculations SVM relies on internally. **However, look closely at this code:** ```python scaler = StandardScaler() x_train_scaled = scaler.fit_transform(x_train) x_test_scaled = scaler.transform(x_test) svm = SVC(kernel='rbf', C=1) svm.fit(x_train, y_train) # <-- uses x_train, NOT x_train_scaled ... y_pred = svm.predict(x_test) # <-- uses x_test, NOT x_test_scaled ``` The scaled versions, `x_train_scaled` and `x_test_scaled`, are created but **never actually used**. The model is trained and tested on the original, unscaled `x_train` and `x_test`. This is a common small mistake, easy to make and easy to miss. In this particular case, it does not hurt the result, because the Iris dataset's 4 features are all already measured in the same unit (centimetres) and similar ranges, so scaling would not have changed much here. But if you are asked about this in a viva, it is worth pointing out this catch, and mentioning that the fix would simply be to change `svm.fit(x_train, y_train)` to `svm.fit(x_train_scaled, y_train)`, and `svm.predict(x_test)` to `svm.predict(x_test_scaled)`. *** ### 5.6 Visualizing the idea of the margin and kernel ```mermaid flowchart TD Start[Iris measurements: 4 numbers per flower] --> Kernel[RBF kernel reshapes the data] Kernel --> Boundary[SVM finds the widest margin boundary between species] Boundary --> Predict[New flower measurements are checked: which side of the boundary do they fall on?] Predict --> Result[Predicted species: Setosa, Versicolor, or Virginica] ``` *** ### 5.7 Understanding the output metrics #### Accuracy The simplest measure: out of all test predictions, what fraction were correct. ``` Accuracy: 1.000 ``` This means **100%** of the 30 test flowers were classified correctly. A perfect score. This happens fairly often with the Iris dataset because it is a small, clean, and relatively easy dataset for ML algorithms to learn. #### Classification Report ``` precision recall f1-score support 0 1.00 1.00 1.00 10 1 1.00 1.00 1.00 9 2 1.00 1.00 1.00 11 accuracy 1.00 30 macro avg 1.00 1.00 1.00 30 weighted avg 1.00 1.00 1.00 30 ``` This breaks performance down **per class** (per species): * **Precision:** out of everything the model predicted as this class, what fraction was actually correct. For example, of all the flowers the model called "class 0," 100% of them really were class 0. * **Recall:** out of everything that truly belongs to this class, what fraction did the model correctly catch. For class 0, 100% of the real class 0 flowers were correctly identified. * **F1-score:** a combined average of precision and recall, useful when you want a single number balancing both. * **Support:** how many actual test samples belonged to this class. Class 0 had 10 test samples, class 1 had 9, class 2 had 11, totalling 30, which matches our 20% test split. #### Confusion Matrix ``` [[10 0 0] [ 0 9 0] [ 0 0 11]] ``` This is a grid where **rows are the true labels** and **columns are the predicted labels**. Reading it: * Row 1 (`[10, 0, 0]`): of the 10 flowers that were truly class 0, all 10 were predicted as class 0, none were mistaken for class 1 or 2. * Row 2 (`[0, 9, 0]`): of the 9 flowers that were truly class 1, all 9 were predicted correctly. * Row 3 (`[0, 0, 11]`): of the 11 flowers that were truly class 2, all 11 were predicted correctly. A perfect confusion matrix has numbers **only on the diagonal** (top-left to bottom-right), which is exactly what we see here, confirming the 100% accuracy. Any number appearing off the diagonal would represent a misclassification. ```mermaid flowchart LR subgraph Confusion Matrix Diagonal = Correct T0["True: Setosa (10)"] --> P0["Predicted: Setosa (10)"] T1["True: Versicolor (9)"] --> P1["Predicted: Versicolor (9)"] T2["True: Virginica (11)"] --> P2["Predicted: Virginica (11)"] end ``` *** ### 5.8 Actual output (verified by running this exact code) ``` Accuracy: 1.000 Classification Report: precision recall f1-score support 0 1.00 1.00 1.00 10 1 1.00 1.00 1.00 9 2 1.00 1.00 1.00 11 accuracy 1.00 30 macro avg 1.00 1.00 1.00 30 weighted avg 1.00 1.00 1.00 30 Confusion Matrix: [[10 0 0] [ 0 9 0] [ 0 0 11]] ``` *** ### 5.9 Quick recap for viva * SVM finds the boundary (line or curve) that best separates classes, using the widest possible margin between them. * The RBF kernel allows SVM to draw curved, flexible boundaries instead of only straight lines, by reshaping data into a higher dimension. * `C` controls the trade off between a wider margin and classifying every training point correctly, higher C risks overfitting. * `StandardScaler` is meant to put features on the same scale before training, but in this specific code it was created and applied, then never actually passed into `.fit()` or `.predict()`, worth mentioning if asked to review or debug this code. * Accuracy, precision, recall, f1-score, and the confusion matrix are all ways of measuring how good a classifier is, each highlighting a slightly different aspect of performance. * A perfect 100% accuracy and a fully diagonal confusion matrix mean the model made zero mistakes on the test set. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://alham-rizvi.gitbook.io/alhamrizvi/ai-college-notes/practical-2-a-support-vector-machine-svm-on-the-iris-dataset.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.