> For the complete documentation index, see [llms.txt](https://alham-rizvi.gitbook.io/alhamrizvi/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://alham-rizvi.gitbook.io/alhamrizvi/ai-college-notes/practical-3-decision-tree-classifier.md).

# Practical 3 — Decision Tree Classifier

**Language used:** Python 3 with the **scikit-learn** (`sklearn`) machine learning library. Run this practical in Google Colab.

**Aim:** Build a Decision Tree Classifier that predicts whether a customer will **Buy** or **Not Buy** a product, based on their **Age** and **Gender**.

***

### 3.1 Before the code: what IS a Decision Tree? 

A decision tree is basically a **flowchart of yes/no questions** that ends in a decision. You already build these in your head all the time:

> "Is it raining?" → Yes → "Take umbrella" | No → "Don't take umbrella"

A Decision Tree Classifier in ML does the same thing, except **the computer figures out which questions to ask, and in what order**, by looking at training data.

```mermaid
flowchart TD
    A[Root: Is Age <= 31.5?] -->|Yes| B[Don't Buy]
    A -->|No| C[Buy]
```

#### How does the computer decide *which question* to ask first?

It uses a measure of "messiness" in the data called **Entropy**.

* **Entropy = 0** → the group is perfectly "pure" (everyone in it has the same label, e.g. all "Buy").
* **Entropy = 1** (max, for 2 classes) → the group is a perfect 50/50 mix — completely uncertain.

The formula (for 2 classes, Buy `p` and Don't Buy `1-p`):

```
Entropy = - p*log2(p) - (1-p)*log2(1-p)
```

The tree tries every possible question (e.g. "Age ≤ 20?", "Age ≤ 21?", "Age ≤ 22?"... "Gender == 0?") and picks the one that gives the **biggest drop in entropy** after splitting, this drop is called **Information Gain**. That's exactly why our code says:

```python
model = DecisionTreeClassifier(criterion="entropy")
```

We're explicitly telling scikit-learn: *"use entropy/information gain to decide your splits"* (the other common option is `"gini"`, a slightly different but similar messiness measure).

#### Key vocabulary

| Term               | Meaning                                                             |
| ------------------ | ------------------------------------------------------------------- |
| **Root node**      | The very first question at the top of the tree                      |
| **Leaf node**      | An end point of the tree — a final decision, no more questions      |
| **Feature**        | An input column used to ask questions (here: `Age`, `Gender`)       |
| **Label / Target** | What we're trying to predict (here: `Buy` = 1, `Don't Buy` = 0)     |
| **Training data**  | Examples with known answers, used to build/"grow" the tree          |
| **Test data**      | Examples kept aside to check if the tree actually learned correctly |

***

### 3.2 The full code

```python
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

# Sample dataset
X = [
    [22, 0],
    [25, 1],
    [47, 1],
    [52, 0],
    [46, 1],
    [56, 0],
    [23, 1],
    [40, 0]
]

# Target labels
# 1 = Buy, 0 = Don't Buy
y = [0, 0, 1, 1, 1, 1, 0, 1]

# Split the dataset
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42
)

# Create Decision Tree model
model = DecisionTreeClassifier(criterion="entropy")

# Train the model
model.fit(X_train, y_train)

# Predict
y_pred = model.predict(X_test)

# Accuracy
print("Accuracy:", accuracy_score(y_test, y_pred))

# Predict a new sample
new_data = [[30, 1]]
prediction = model.predict(new_data)

if prediction[0] == 1:
    print("Prediction: Buy")
else:
    print("Prediction: Don't Buy")
```

***

### 3.3 Line-by-line explanation

#### Imports

```python
from sklearn.tree import DecisionTreeClassifier
```

Brings in the actual Decision Tree algorithm, pre-built inside scikit-learn — you don't have to code entropy calculations yourself.

```python
from sklearn.model_selection import train_test_split
```

A ready-made function to randomly split your dataset into a **training set** (to build the tree) and a **testing set** (to check if it actually works on unseen data).

```python
from sklearn.metrics import accuracy_score
```

A ready-made function that compares predicted labels vs actual labels and gives you a percentage correct.

#### The dataset

```python
X = [
    [22, 0],
    [25, 1],
    ...
]
```

`X` is our **feature matrix** — each inner list is one customer: `[Age, Gender]`. Here `Gender` is encoded as a number (`0` and `1`) because ML models only work with numbers, not text like `"Male"`/`"Female"`.

| Age | Gender | Meaning    |
| --- | ------ | ---------- |
| 22  | 0      | customer 1 |
| 25  | 1      | customer 2 |
| 47  | 1      | customer 3 |
| 52  | 0      | customer 4 |
| 46  | 1      | customer 5 |
| 56  | 0      | customer 6 |
| 23  | 1      | customer 7 |
| 40  | 0      | customer 8 |

```python
y = [0, 0, 1, 1, 1, 1, 0, 1]
```

`y` is the **label/target** for each corresponding row in `X` — whether that customer bought (`1`) or didn't (`0`). Row 1 in `X` (`[22, 0]`) maps to `y[0] = 0` → that 22-year-old did **not** buy.

#### Splitting the data

```python
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42
)
```

* `test_size=0.25` → set aside **25%** of the data (2 out of 8 rows) for testing, keep **75%** (6 rows) for training.
* `random_state=42` → a "seed" number that makes the random split **reproducible** — anyone who runs this exact code gets the exact same split every time (this is why we can show you the *exact* real output below, not a guess).
* The function returns **4 things at once**: training features, testing features, training labels, testing labels — unpacked into 4 variables in one line.

#### Building and training the model

```python
model = DecisionTreeClassifier(criterion="entropy")
```

Creates an (empty, untrained) decision tree object, configured to use entropy/information gain for choosing splits.

```python
model.fit(X_train, y_train)
```

This is where the actual "learning" happens — the model looks at `X_train` and `y_train` together and **builds the tree structure** (decides the questions and thresholds) that best separates buyers from non-buyers.

#### Predicting and measuring accuracy

```python
y_pred = model.predict(X_test)
```

Feeds the **test features** (which the model has never seen labels for) through the now-trained tree, and gets back predicted labels.

```python
print("Accuracy:", accuracy_score(y_test, y_pred))
```

Compares `y_pred` (what the model guessed) against `y_test` (the real, correct answers) and prints what fraction it got right.

#### Predicting a brand-new customer

```python
new_data = [[30, 1]]
prediction = model.predict(new_data)
```

Simulates a real-world use case: a new customer who is **30 years old, Gender = 1** walks in will they buy?

```python
if prediction[0] == 1:
    print("Prediction: Buy")
else:
    print("Prediction: Don't Buy")
```

`prediction` is a list (because `.predict()` can handle many rows at once) — `prediction[0]` gets the answer for our one new customer, and we translate the `1`/`0` number back into a human-readable label.

***

### 3.4 What the tree actually learned (real diagram, generated by running your exact code)

![Decision Tree Diagram](https://claude.ai/chat/decision_tree.png)

Reading this diagram:

* **Top box (root):** the tree asks `Age <= 31.5?` — this was the single best question it could ask to separate buyers from non-buyers, based on entropy.
* **Left branch (Age ≤ 31.5 → True):** entropy = 0, meaning this group is **100% pure** — everyone here is "Don't Buy" (class 0).
* **Right branch (Age ≤ 31.5 → False):** entropy = 0 too — everyone here is "Buy" (class 1).
* Interestingly, the tree **never even used `Gender`** — it found that `Age` alone perfectly separates this particular tiny dataset, so Gender added no extra information gain.

#### Same tree, shown as text:

```
|--- Age <= 31.50
|   |--- class: 0        (Don't Buy)
|--- Age >  31.50
|   |--- class: 1        (Buy)
```

***

### 3.5 Actual output (verified by running the exact code)

```
Accuracy: 1.0
Prediction: Don't Buy
```

**Why Accuracy = 1.0 (100%)?** With `random_state=42`, the test set happened to be `[25, 1]` (true label: Don't Buy) and `[56, 0]` (true label: Buy). Since Age 25 ≤ 31.5 → tree predicts "Don't Buy" ✅ correct, and Age 56 > 31.5 → tree predicts "Buy" ✅ correct. Both test predictions matched reality, so accuracy = 2/2 = **1.0**.

**Why "Don't Buy" for `new_data = [[30, 1]]`?** Age = 30, and 30 ≤ 31.5 → the tree routes this customer down the **left** branch → predicted class = 0 → **"Don't Buy."** Note that Gender (`1`) played no role at all in this decision, since the tree only split on Age.

***

### 3.6 Quick recap for viva/oral exam

* A **Decision Tree** splits data repeatedly using yes/no questions on feature values, chosen to reduce **entropy** (uncertainty) the most at each step — this is called **Information Gain**.
* `criterion="entropy"` tells scikit-learn to use entropy-based information gain (alternative: `"gini"`).
* `train_test_split` divides data into training (build the model) and testing (evaluate the model) sets.
* `.fit()` = train the model. `.predict()` = use the trained model to guess labels for new/unseen data.
* `accuracy_score` = fraction of test predictions that matched the true labels.
* This is a **supervised learning** technique — it needs labeled data (`y`) to learn from, unlike unsupervised methods like clustering.
* This dataset is tiny (8 rows) — for real exams they may ask you what would happen with more data, or if the tree could **overfit** (become overly complex and memorize training data instead of generalizing) — a common Decision Tree weakness, usually controlled using `max_depth`.

Output:


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://alham-rizvi.gitbook.io/alhamrizvi/ai-college-notes/practical-3-decision-tree-classifier.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.