Decision Tree

ClassificationBeginner~7 min

Decision Tree — Split the data with a sequence of yes/no questions.

A decision tree classifies by asking a series of simple yes/no questions, splitting the data at each step. Follow the branches down to a leaf and you get your prediction.

Class 0
Class 1
Selected region

Tree controls

Data

Dataset

Points per class22

Spread1.0×

Add points as

Model

Max depth3

Min samples per leaf1

Playback

Show the split search

Click any tree node to highlight the region it governs. Drag a point, or click empty space to drop a new one, and the tree re-splits live.

Break it

The idea in plain words

A decision tree classifies by asking a sequence of yes/no questions, each one a threshold on a single feature. Every split appears twice at once: as a branch in the tree and as a straight cut in the feature space. Follow the branches to a leaf and you have your prediction.

Each split is chosen to make the resulting groups as pure as possible. Let the tree grow without limit and it will carve a separate box around every point — perfect on training data, hopeless on anything new. That fragility is exactly what forests fix.

Now, the math

Splits are chosen to reduce Gini impurity, a measure of class mixing:

\text{Gini} = 1 - \sum_k p_k^2

$p_k$: the fraction of a node’s points belonging to class k.
$\text{Gini}=0$: a pure node — every point is the same class.

▸ Show the derivation

At each node the tree tries every threshold on every feature and picks the split that most reduces the weighted Gini of the children. Because splits are axis-aligned, boundaries are always staircases of horizontal and vertical cuts — never diagonal.

Trace it by hand

A node holds 8 points, 4 of class A and 4 of class B. Candidate split: x at most 2.5, which sends 3 A and 1 B left, and 1 A and 3 B right. All Gini values computed by the tree core.

Gini of the parent node
$\text{Gini}_{\text{parent}} = 1 - (0.5^2 + 0.5^2) = 0.5$
0.5 is the worst possible mixing for two classes.
Gini of the left child (3 A, 1 B)
$\text{Gini}_{\text{left}} = 1 - (0.75^2 + 0.25^2) = 1 - 0.625 = 0.375$
Gini of the right child (1 A, 3 B)
$\text{Gini}_{\text{right}} = 1 - (0.25^2 + 0.75^2) = 0.375$
Weighted average and gain
$\tfrac{4}{8}(0.375) + \tfrac{4}{8}(0.375) = 0.375, \qquad \text{gain} = 0.5 - 0.375 = 0.125$
The tree tries every threshold on every feature and keeps the split with the largest gain.

What just happened: One vertical cut turned a 50-50 node of Gini 0.5 into two 75-25 children of Gini 0.375 — a gain of 0.125. Growing a tree is just repeating this arithmetic greedily until the leaves are pure.

Now Break It

Try this: Unlimited depth grows a leaf for every point — perfect on train data, jagged and overfit everywhere else.

Control: Max depth slider (set to maximum)

What happens: Overfitting! With unlimited depth the tree isolates every single point — pure memorization.

Where decision tree is used

A decision tree classifies by asking a sequence of yes-or-no questions about the features, following the answers down branches until it reaches a leaf that assigns a label. This mirrors how humans reason through checklists, which is why trees are prized in settings that demand transparency. Banks use them for loan approval rules, doctors use them for triage and diagnostic pathways, and businesses use them to segment customers or to encode operational policies that staff can follow by hand. Trees handle mixed numeric and categorical data, need little preprocessing, and require no feature scaling. Just as importantly, a trained tree can be printed and read directly, so a stakeholder can trace exactly why a particular decision was made, which is invaluable when decisions must be justified or audited.

The main pitfall is that a single unconstrained decision tree overfits easily, growing until it memorizes the training data with a leaf for nearly every point. Such a tree looks flawless on training data yet generalizes poorly, which is why pruning, limiting depth, or requiring a minimum number of samples per leaf are essential. Another misconception is that trees are stable; in reality they are high-variance, and a small change in the data can produce a very different tree. This instability is exactly what ensemble methods like random forests and gradient boosting exploit and correct by averaging many trees. Finally, standard trees split on one feature at a time, so they struggle with relationships that depend on features in combination.

Frequently asked questions

How does a decision tree decide where to split?

At each node it searches for the feature and threshold that best separates the classes according to a purity measure, most often Gini impurity or information gain based on entropy. It greedily picks the split that most reduces impurity, then repeats the process on each resulting subset. This continues until a stopping rule is met.

What is pruning and why does it matter?

Pruning removes branches that add little predictive value, shrinking a tree that has grown too complex. It combats overfitting by trading a small amount of training accuracy for better generalization. Pruning can be done after growing the full tree or prevented up front by limiting depth and leaf size.

Do decision trees need feature scaling?

No. Trees split on thresholds within each feature independently, so the relative scale of different features does not affect the splits. This is a practical advantage over distance-based and gradient-based methods. You can feed raw numeric and categorical features with minimal preprocessing.

Why are decision trees considered unstable?

They are high-variance learners, meaning a small change in the training data can change which feature is chosen at the top and cascade into a very different structure. This sensitivity is a weakness for a single tree but is turned into a strength by ensembles that combine many diverse trees. Random forests and boosting rely on this diversity.

How are decision trees different from random forests?

A decision tree is a single model that is easy to read but prone to overfitting. A random forest trains many trees on random subsets of the data and features, then averages their votes to reduce variance and improve accuracy. The trade-off is that the forest is far harder to interpret than one tree.

← Back to all visualizations Continue on the Learning Path →

Written & reviewed by the ML Visualization team · Last updated August 6, 2026.