204.3.4 How to Calculate Entropy for Decision Tree Split?
Link to the previous post : https://statinfer.com/204-3-3-how-decision-tree-splits-works/
Entropy Calculation – Example
- Entropy at root
- Total population at root 100 [50+,50-]
- Entropy(S) = −p+log2p+−p−log2p−
- −0.5log2(0.5)−0.5log2(0.5)
- -(0.5)(-1) – (0.5)(-1)
- 1
- 100% Impurity at root
Entropy(S)=−(p+)(log2(p+))−(p−)(log2(p−))
Entropy Calculation
- Gender Splits the population into two segments
- Segment-1 : Age=”Young”
- Segment-2: Age=”Old”
- Entropy at segment-1
Practice : Entropy Calculation – Example
- Calculate entropy at the root for the given population
- Calculate the entropy for the two distinct gender segments
Code- Entropy Calculation
- Entropy at root 100%
- Male Segment : (-48/60)log(48/60,2)-(12/60)log(12/60,2)
- FemaleSegment : (-2/40)log(2/40,2)-(38/40)log(38/40,2)
The next post is about information gain in decision tree split.