204.3.4 How to Calculate Entropy for Decision Tree Split?

Link to the previous post : https://statinfer.com/204-3-3-how-decision-tree-splits-works/

Entropy Calculation – Example

E n t r o p y (S) = -(p +)(l o g 2(p +)) -(p -)(l o g 2(p -))

Gender Splits the population into two segments
Segment-1 : Age=”Young”
Segment-2: Age=”Old”
Entropy at segment-1
- Age=”Young” segment has 60 records [31+,29-] $E n t r o p y (S) = -(p +)(l o g 2(p +)) -(p -)(l o g 2(p -))$
- $- 31 / 60 l o g_{2} 31 / 60 - 29 / 60 l o g_{2} 29 / 60$
- (-31/60)log(31/60,2)-(29/60)log(29/60,2)
- 0.9991984 (99% Impurity in this segment)

Entropy at segment-2
- Age=”Old” segment has 40 records [19+,21-] $E n t r o p y (S) = -(p +)(l o g 2(p +)) -(p -)(l o g 2(p -))$
- $- 19 / 40 l o g_{2} 19 / 40 - 21 / 40 l o g_{2} 21 / 40$
- (-19/40)log(19/40,2)-(21/40)log(21/40,2)
- 0.9981959(99% Impurity in this segment too)

The next post is about information gain in decision tree split.

24th January 2018