Statinfer

203.4.6 Model-Bias Variance Tradeoff

Sweet spot for Bias and Variance.

Model Bias and Variance

In previous section, we studied about Type of Datasets, Type of Errors and Problem of Overfitting

  • Over fitting
  • Low Bias with High Variance
  • Low training error – ‘Low Bias’
  • High testing error
  • Unstable model – ‘High Variance’
  • The coefficients of the model change with small changes in the data
  • Under fitting
  • High Bias with low Variance
  • High training error – ‘high Bias’
  • testing error almost equal to training error
  • Stable model – ‘Low Variance’
  • The coefficients of the model doesn’t change with small changes in the data

The Bias-Variance Decomposition

\[Y = f(X)+\epsilon\] \[Var(\epsilon) = \sigma^2\] \[Squared Error = E[(Y -\hat{f}(x_0))^2 | X = x_0 ]\] \[= \sigma^2 + [E\hat{f}(x_0)-f(x_0)]^2 + E[\hat{f}(x_0)-E\hat{f}(x_0)]^2\] \[= \sigma^2 + (Bias)^2(\hat{f}(x_0))+Var(\hat{f}(x_0 ))\]

Overall Model Squared Error = Irreducible Error + \(Bias^2\) + Variance

Bias-Variance Decomposition

  • Overall Model Squared Error = Irreducible Error + \(Bias^2\) + Variance
  • Overall error is made by bias and variance together
  • High bias low variance, Low bias and high variance, both are bad for the overall accuracy of the model
  • A good model need to have low bias and low variance or at least an optimal where both of them are jointly low
  • How to choose such optimal model. How to choose that optimal model complexity

Choosing optimal model-Bias Variance Tradeoff

Bias Variance Tradeoff

Test and Training Error

The next post is about Cross validation.

0 responses on "203.4.6 Model-Bias Variance Tradeoff"

Leave a Message

Blog Posts

Hurry up!!!

"use coupon code for FLAT 30% discount"  datascientistoffer        ___________________________________      Subscribe to our youtube channel. Get access to video tutorials.                

Contact Us

Statinfer Software Solutions#647 2nd floor 1st Main, Indira Nagar 1st Stage, 100 feet road,Indranagar Bangalore,Karnataka, Pin code:-560038 Landmarks: Opp. Namma Metro Pillar 48.

Connect with us

linkin fn twitter g

How to become a Data Scientist.?

top