Link to the previous post : https://statinfer.com/204-4-2-calculating-sensitivity-and-specificity-in-python/
In this post we will cover a bit more on Sensitivity and Specificity. We will see how it get’s affected if we change the threshold for classification and preferred values of Sensitivity and Specificity based on cases.
Sensitivity and Specificity
- By changing the threshold, the good and bad customers classification will be changed hence the sensitivity and specificity will be changed.
- Which one of these two we should maximize? What should be ideal threshold?
- Ideally we want to maximize both Sensitivity & Specificity. But this is not possible always. There is always a trade-off.
- Sometimes we want to be 100% sure on Predicted negatives, sometimes we want to be 100% sure on Predicted positives.
- Sometimes we simply don’t want to compromise on sensitivity sometimes we don’t want to compromise on specificity
- The threshold is set based on business problem
When Sensitivity is a High Priority
- Predicting a bad customers or defaulters before issuing the loan
- Predicting a bad defaulters before issuing the loan
- The profit on good customer loan is not equal to the loss on one bad customer loan.
- The loss on one bad loan might eat up the profit on 100 good customers.
- In this case one bad customer is not equal to one good customer.
- If p is probability of default then we would like to set our threshold in such a way that we don’t miss any of the bad customers.
- We set the threshold in such a way that Sensitivity is high.
- We can compromise on specificity here. If we wrongly reject a good customer, our loss is very less compared to giving a loan to a bad customer.
- We don’t really worry about the good customers here, they are not harmful hence we can have less Specificity.
When Specificity is a High Priority
- Testing a medicine is good or poisonous
- Testing a medicine is good or poisonous
- In this case, we have to really avoid cases like , Actual medicine is poisonous and model is predicting them as good.
- We can’t take any chance here.
- The specificity need to be near 100.
- The sensitivity can be compromised here. It is not very harmful not to use a good medicine when compared with vice versa case.
Sensitivity vs Specificity – Importance
- There are some cases where Sensitivity is important and need to be near to 1.
- There are business cases where Specificity is important and need to be near to 1.
- We need to understand the business problem and decide the importance of Sensitivity and Specificity.
The next post is about roc and auc.
Link to the next post : https://statinfer.com/204-4-4-roc-and-auc/
