In previous section, we studied about Neural Network Appendix
The first post is about basics of classification and slowly we will lean into SVM with full throttle.
Support Vector Machines
Contents
- Introduction
- The decision boundary with largest margin
- SVM- The large margin classifier
- SVM algorithm
- The kernel trick
- Building SVM model
- Conclusion
Introduction
- SVM is another black box method in Machine Learning space
- Compared to other ml algorithms, SVM totally a different approach to learning.
- The in-depth theory and mathematics of SVM needs great knowledge in vector algebra and numerical analysis
- We will try to learn the basic principal, philosophy, implementation of SVM
- SVM was first introduced by Vapnik and Chervonenkis
- Neural networks try to reduce the squared error and often suffer from overfitting.
- SVM algorithm has better generalization ability. There are many applications where SVM works better than neural networks
The Classifier
- To understand the SVM algorithm easily, we will start with the decision boundary
- The line or margin that separates the classes
- Classification algorithms are all about finding the decision boundaries
- A good classifier is the one that generalizes well. It should work well on both training and testing data
- It need not be a straight line always
Many Classifiers
The Margin of Classifier
Out of all the classifiers, the one that has maximum margin will generalize well. But why?
The Best Decision Boundary
- Imagine two more data points. The classifier with maximum margin will be able to classify them more accurately.
The Maximum Margin Classifier
- So, the best classifier has maximum margin
- The classifier that maximizes the distance between itself and the nearest training data
- In our example a,b,c are the training data points that are near to m1, and a,c,d are the training examples that are near to model m2.
- The model m1 has maximum margin
- The model m1 works well with the unseen examples
- The model m1 does good generalization
- For a given dataset, if we can find a classifier that has maximum margin, then it will assure maximum accuracy.
The next post is about a practice session on a Simple Classifier.
