In plain words
Picture dots of two colours on a page, and you have to draw one straight line between them. Many lines might work. A support vector machine, or SVM, picks the line with the most breathing room: as far as possible from the nearest dot of each colour.
That empty band either side of the line is called the margin. Only the few dots right at its edge decide where the line goes. These are the support vectors, and they give the method its name. Every other dot could move a little and the line would not change.
A wide margin tends to make fewer mistakes on new examples, because new points have room to land on the correct side. SVMs are a supervised method for classification, and the original papers tested them on reading characters in images.
Everyday example
Laying a footpath between two rows of houses
A town wants a footpath between two rows of houses, and it should be as wide as possible. The planner does not care about houses at the back. Only the houses that stick out nearest the gap decide where the path can run. Those front houses are the support vectors, the path's centre line is the boundary, and its width is the margin.
Where the example breaks: Real data is rarely this tidy. Some dots sit on the wrong side, so a real SVM lets a few points into the path and pays a penalty for each, and its boundary can be curved rather than a straight footpath.
How it actually works
Formally, an SVM finds a hyperplane (a line in two dimensions, a flat surface in more) that separates two classes with the largest margin.
Maximise the margin
Each training example is a point described by its features. Training solves an optimisation problem: find the boundary whose distance to the closest points of either class is as large as possible. A larger margin generally means lower error on new data.
Only support vectors count
The solution can be written using only the points that sit on or inside the margin. That makes the final model compact: it keeps a subset of the training points, not all of them.
Soft margin and the C setting
When classes overlap, a soft margin lets some points cross the line at a cost. A hyperparameter called C sets that cost: a low C gives a smoother, more forgiving boundary, a high C tries to classify every training point correctly and risks overfitting.
The kernel trick for curved boundaries
If no straight line works, a kernel function compares points as if they had been mapped into a space with many more dimensions, where a flat boundary can separate them. Back in the original space, that boundary looks curved. Common kernels are linear, polynomial and radial basis function (RBF).
Practical notes
SVMs work well when there are many features and not huge numbers of examples. They are sensitive to feature scale, so inputs are usually standardised first. They output a decision score, not a probability; probabilities need an extra calibration step.
- Key papers
- Boser, Guyon and Vapnik (1992) on maximum-margin training; Cortes and Vapnik (1995) on the soft margin, named "support-vector networks"
- Where
- AT&T Bell Labs, Holmdel, USA (Cortes and Vapnik's affiliation in 1995)
- Early test
- Optical character recognition: reading characters from images
- Also does
- Regression (support vector regression) and outlier detection
Questions people ask
What are support vectors?
The training points closest to the boundary, sitting on the edge of the margin or inside it. They alone fix where the boundary goes. Remove a point far from the line and nothing changes; move a support vector and the line moves.
What is the kernel trick in SVM?
A shortcut that lets an SVM draw curved boundaries. A kernel measures how similar two points are as if they lived in a much higher-dimensional space, where a flat cut can separate them, without ever computing that space directly.
Is an SVM a neural network?
No. An SVM solves one optimisation problem to find a single boundary, using kernels for curves. A neural network learns layers of features by gradual adjustment. Both are supervised methods, and the 1995 paper called SVMs "support-vector networks", which adds to the confusion.
Are SVMs still used?
Yes. They remain a standard tool in libraries such as scikit-learn, especially for smaller datasets with many features, where they train quickly. Today's large image, speech and language systems are built on deep learning instead.
Was this clear?
1 is confusing, 5 is crystal clear. One tap, no account needed.
Spotted something wrong, or know a better example? Suggest a change