K-Nearest Neighbors Algorithm

João Victor · Mar 20, 2021

K-Nearest Neighbors(KNN) is a machine learning algorithm used for classification and regression. In this article you will learn the classification method.

In KNN classification, the output is a class membership. An object is classified by a plurality vote of its neighbors, with the object being assigned to the class most common among its k nearest neighbors (k is a positive integer, typically small). If k = 1, then the object is simply assigned to the class of that single nearest neighbor.

Look this example using tomatoes as model:

We want to discover if the the middle tomato is classified as similar to the first or the third one.

let’s plot this tomatoes:

Seeing the graphic it’s possible identify visually that the tomato B is similar to the tomato A. Let’s calculate how close they are. Remember that to find the distance between two points we have to use this formula here:

Calculating the distance between tomatoes B and A the answer is: 1, and tomatoes B and C is: 3,60. So the distance formula confirms what you visually observed: tomatoes B and A are similar.

Nowadays there are several libraries that do this for you! one of them is the scikit-learn library which has already implemented the knn algorithm. I recently did the iris data set project using knn classification and you can check it out here.

I hope you enjoyed and learned from this article. If you liked it, don’t forget to share and feel free to ask me anything by email. bye.