Summary

The article provides a beginner-friendly explanation of Manhattan and Euclidean Distance, two fundamental concepts in measuring distance in deep learning and machine learning, and discusses their practical applications and differences.

Abstract

The article "Understand Manhattan vs Euclidean Distance in Minutes" serves as a primer on two key distance metrics used in machine learning. It explains the Manhattan Distance, which is akin to city block distances and is particularly relevant in grid-like environments, and the Euclidean Distance, which represents the straight-line distance between two points. The text illustrates these concepts with visual examples and references to the Pythagorean theorem, emphasizing their relevance in regression models and continuous numeric output scenarios. The article also touches on the importance of these distances in the context of machine learning, such as in the selection of appropriate cost functions and their impact on model performance. It encourages readers to understand these concepts through experimentation and provides additional resources, including a companion Youtube video and related articles on regularization techniques.

Opinions

The author suggests that domain expertise should guide the choice between Manhattan and Euclidean Distance, as certain data types (like real estate and taxi data) may be better suited to one over the other.
Experimentation is highlighted as crucial for determining which distance metric yields better results for a specific machine learning model and domain.
The article promotes the idea that beginners should not be discouraged if they do not immediately grasp the concepts, recommending comparison and contrast as a learning strategy.
The author emphasizes the historical significance and naming conventions of these distance metrics, such as the L1 and L2 norms, and their relation to regularization techniques like Lasso and Ridge.
The preference for Euclidean Distance in many regression models is noted, although the article maintains that only through experimentation can the most effective metric be determined for a given scenario.
The article encourages the use of its content and resources for educational purposes, with a reminder to respect copyright by not reposting or scraping the content to other sites.

Understand Manhattan vs Euclidean Distance in Minutes

In this article, Manhattan and Euclidean Distance, two way of measuring distance and performance in deep learning, is explained in simple terms. It is a beginner, basic guide to machine learning distance functions and cost functions. As always our content is beginner friendly. Don’t use Medium? Subscribe [email protected] for free content and notifications whenever we publish. Prefer to watch a Youtube tutorial instead? Scroll all the way down for our companion study video on Youtube. Your claps are greatly appreciated and help pay for our bagels :)

Machine learning is an interdisciplinary field (computer science, engineering, statistics, math). There are many related terms. For this topic related topics include: distance function, metrics, criterion (Pytorch). There’s also a companion Youtube video.

Introduction

Let’s learn about two common ways to measure distance between two points. Depends on our machine learning model and metric, we may get better result using Manhattan or Euclidean distance. While both are used in regression models, or models with continuous numeric output, unlike classifications, Euclidean seems to be slightly more common. But only experimentation can reveal the better result for the specific scenario.

Visual Intuition of Manhattan Distance VS Euclidean Distance

Imagine the grey boxes are buildings in the streets of Manhattan. A pedestrian has to walk length A, then length B to get from START to DESTINATION. A helicopter on the other hand, does not need to trace the streets along Side A and Side B. It can fly over the buildings using route C, a direct line, the shortest distance between two points.

This may remind you of high school math — Pythagorean theorem A squared+ B squared = C squared. C is the longest side of a right triangle. By calculating the reverse of the function we can get the actual length of C as opposed to C squared. See slide below.

A quick reminder the relationship between A, B, C is explained using the Pythagorean Theorem.

Manhattan Distance

In many United States cities, streets are divided into grids, as seen on Google map. To travel from Start to Destination, we can only traverse the horizontal and vertical distances in a grid, instead of taking a shortcut / going direct. You can remember it by the name — Manhattan, referring to New York, city streets. You can also think of this as the car distance. Pedestrians can generally travel more freely, sometimes take direct line — shortest distance. So may be Manhattan distance is better for measuring models that have to do with cars than pedestrians.

Pro tip from Uniqtech to beginners: don’t sweat it if Manhattan Distance makes no sense yet, you can understand it better by comparing and constrast with Euclidean Distance.

Manhattan Distance is also called taxicab geometry, absolute distance (Wikipedia). Absolute distance matters, because no matter whether the cab needs to go up down, left or right, we don’t want the distance to be negative and cancel each other out, we want to account for all directions traveled, because of the grid restriction.

Pro tip from Uniqtech to beginners: Another name for Manhattan Distance is more important: it is called the L1 Norm. It is very important because we will see L2 Norm in Euclidean distance, and there are generalizations to unlimited higher dimensions Ln Norm.

Pro tip from Uniqtech to beginners: A related though different concept is L1 Regularization. The history behind the name lasso

“The geometry has been used in regression analysis since the 18th century, and today is often referred to as LASSO. The geometric interpretation dates to non-Euclidean geometry of the 19th century and is due to Hermann Minkowski.” (Wikipedia)

There’s also ridge regularization. Check out our articles on both Lasso and Ridge here.

Having the absolute function in the formula, also makes the result of the formula more discrete (step-like, kink join points), versus continuous as in the case of Euclidean distance.

Formal Definition, Formula

It’s a little harder to understand. The plain English is : sum up all the absolute distance of horizontal distances and vertical distances between Start and Destination on a grid.

Euclidean Distance Formula

The Euclidean distance figure below is from wikipedia. In three dimension, to put it in plain English, it is the hypotenuse of a triangle, that shares a leg with a lower dimension right triangle. We will explain this more in a future knowledge card on https://ml.learn-to-code.co our website, but here what you can see is, with some math, it generalize well to higher dimensional data!

L2 Norm

Summary

To choose between Manhattan Distance and Euclidean Distance, we want to rely on domain expertise (Manhattan Distance can work better for real estate and taxi data), the pros and cons of the formula (the nature, behavior of the distance it captures), and most importantly experimentation. As data engineers, we should first make an educated guess choice between the two, and then use our code to run experiments and see if our test metric improves or successfully capture the important metric specific to our model and domain.

Old Video

Here’s an older video of us explaining : Euclidean Manhattan distance l1 l2 norm technical interview machine https://youtu.be/7a1lj4RBfvU via @YouTube #machinelearning #data #math #technicalinterview #job #developer

L1 vs L2 norm