Lp norm

The Lp norm is a general function that extends measuring distances beyond the familiar Euclidean distance. Change “p” and you have a new norm!

A norm defines the magnitude of a vector in the vector space. The most commonly used norms are L1 and L2

Both L1 and L2 are derived from the Lp norm

  • L1 Normalization (Manhattan Norm)
  • L2 Normalization (Unit Vector Normalization)
  • Lp norm
  • L∞ norm

Lp norm

||x|| (double bars) is a notation meaning “norm of x”.

Both are ways of scaling vectors so that their length (or magnitude) becomes 1.

  • L1 Normalization makes the sum of absolute values equal to 1.
  • L1 normalization: scales so total “absolute weight” = 1.
  • L2 normalization: scales so “length of vector” = 1.
  • L2 Normalization makes the square root of the sum of squared values equal to 1.

||x|| (double bars) is a notation meaning “norm of x”.

L1 Normalization(Manhattan Norm)

  1. This technique is used to normalize data.
  2. It transforms the data in such a way that the sum of the absolute values of the vector is equal to 1
  3. The L1 norm prefers sparse coefficient vectors
  4. L1 norm performs feature selection and you can delete all features where the coefficient is 0. A reduction of the dimensions is useful in almost all cases.
  5. The L1 norm optimizes the median. Therefore the L1 norm is not sensitive to outliers.
  6. ||x|| (double bars) is a notation meaning “norm of x”.
  7. L1 norm:  Here p = 1
  9. The distance between (0,0) to (3,4)  can be calculated as shown below. The route to reach the end point could be different.
  10. Manhattan Distance

  11. By using a Bike, you can go east then north (horizontal then vertical in the 2D plane):

 

Route 1 from starting point to end point (east and north)

 

Route 2) you can also go north and then east to reach the end point

Route 3: You  can turn at every corner (making a ’ladder’ shape in the 2D plane):

Count the blocks from starting point to end point. In all the above cases it is equal to 7 irrespective of the route you follow. That was why it is called “Manhattan distance” or even “taxicab distance”.

L2 Norm:

Here p = 2

L2 is like a helicopter traveling in a perfect diagonal, going over the buildings: It forms a triangle.

L2 is just Pythagoras’ theorem

 

This is the “Euclidean distance”.

Difference between L1 norm and L2 Norm