Euclidean Norm of a Vector: Exploring Its Importance

[aaaa202]

So I'm taking some courses in calculus, and I am surprised by how little explaining there is to the definition of the euclidean norm.
I have never understood why you want to define the length of a vector through the pythagorean way. I mean sure, it does seem that nature likes that measure of distance, and sure classic geometry is based on it.
But in reality when defining continuity and all the other things specific to analysis you shouldn't in my view really be restricted by geometry. Couldnt you choose other kinds of norms which would create an equally well described calculus, or is there some kind of special property to the euclidean norm that I should know of? I mean all I can see it really doing in analysis is that acts as a numerical value. But to fews surprise, you can find a lot of other algebraic relations between vector coordinates that would have the same property.
So please just explain why the euclidean norm is so important, and don't be confused by all my talk, some of it probably doesn't make sense.

 

[jbunniii]

Continuity and other limiting concepts in analysis can be defined in any metric space, which is simply a set $S$ along with a distance function $d$ satisfying the following conditions for all $x,y \in S$: (1) $d(x,y) \geq 0$, and $d(x,y) = 0$ if and only if $x = y$; (2) $d(x,y) = d(y,x)$; (3) $d(x,z) \leq d(x,y) + d(y,z)$.

Continuity, for example, is defined as follows in a metric space: a function $f$ is continuous at a point $x$ if for every $\epsilon > 0$, there is a $\delta > 0$ such that $d(f(x),f(y)) < \epsilon$ whenever $d(x,y) < \delta$.

A special type of metric space is a normed vector space, where in addition to measuring distance between elements, you can also add them to each other. If $S$ is a vector space over a field $F$, then we can define a norm on $S$. This is a function $N$ from $S$ to the nonnegative real numbers, satisfying the following conditions for all $x,y \in S$ and $a \in F$: (1) $N(x) \geq 0$, and $N(x) = 0$ if and only if $x = 0$; (2) $N(x+y) \leq N(x) + N(y)$; (3) $N(ax) = |a|N(x)$. Given a norm, we can always define a metric by $d(x,y) = N(x-y)$.

Some examples of norms on $\mathbb{R}^n$, where we assume $x = (x_1,x_2,\ldots,x_N)$:
(1) The euclidean norm, $||x||_2 = \sqrt{x_1^2 + x_2^2 + \ldots + x_N^2}$
(2) More generally, the $p$-norm, where $p$ is any real number satisfying $p \geq 1$: $||x||_p = (|x_1|^p + |x_2|^p + \ldots |x_N|^p)^{1/p}$
(3) The $\infty$-norm, which is a limiting case of the $p$-norm: $||x||_{\infty} = \max\{|x_1|, |x_2|, \ldots, |x_N|\}$

You can define analogous norms on spaces of sequences or functions.

All of these are very useful for different purposes. For example, the $\infty$-norm is a useful way to measure uniform convergence. The $p$-norms show up in many contexts in analysis, where you have to deal with the fact that not every sequence or function will have a finite norm of a given type. In $\mathbb{R}^n$, the above norms are all equivalent in the sense that a function which is continuous with respect to one of the norms will be continuous with respect to the others, and similarly with limits, open sets, etc.

Euclidean distance is obtained by using the 2-norm. This norm is special because it is the only one out of the examples listed above which comes from an inner product: $||x||_2 = \sqrt{\langle x, x \rangle}$. Thus it is the natural norm (and hence distance) to use in any space which has an inner product. Inner product spaces are special types of normed spaces. In addition to providing a norm, the inner product also allows one to define the angle between two vectors. $\mathbb{R}^N$ is an inner product space, the standard inner product being the usual dot product.

You don't have to use the 2-norm in an inner product space, but the 2-norm is the one that interacts the most nicely with the inner product. For example, if $x$ and $y$ are orthogonal (meaning $\langle x,y \rangle = 0$), then the pythagorean theorem holds for the 2-norm: $||x+y||_2^2 = ||x||_2^2 + ||y||_2^2$, but not for other norms. Also, the Cauchy-Schwarz inequality, $|\langle x,y \rangle| \leq ||x||_2 ||y||_2$, holds only for the 2-norm, although a more general inequality (Holder's inequality) holds for $p$-norms.

 

[micromass]

You are completely correct. There are other norms out there. And they describe continuity and differentiability equally well! So the Euclidean norm isn't really all that special other than that it agrees without intuition (and that it induces an inner product, but that's something for another time).

You're only in calculus right now, but if you ever take real analysis, then you will see more general structures like normed spaces, metric spaces, manifolds, topological spaces,... The point of these structures is to generalize things like continuity, differentiability, integrals,...

So in short, we can indeed put other norms on $\mathbb{R}^n$. Two of these norms are

$$\|(x_1,...,x_n)\|_\infty = \max_{1\leq i \leq n} |x_i|$$

and for $p\geq 1$ (note the special case of p=2)

$$\|(x_1,...,x_n)\|_p = \sqrt[p]{\sum_{i=1}^n |x_i|^p}$$

These norms describe the same continuous functions as the Euclidean norm.

 

[micromass]

Continuity and other limiting concepts in analysis can be defined in any metric space, which is simply a set $S$ along with a distance function $d$ satisfying the following conditions for all $x,y \in S$: (1) $d(x,y) \geq 0$, and $d(x,y) = 0$ if and only if $x = y$; (2) $d(x,y) = d(y,x)$; (3) $d(x,z) \leq d(x,y) + d(y,z)$.

Continuity, for example, is defined as follows in a metric space: a function $f$ is continuous at a point $x$ if for every $\epsilon > 0$, there is a $\delta > 0$ such that $d(f(x),f(y)) < \epsilon$ whenever $d(x,y) < \delta$.

A special type of metric space is a normed vector space, where in addition to measuring distance between elements, you can also add them to each other. If $S$ is a vector space over a field $F$, then we can define a norm on $S$. This is a function $N$ from $S$ to the nonnegative real numbers, satisfying the following conditions for all $x,y \in S$ and $a \in F$: (1) $N(x) \geq 0$, and $N(x) = 0$ if and only if $x = 0$; (2) $N(x+y) \leq N(x) + N(y)$; (3) $N(ax) = |a|N(x)$. Given a norm, we can always define a metric by $d(x,y) = N(x-y)$.

Some examples of norms on $\mathbb{R}^n$, where we assume $x = (x_1,x_2,\ldots,x_N)$:
(1) The euclidean norm, $||x||_2 = \sqrt{x_1^2 + x_2^2 + \ldots + x_N^2}$
(2) More generally, the $p$-norm, where $p$ is any real number satisfying $p \geq 1$: $||x||_p = (x_1^p + x_2^p + \ldots x_N^p)^{1/p}$
(3) The $\infty$-norm, which is a limiting case of the $p$-norm: $||x||_{\infty} = \max\{|x_1|, |x_2|, \ldots, |x_N|\}$

You can define analogous norms on spaces of sequences or functions.

All of these are very useful for different purposes. For example, the $\infty$-norm is a useful way to measure uniform convergence. The $p$-norms show up in many contexts in analysis, where you have to deal with the fact that not every sequence or function will have a finite norm of a given type. In $\mathbb{R}^n$, the above norms are all equivalent in the sense that a function which is continuous with respect to one of the norms will be continuous with respect to the others, and similarly with limits, open sets, etc.

Euclidean distance is obtained by using the 2-norm. This norm is special because it is the only one out of the examples listed above which comes from an inner product: $||x||_2 = \sqrt{\langle x, x \rangle}$. Thus it is the natural norm (and hence distance) to use in any space which has an inner product. Inner product spaces are special types of normed spaces. In addition to providing a norm, the inner product also allows one to define the angle between two vectors. $\mathbb{R}^N$ is an inner product space, the standard inner product being the usual dot product.

You don't have to use the 2-norm in an inner product space, but the 2-norm is the one that interacts the most nicely with the inner product. For example, if $x$ and $y$ are orthogonal (meaning $\langle x,y \rangle = 0$), then the pythagorean theorem holds for the 2-norm: $||x+y||_2^2 = ||x||_2^2 + ||y||_2^2$, but not for other norms. Also, the Cauchy-Schwarz inequality, $|\langle x,y \rangle| \leq ||x||_2 ||y||_2$, holds only for the 2-norm, although a more general inequality (Holder's inequality) holds for $p$-norms.

Damn you. You were not only earlier, but your reply was also better

 

[jbunniii]

Damn you. You were not only earlier, but your reply was also better

However, there is one thing I should have stated more carefully. The norm induced by the inner product is not always the 2-norm. That's only the case if you use the standard dot product. You can define more general inner products on $\mathbb{R}^N$, of the form $\langle x,y \rangle = x^T A y$, where $A$ is a positive definite matrix. Then the corresponding norm is $||x|| = \sqrt{x^T A x}$, and this (not the standard 2-norm) is the one that plays nicely with the inner product.