What Is an Example of a Matrix Norm?

[ericm1234]

My book goes on to say:
"If we consider both C^n and C^m with norms, then we define the norm of an M x N matrix A by.."
Then the formula says norm of A=sup (over abs(v)=1) of abs(Av) = sup (over v does not equal 0) abs(Av)/abs(v)
Can someone please provide me at least one example of what this means?

 

[jbunniii]

Suppose we take
$$A = \left[\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix}\right]$$
For simplicity, let's work in $\mathbb{R}^2$ and use the 2-norm (euclidean length). The unit vectors in $\mathbb{R}^2$ consist of all points on the unit circle, i.e., any vector of the form
$$v = \left[\begin{matrix} \cos(\theta) \\ \sin(\theta) \end{matrix} \right]$$
The norm of $A$ is obtained by maximizing $||Av||$ with respect to $\theta$, i.e., by choosing the unit-length $v$ in the direction that maximizes the length of $Av$. We can carry out this computation explicitly:
$$Av = \left[\begin{matrix} \cos(\theta) + 2\sin(\theta) \\ 3\cos(\theta) + 4\sin(\theta) \end{matrix} \right]$$
so
$$||Av|| = \sqrt{(\cos(\theta) + 2\sin(\theta))^2 + (3\cos(\theta) + 4\sin(\theta))^2}$$
Wolfram Alpha tells me that this expression is maximized by $\theta \approx -2.18468$, which results in $v \approx \begin{matrix}[-0.576 & -0.8174]^T\end{matrix}$, and for this $v$, we have $||Av|| \approx 5.465$. Thus $||A|| \approx 5.465$.

 

[jbunniii]

By the way, the way I calculated the norm above is not how you would do it in practice, especially for larger matrices. There is a key theorem which says that the 2-norm of a matrix $A$ is the largest singular value of $A$, which equals the square root of the largest eigenvalue of $A^* A$, where $A^*$ is the conjugate transpose of $A$. This fact is almost obvious if you know the geometrical meaning of the singular value decomposition. See Trefethen and Bau, Numerical Linear Algebra for a really nice discussion of this, and of matrix norms in general.

There are also short cuts for computing the 1-norm and $\infty$-norm. These are the maximum absolute column sum and maximum absolute row sum, respectively. Both of these short cuts are pretty easy to prove.

 

[NemoReally]

I found it useful to visualize what happens when a matrix transforms the vector. The attached images show a Mathcad implementation. The first image shows one way of calculating the norm and finding the angle for which the norm is a maximum (another might be to find where the derivative is zero). I used jbunniii's value for A to verify the solution. Note one difference (which should be slightly more obvious in the plot) is that I have chosen to look for the "maximum" angle within the range 0 to ∏, so I get a different value - however, as the norm function is cyclic, this doesn't really matter, the result is the same.

The second image plots the norm function versus angle, making clear its cyclic nature. It also plots the unit circle (v(θ)) and the circle as transformed under A (a(θ)). The right hand of the 2 lower plots connects the values for v to those of a, thus making the mapping explicitly visible. I changed the value of A to show the mapping more clearly (the original value of A results in a messier diagram!).

 

[Fredrik]

My book goes on to say:
"If we consider both C^n and C^m with norms, then we define the norm of an M x N matrix A by.."
Then the formula says norm of A=sup (over abs(v)=1) of abs(Av) = sup (over v does not equal 0) abs(Av)/abs(v)
Can someone please provide me at least one example of what this means?

If you read the LaTeX guide, you will be able to write things like this:
$$\|A\|=\sup_{|v|=1}|Av| =\sup_{v\neq 0}\frac{|Av|}{|v|}.$$ Hit the quote button to see how I did this.

Do you understand the concept of "supremum" (="least upper bound")?

The idea here is that the norm of A tells us how "long" A can make a unit vector. $\|A\|$ is the smallest real number such that no |Av| with |v|=1 is bigger. Equivalently, $\|A\|$ is the radius of the smallest closed ball around 0 that contains A(S), where S is the unit sphere.

Note that
$$\left\{|Av|:v\in\mathbb C^N, |v|=1\right\}=\left\{\frac{|Av|}{|v|}:v\in\mathbb C^N, v\neq 0\right\}.$$ Since the sets are the same, their supremums (least upper bounds) are the same.

This norm is more interested on infinite-dimensional vector spaces, because there the corresponding sets may not be bounded from above. If it is bounded from above, then the operator is said to be bounded. One of the theorems in functional analysis says that a linear operator is continuous if and only if it's bounded.

Actually, now that I think of it, that theorem is interesting here too, because it implies that every linear transformation between finite-dimensional vector spaces is continuous (with respect to the topology induced by the standard norm).