Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Example please for matrix norm

  1. Jan 26, 2013 #1
    My book goes on to say:
    "If we consider both C^n and C^m with norms, then we define the norm of an M x N matrix A by.."
    Then the formula says norm of A=sup (over abs(v)=1) of abs(Av) = sup (over v does not equal 0) abs(Av)/abs(v)
    Can someone please provide me at least one example of what this means???
  2. jcsd
  3. Jan 27, 2013 #2


    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    Suppose we take
    $$A = \left[\begin{matrix} 1 & 2 \\ 3 & 4 \end{matrix}\right]$$
    For simplicity, let's work in [itex]\mathbb{R}^2[/itex] and use the 2-norm (euclidean length). The unit vectors in [itex]\mathbb{R}^2[/itex] consist of all points on the unit circle, i.e., any vector of the form
    $$v = \left[\begin{matrix} \cos(\theta) \\ \sin(\theta) \end{matrix} \right]$$
    The norm of [itex]A[/itex] is obtained by maximizing [itex]||Av||[/itex] with respect to [itex]\theta[/itex], i.e., by choosing the unit-length [itex]v[/itex] in the direction that maximizes the length of [itex]Av[/itex]. We can carry out this computation explicitly:
    $$Av = \left[\begin{matrix} \cos(\theta) + 2\sin(\theta) \\ 3\cos(\theta) + 4\sin(\theta) \end{matrix} \right]$$
    $$||Av|| = \sqrt{(\cos(\theta) + 2\sin(\theta))^2 + (3\cos(\theta) + 4\sin(\theta))^2}$$
    Wolfram Alpha tells me that this expression is maximized by [itex]\theta \approx -2.18468[/itex], which results in [itex]v \approx \begin{matrix}[-0.576 & -0.8174]^T\end{matrix}[/itex], and for this [itex]v[/itex], we have [itex]||Av|| \approx 5.465[/itex]. Thus [itex]||A|| \approx 5.465[/itex].
  4. Jan 27, 2013 #3


    User Avatar
    Science Advisor
    Homework Helper
    Gold Member

    By the way, the way I calculated the norm above is not how you would do it in practice, especially for larger matrices. There is a key theorem which says that the 2-norm of a matrix [itex]A[/itex] is the largest singular value of [itex]A[/itex], which equals the square root of the largest eigenvalue of [itex]A^* A[/itex], where [itex]A^*[/itex] is the conjugate transpose of [itex]A[/itex]. This fact is almost obvious if you know the geometrical meaning of the singular value decomposition. See Trefethen and Bau, Numerical Linear Algebra for a really nice discussion of this, and of matrix norms in general.

    There are also short cuts for computing the 1-norm and [itex]\infty[/itex]-norm. These are the maximum absolute column sum and maximum absolute row sum, respectively. Both of these short cuts are pretty easy to prove.
  5. Jan 27, 2013 #4
    I found it useful to visualize what happens when a matrix transforms the vector. The attached images show a Mathcad implementation. The first image shows one way of calculating the norm and finding the angle for which the norm is a maximum (another might be to find where the derivative is zero). I used jbunniii's value for A to verify the solution. Note one difference (which should be slightly more obvious in the plot) is that I have chosen to look for the "maximum" angle within the range 0 to ∏, so I get a different value - however, as the norm function is cyclic, this doesn't really matter, the result is the same.

    The second image plots the norm function versus angle, making clear its cyclic nature. It also plots the unit circle (v(θ)) and the circle as transformed under A (a(θ)). The right hand of the 2 lower plots connects the values for v to those of a, thus making the mapping explicitly visible. I changed the value of A to show the mapping more clearly (the original value of A results in a messier diagram!).

    Attached Files:

  6. Jan 27, 2013 #5


    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    If you read the LaTeX guide, you will be able to write things like this:
    $$\|A\|=\sup_{|v|=1}|Av| =\sup_{v\neq 0}\frac{|Av|}{|v|}.$$ Hit the quote button to see how I did this.

    Do you understand the concept of "supremum" (="least upper bound")?

    The idea here is that the norm of A tells us how "long" A can make a unit vector. ##\|A\|## is the smallest real number such that no |Av| with |v|=1 is bigger. Equivalently, ##\|A\|## is the radius of the smallest closed ball around 0 that contains A(S), where S is the unit sphere.

    Note that
    $$\left\{|Av|:v\in\mathbb C^N, |v|=1\right\}=\left\{\frac{|Av|}{|v|}:v\in\mathbb C^N, v\neq 0\right\}.$$ Since the sets are the same, their supremums (least upper bounds) are the same.

    This norm is more interested on infinite-dimensional vector spaces, because there the corresponding sets may not be bounded from above. If it is bounded from above, then the operator is said to be bounded. One of the theorems in functional analysis says that a linear operator is continuous if and only if it's bounded.

    Actually, now that I think of it, that theorem is interesting here too, because it implies that every linear transformation between finite-dimensional vector spaces is continuous (with respect to the topology induced by the standard norm).
    Last edited: Jan 27, 2013
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook