Igor Khavkine
Nov28-04, 06:02 AM
<jabberwocky><div class="vbmenu_control"><a href="jabberwocky:;" onClick="newWindow=window.open('','usenetCode','toolbar=no, location=no,scrollbars=yes,resizable=yes,status=no ,width=650,height=400'); newWindow.document.write('<HTML><HEAD><TITLE>Usenet ASCII</TITLE></HEAD><BODY topmargin=0 leftmargin=0 BGCOLOR=#F1F1F1><table border=0 width=625><td bgcolor=midnightblue><font color=#F1F1F1>This Usenet message\'s original ASCII form: </font></td></tr><tr><td width=449><br><br><font face=courier><UL><PRE>On Wed, 24 Nov 2004 07:46:37 +0000, Blake Winter wrote:\n\n> Yes, I feel the same way about the horrors of matrix notation, what with\n> the row vectors and column vectors hiding what\'s actually going on...\n\nI think matrix notation is not as bad as some people make it seem. Being\nfamiliar with matrices provides a great step towards either index or\nindex-free tensor notation. One just has to realize that there is a hidden\nstructure when things are written in row and column notation.\n\nI don\'t think anyone would disagree that column vectors form a good honest\nvector space, just as well as row vectors do. The key observation is that,\nonce things are written in this form, we have access to two things: matrix\nmultiplication and transposition. With matrix multiplication we can\nnaturally pair row and column vectors to give real numbers:\n\n[b_1]\n[b_2]\n[a_1 a_2 a_3 ... a_n] [b_3] = a_1 b_1 + a_2 b_2 + ... + a_n b_n .\n[...]\n[b_n]\n\nThis pairing clearly makes the space of row vectors dual to that of the\ncolumn vectors. Immediately we have a direct, almost pictorial,\nillustration of a concept that\'s a little hard to grasp when starting\nout in abstract linear algebra.\n\nThis even makes sense when we consider the definitions of matrices and\ntheir multiplication whith which we were brought up. An n x m matrix is\na linear map from the space of m-component column vectors to the space\nof n-component column vectors. And matrix multiplication is just\ncomposition of these maps. By that logic, row vectors (1 x n matrices)\nare maps from n-component column vectors to real numbers (1-component\nvectors), and that\'s exactly the definition of a dual space!\n\nTransposition provides an identification between column vectors and\nrow vectors:\n\nT\n[a_1] T [a_1]\n[a_2] = [a_1 a_2 ... a_n] , [a_1 a_2 ... a_n] = [a_2] .\n[...] [...]\n[a_n] [a_n]\n\nBut, since row vectors are the duals of column vectors, what we have is\na canonoical identification between vectors and their duals that is\nnon-degenerate (invertible) and positive definite (a^T a > 0 for every\nnon-zero a). In other words we have a metric on our hands! Since we have\na metric, any 2-form can be associated with an operator (a matrix) just\nby raising an index: g(a,b) = a^T g b, where a and b are row vectors and\nnow the g in between is the matrix corresponding to the 2-form g(.,.).\nIn particular the metric itself can be written\n\n[1 0 ... 0] [b_1]\n[0 1 ... 0] [b_2]\n<a,b> = a^T b = [a_1 a_2 ... a_n] [ ... ] [...] ,\n[0 0 ... 1] [b_n]\n\nit is represented by the identity matrix.\n\nSome basic theorems of abstract (finite dimensional) linear algebra\nbecome trivial when written in terms of matrices. For instance, a vector\nspace V is isomorphic to its dual V^*. Well of course, just take the\ntranspose! A vector space V is _canonically_ isomorphic to its double\ndual V^**. Well, of course, just take the transpose twice and you end up\nwhere you started!\n\nThere is also a nice duality between column and row notation. We were\ntaught as toddlers to vectors are written as columns and that linear\noperators are applied to them by matrix multiplication on the left.\nHowever, there is nothing wrong with doing things backwards, writing\nvectors as rows and applying linear operators to them by matrix\nmultiplication on the right. There is even a deeper (somewhat hidden)\nmeaning to this duality. It\'s called the adjoint!\n\nDenoting by <a,b> the pairing between a vector b and a dual vector a, we\ncall A the adjoint of B if <Aa,b> = <a,Bb> for all a and b. Notice that\nwhile B is a map from V to W (with b in V), the adjoint A will be a map\nfrom the dual W^* to the dual V^* (with a in W^*). In matrix notation\nthis becomes trivial. If a is a row vector with n components, b is a\ncolumn vector with m components, and B is an n x m matrix, to find A the\nadjoint of B all we have to do is move some parentheses around:\n\na (B b) = (a B) b = (a A) b, for all a and b,\n\nwhich implies that A = B! In other words the matrix for an operator B\nand its adjoint A are exactly the same. Except that B is treated as\nacting on the left on column vectors, wile A is treated as acting on row\nvectors but on the right. If you don\'t like this writing things\nbackward, everything can be put right using the transpose:\n\na (B b) = (a B) b = (B^T a^T)^T b.\n\nIn other words, if B is an n x m matrix, its transpose B^T is an m x n\nmatrix which is the adjoint of B that acts by left multiplication on\nn-component column vectors which are actually transposes of n-component\nrow vectors. Phew! that was a mouthful.\n\nThe only danger here is forgetting that the meaning of the transpose is\nnot invariant under change of basis. One has to remember that a change\nof basis changes the matrix elements of the metric, so it is no longer\nrepresented by the identity matrix in the new basis. Also, abstract\nnotation for linear algebra sheds new light on classic theorems of\nmatrix algebra. For example, using the relation between the transpose\nand the metric, we can easily show that the orthogonal complement of the\ncolumn space of a matrix is the null space of its transpose. (Exercise:\nprove it!)\n\nAnother disadvantage of matrix notation is its two-dimensionality. In\nother words, it is hard to work with tensors of rank higher than two.\nAny second rank tensor can be written as a matrix with the appropriate\nindices raised or lowered by the implied metric. However, there is no\nconvenient notation for tensors of higher rank.\n\nWorking with higher order tensor products is where diagramatic notation\nfor linear algebra (with which some patrons of this group are enamoured)\nreally shines. It is interesting that the area where matrix notation\nshines equally brightly is working with direct sums of vector spaces.\n\nNamely, if n = n1 + n2 + ... + nm, then the vector space R^n can be\nwritten as a direct sum R^n = R^n1 (+) R^n2 (+) ... (+) R^nm. The\nelements of R^n are just column vectors in n components and the fact\nthat they come from the above direct sum is illustrated by writing the\nn-component column as a stack of shorter columns of n1, n2, ..., nm\ncomponents:\n\n[a^1_1 ]\n[ ... ]\n[a^1_n1]\n------\n[a^2_1 ]\n[ ... ]\n[a^1_n2] .\n------\n...\n------\n[a^m_nm]\n[ ... ]\n[a^m_nm]\n\nThis representation of a direct sum gives a very hands on picture of\nwhat is in essence a very abstract concept. The abstract (category\ntheoretial) definition of the direct sum is illustrated by the following\ncommutative diagram\n\ni_A i_B\nA ---------> A(+)B <--------- B\n| | |\n| | |\n| f_A | f f_B |\n\\ | /\n\\ | /\n\\ V /\n+--------> C <--------+\n\nThis diagram says that given two vector spaces A and B, their direct sum\nis a vector space A(+)B equipped with embeddings i_A:A->A(+)B and\ni_B:A->A(+)B such that linear maps f_A:A->C and f_B:B->C into any space C\ndefine a unique linear map f:A(+)B->C which commutes with the\nembeddings. This definition is indeed quite abstract and at first hard\nto wrap ones brain around, but ane example with matrices clears\neverything up.\n\nNamely, if A = R^n and B = R^m, with f_A and f_B being k x n and k x m\nmatrices respectively (i.e. they map A and B into C=R^k). Then the\ndesired linear map f from R^(n+m) = A(+)B into C is defined by writing\nits matrix in block form:\n\n[a_1] \\\n[...] <- n components |\n[ | ] [a_n] |\nf = [ f_A | f_B ] acting on --- > n+m components\n[ | ] [b_1] |\n^ ^ [...] <- m components |\n| | [b_m] /\nk x n k x m\n\\----- -----/\nV\nk x (n+m)\n\nThe embeddings of R^n and R^m into R^(n+m) are quite obviously defined\nand existence and uniqueness of f given f_A and f_B is clear and\nindisputable.\n\nThis illustration of category theoretical concepts can be pushed even\nfurther. The category theoretical definition of the direct sum is in\nfact the coproduct in the category of linear spaces. But, by a happy\ncoincidence, if we look at vector spaces as simple sets instead of\nlinear spaces, the direct sum in in fact the (cartesian) product of two\nspaces in this category (the coproduct in the category of sets is the\ndisjoint union, while the product in the category of linear spaces is\nthe tensor product). Perhaps the happy coincidence is a contravariant\nfunctor that\'s lurking in the background somewhere.\n\nLets look at the category theoretical definition of the product A x B of\ntwo spaces A and B (just reverse all the arrows from the definition of a\ncoproduct):\n\np_A p_B\nA <--------- A x B ---------> B\n^ ^ ^\n| | |\n| f_A | f f_B |\n\\ | /\n\\ | /\n\\ | /\n+--------- C ---------+\n\nThis commutative diagram tells us that the cartesian product of two\nsets A and B is another set A x B together with projections p_A:AxB->A\nand p_B:AxB->B (think (a,b) goes to a or b respectively) such that given\nany function functions f_A:C->A and f_B:C->B there is a unique function\nf:C->AxB which commutes with the projections [think f(c)=(f_A(c),f_B(c))].\n\nOnce again, this construction can be illustrated by writing matrix\nassociated with f in block form. Namely, as before, let C=R^k, A=R^n,\nB=R^m, and AxB=R^(n+m). Consequently, if f_A and f_B are respectively\nn x k and m x k matrices, we can write the matrix for the linear map f\nas\n\nn x k\n|\nV\n[a_1] [ ]\n[...] [ f_A ]\n[a_n] [ ] [c_1]\n--- = f c = ----- [...] <-- k components\n[b_1] [ ] [c_k]\n[...] [ f_B ]\n[b_m] [ ]\n^\n|\nm x k\n\nOnce again, projections the projections are naturally defined, and the\nexistence and uniquness of f is also beyond dispute.\n\nWhat is really neat is that we can combine these two pieces of notation\nand write a linear map between two direct sums A(+)B and C(+)D in block\nform as well. Namely a map f:A(+)B->C(+)D can be represented as\n\nk x n k x m\n[c_1] [ | ] [a_1]\n[...] [ f_AC | f_BC ] [...]\n[c_k] [ | ] [a_n]\n--- = ------ ------ ---\n[d_1] [ | ] [b_1]\n[...] [ f_AD | f_BD ] [...]\n[d_l] [ | ] [b_m]\nl x n l x m\n\nTo me this seems like a perfect union between the practical and the\nabstract. This category theory provides the deep reason why working with\nmatrices in block form always works (which can appear a little\nmysterious when first encountered) while, at the same time, block\ndecomposition of matrices provides a very hands on example of some\nabstract constructions in linear algebra or category theory.\n\nI wonder if there are similar tricks that can be played with the tensor\nproduct of vector spaces. Can the tensor product be represented as a\ncoproduct in another category? If so, is there an analogous "block\ndecomposition" of maps between tensor products of spaces in diagramatic\nnotation?\n\nI wish there was a way to combine column-row notation with diagramatic\nnotation could somehow be combined so that the respective ease of working\nwith direct sums and tensor products could be harnessed at the same\ntime.\n\nIf you got this far, I hope you enjoyed reading this post as much as I\nenjoyed writing it. :-)\n\nIgor\n\n</UL></PRE></font></td></tr></table></BODY><HTML>');"> <IMG SRC=/images/buttons/ip.gif BORDER=0 ALIGN=CENTER ALT="View this Usenet post in original ASCII form"> View this Usenet post in original ASCII form </a></div><P></jabberwocky>On Wed, 24 Nov 2004 07:46:37 +0000, Blake Winter wrote:
> Yes, I feel the same way about the horrors of matrix notation, what with
> the row vectors and column vectors hiding what's actually going on...
I think matrix notation is not as bad as some people make it seem. Being
familiar with matrices provides a great step towards either index or
index-free tensor notation. One just has to realize that there is a hidden
structure when things are written in row and column notation.
I don't think anyone would disagree that column vectors form a good honest
vector space, just as well as row vectors do. The key observation is that,
once things are written in this form, we have access to two things: matrix
multiplication and transposition. With matrix multiplication we can
naturally pair row and column vectors to give real numbers:
[b_1][b_2][a_1 a_2 a_3 .[/itex].. a_n] [b_3] = a_1 b_1 + a_2 b_2 + ... + a_n b_n .
[...]
[b_n]
This pairing clearly makes the space of row vectors dual to that of the
column vectors. Immediately we have a direct, almost pictorial,
illustration of a concept that's a little hard to grasp when starting
out in abstract linear algebra.
This even makes sense when we consider the definitions of matrices and
their multiplication whith which we were brought up. An n x m matrix is
a linear map from the space of m-component column vectors to the space
of n-component column vectors. And matrix multiplication is just
composition of these maps. By that logic, row vectors (1 x n matrices)
are maps from n-component column vectors to real numbers (1-component
vectors), and that's exactly the definition of a dual space!
Transposition provides an identification between column vectors and
row vectors:
T
[a_1] T [a_1][a_2] = [a_1 a_2 ... a_n] , [a_1 a_2 ... a_n] = [a_2] .
[...] [...]
[a_n] [a_n]
But, since row vectors are the duals of column vectors, what we have is
a canonoical identification between vectors and their duals that is
non-degenerate (invertible) and positive definite (a^T a > for every
non-zero a). In other words we have a metric on our hands! Since we have
a metric, any 2-form can be associated with an operator (a matrix) just
by raising an index: g(a,b) = a^T g b, where a and b are row vectors and
now the g in between is the matrix corresponding to the 2-form g(.,.).
In particular the metric itself can be written
[1 ... 0] [b_1][0 1 ... 0] [b_2]
<a,b> = a^T b = [a_1 a_2 ... a_n] [ ... ] [...] ,
[0 ... 1] [b_n]
it is represented by the identity matrix.
Some basic theorems of abstract (finite dimensional) linear algebra
become trivial when written in terms of matrices. For instance, a vector
space V is isomorphic to its dual V^*. Well of course, just take the
transpose! A vector space V is _canonically_ isomorphic to its double
dual V^**. Well, of course, just take the transpose twice and you end up
where you started!
There is also a nice duality between column and row notation. We were
taught as toddlers to vectors are written as columns and that linear
operators are applied to them by matrix multiplication on the left.
However, there is nothing wrong with doing things backwards, writing
vectors as rows and applying linear operators to them by matrix
multiplication on the right. There is even a deeper (somewhat hidden)
meaning to this duality. It's called the adjoint!
Denoting by <a,b> the pairing between a vector b and a dual vector a, we
call A the adjoint of B if <Aa,b> = <a,Bb> for all a and b. Notice that
while B is a map from V to W (with b in V), the adjoint A will be a map
from the dual W^* to the dual V^* (with a in W^*). In matrix notation
this becomes trivial. If a is a row vector with n components, b is a
column vector with m components, and B is an n x m matrix, to find A the
adjoint of B all we have to do is move some parentheses around:
a (B b) = (a B) b = (a A) b, for all a and b,
which implies that A = B! In other words the matrix for an operator B
and its adjoint A are exactly the same. Except that B is treated as
acting on the left on column vectors, wile A is treated as acting on row
vectors but on the right. If you don't like this writing things
backward, everything can be put right using the transpose:
a (B b) = (a B) b = (B^T a^T)^T b.
In other words, if B is an n x m matrix, its transpose B^T is an m x n
matrix which is the adjoint of B that acts by left multiplication on
n-component column vectors which are actually transposes of n-component
row vectors. Phew! that was a mouthful.
The only danger here is forgetting that the meaning of the transpose is
not invariant under change of basis. One has to remember that a change
of basis changes the matrix elements of the metric, so it is no longer
represented by the identity matrix in the new basis. Also, abstract
notation for linear algebra sheds new light on classic theorems of
matrix algebra. For example, using the relation between the transpose
and the metric, we can easily show that the orthogonal complement of the
column space of a matrix is the null space of its transpose. (Exercise:
prove it!)
Another disadvantage of matrix notation is its two-dimensionality. In
other words, it is hard to work with tensors of rank higher than two.
Any second rank tensor can be written as a matrix with the appropriate
indices raised or lowered by the implied metric. However, there is no
convenient notation for tensors of higher rank.
Working with higher order tensor products is where diagramatic notation
for linear algebra (with which some patrons of this group are enamoured)
really shines. It is interesting that the area where matrix notation
shines equally brightly is working with direct sums of vector spaces.
Namely, if n = n1 + n2 + ... + nm, then the vector space R^n can be
written as a direct sum R^n = R^{n1} (+) R^{n2} (+) ... (+) R^{nm}. The
elements of R^n are just column vectors in n components and the fact
that they come from the above direct sum is illustrated by writing the
n-component column as a stack of shorter columns of n1, n2, ..., nm
components:
[a^{1_1} ]
[ ... ][a^{1_}{n1}]
------
[a^{2_1} ]
[ ... ][a^{1_}{n2}] .
------
...
------
[a^{m_}{nm}]
[ ... ][a^{m_}{nm}]
This representation of a direct sum gives a very hands on picture of
what is in essence a very abstract concept. The abstract (category
theoretial) definition of the direct sum is illustrated by the following
commutative diagram
i_A i_B
A ---------> A(+)B <--------- B
| | || | || f_A | f f_B |\ | /\ | /
\ V /
+--------> C <--------+
This diagram says that given two vector spaces A and B, their direct sum
is a vector space A(+)B equipped with embeddings i_A:A->A(+)B and
i_B:A->A(+)B such that linear maps f_A:A->C and f_B:B->C into any space C
define a unique linear map f:A(+)B->C which commutes with the
embeddings. This definition is indeed quite abstract and at first hard
to wrap ones brain around, but ane example with matrices clears
everything up.
Namely, if A = R^n and B = R^m, with f_A and f_B being k x n and k x m
matrices respectively (i.e. they map A and B into C=R^k). Then the
desired linear map f from R^(n+m) = A(+)B into C is defined by writing
its matrix in block form:
[a_1] \
[...] <- n components |
[ | ] [a_n] |f = [ f_A | f_B ] acting on --- > n+m components
[ | ] [b_1] |^ ^ [...] <- m components |
| | [b_m] /k x n k x m
\----- -----/
V
k x (n+m)
The embeddings of R^n and R^m into R^(n+m) are quite obviously defined
and existence and uniqueness of f given f_A and f_B is clear and
indisputable.
This illustration of category theoretical concepts can be pushed even
further. The category theoretical definition of the direct sum is in
fact the coproduct in the category of linear spaces. But, by a happy
coincidence, if we look at vector spaces as simple sets instead of
linear spaces, the direct sum in in fact the (cartesian) product of two
spaces in this category (the coproduct in the category of sets is the
disjoint union, while the product in the category of linear spaces is
the tensor product). Perhaps the happy coincidence is a contravariant
functor that's lurking in the background somewhere.
Lets look at the category theoretical definition of the product A x B of
two spaces A and B (just reverse all the arrows from the definition of a
coproduct):
p_A p_B
A <--------- A x B ---------> B
^ ^ ^| | || f_A | f f_B |\ | /\ | /\ | /
+--------- C ---------+
This commutative diagram tells us that the cartesian product of two
sets A and B is another set A x B together with projections p_A:AxB->A
and p_B:AxB->B (think (a,b) goes to a or b respectively) such that given
any function functions f_A:C->A and f_B:C->B there is a unique function
f:C->AxB which commutes with the projections [think f(c)=(f_A(c),f_B(c))].
Once again, this construction can be illustrated by writing matrix
associated with f in block form. Namely, as before, let C=R^k, A=R^n,B=R^m, and AxB=R^(n+m). Consequently, if f_A and f_B are respectively
n x k and m x k matrices, we can write the matrix for the linear map f
as
n x k
|
V
[a_1] [ ]
[...] [ f_A ][a_n] [ ] [c_1]
--- = f c = ----- [...] <-- k components
[b_1] [ ] [c_k]
[...] [ f_B ][b_m] [ ]
^
|
[itex]m x k
Once again, projections the projections are naturally defined, and the
existence and uniquness of f is also beyond dispute.
What is really neat is that we can combine these two pieces of notation
and write a linear map between two direct sums A(+)B and C(+)D in block
form as well. Namely a map f:A(+)B->C(+)D can be represented as
k x n k x m
[c_1] [ | ] [a_1]
[...] [ f_{AC} | f_{BC} ] [...]
[c_k] [ | ] [a_n]
--- = ------ ------ ---
[d_1] [ | ] [b_1]
[...] [ f_{AD} | f_{BD} ] [...]
[d_l] [ | ] [b_m]l x n l x m
To me this seems like a perfect union between the practical and the
abstract. This category theory provides the deep reason why working with
matrices in block form always works (which can appear a little
mysterious when first encountered) while, at the same time, block
decomposition of matrices provides a very hands on example of some
abstract constructions in linear algebra or category theory.
I wonder if there are similar tricks that can be played with the tensor
product of vector spaces. Can the tensor product be represented as a
coproduct in another category? If so, is there an analogous "block
decomposition" of maps between tensor products of spaces in diagramatic
notation?
I wish there was a way to combine column-row notation with diagramatic
notation could somehow be combined so that the respective ease of working
with direct sums and tensor products could be harnessed at the same
time.
If you got this far, I hope you enjoyed reading this post as much as I
enjoyed writing it. :-)
Igor
> Yes, I feel the same way about the horrors of matrix notation, what with
> the row vectors and column vectors hiding what's actually going on...
I think matrix notation is not as bad as some people make it seem. Being
familiar with matrices provides a great step towards either index or
index-free tensor notation. One just has to realize that there is a hidden
structure when things are written in row and column notation.
I don't think anyone would disagree that column vectors form a good honest
vector space, just as well as row vectors do. The key observation is that,
once things are written in this form, we have access to two things: matrix
multiplication and transposition. With matrix multiplication we can
naturally pair row and column vectors to give real numbers:
[b_1][b_2][a_1 a_2 a_3 .[/itex].. a_n] [b_3] = a_1 b_1 + a_2 b_2 + ... + a_n b_n .
[...]
[b_n]
This pairing clearly makes the space of row vectors dual to that of the
column vectors. Immediately we have a direct, almost pictorial,
illustration of a concept that's a little hard to grasp when starting
out in abstract linear algebra.
This even makes sense when we consider the definitions of matrices and
their multiplication whith which we were brought up. An n x m matrix is
a linear map from the space of m-component column vectors to the space
of n-component column vectors. And matrix multiplication is just
composition of these maps. By that logic, row vectors (1 x n matrices)
are maps from n-component column vectors to real numbers (1-component
vectors), and that's exactly the definition of a dual space!
Transposition provides an identification between column vectors and
row vectors:
T
[a_1] T [a_1][a_2] = [a_1 a_2 ... a_n] , [a_1 a_2 ... a_n] = [a_2] .
[...] [...]
[a_n] [a_n]
But, since row vectors are the duals of column vectors, what we have is
a canonoical identification between vectors and their duals that is
non-degenerate (invertible) and positive definite (a^T a > for every
non-zero a). In other words we have a metric on our hands! Since we have
a metric, any 2-form can be associated with an operator (a matrix) just
by raising an index: g(a,b) = a^T g b, where a and b are row vectors and
now the g in between is the matrix corresponding to the 2-form g(.,.).
In particular the metric itself can be written
[1 ... 0] [b_1][0 1 ... 0] [b_2]
<a,b> = a^T b = [a_1 a_2 ... a_n] [ ... ] [...] ,
[0 ... 1] [b_n]
it is represented by the identity matrix.
Some basic theorems of abstract (finite dimensional) linear algebra
become trivial when written in terms of matrices. For instance, a vector
space V is isomorphic to its dual V^*. Well of course, just take the
transpose! A vector space V is _canonically_ isomorphic to its double
dual V^**. Well, of course, just take the transpose twice and you end up
where you started!
There is also a nice duality between column and row notation. We were
taught as toddlers to vectors are written as columns and that linear
operators are applied to them by matrix multiplication on the left.
However, there is nothing wrong with doing things backwards, writing
vectors as rows and applying linear operators to them by matrix
multiplication on the right. There is even a deeper (somewhat hidden)
meaning to this duality. It's called the adjoint!
Denoting by <a,b> the pairing between a vector b and a dual vector a, we
call A the adjoint of B if <Aa,b> = <a,Bb> for all a and b. Notice that
while B is a map from V to W (with b in V), the adjoint A will be a map
from the dual W^* to the dual V^* (with a in W^*). In matrix notation
this becomes trivial. If a is a row vector with n components, b is a
column vector with m components, and B is an n x m matrix, to find A the
adjoint of B all we have to do is move some parentheses around:
a (B b) = (a B) b = (a A) b, for all a and b,
which implies that A = B! In other words the matrix for an operator B
and its adjoint A are exactly the same. Except that B is treated as
acting on the left on column vectors, wile A is treated as acting on row
vectors but on the right. If you don't like this writing things
backward, everything can be put right using the transpose:
a (B b) = (a B) b = (B^T a^T)^T b.
In other words, if B is an n x m matrix, its transpose B^T is an m x n
matrix which is the adjoint of B that acts by left multiplication on
n-component column vectors which are actually transposes of n-component
row vectors. Phew! that was a mouthful.
The only danger here is forgetting that the meaning of the transpose is
not invariant under change of basis. One has to remember that a change
of basis changes the matrix elements of the metric, so it is no longer
represented by the identity matrix in the new basis. Also, abstract
notation for linear algebra sheds new light on classic theorems of
matrix algebra. For example, using the relation between the transpose
and the metric, we can easily show that the orthogonal complement of the
column space of a matrix is the null space of its transpose. (Exercise:
prove it!)
Another disadvantage of matrix notation is its two-dimensionality. In
other words, it is hard to work with tensors of rank higher than two.
Any second rank tensor can be written as a matrix with the appropriate
indices raised or lowered by the implied metric. However, there is no
convenient notation for tensors of higher rank.
Working with higher order tensor products is where diagramatic notation
for linear algebra (with which some patrons of this group are enamoured)
really shines. It is interesting that the area where matrix notation
shines equally brightly is working with direct sums of vector spaces.
Namely, if n = n1 + n2 + ... + nm, then the vector space R^n can be
written as a direct sum R^n = R^{n1} (+) R^{n2} (+) ... (+) R^{nm}. The
elements of R^n are just column vectors in n components and the fact
that they come from the above direct sum is illustrated by writing the
n-component column as a stack of shorter columns of n1, n2, ..., nm
components:
[a^{1_1} ]
[ ... ][a^{1_}{n1}]
------
[a^{2_1} ]
[ ... ][a^{1_}{n2}] .
------
...
------
[a^{m_}{nm}]
[ ... ][a^{m_}{nm}]
This representation of a direct sum gives a very hands on picture of
what is in essence a very abstract concept. The abstract (category
theoretial) definition of the direct sum is illustrated by the following
commutative diagram
i_A i_B
A ---------> A(+)B <--------- B
| | || | || f_A | f f_B |\ | /\ | /
\ V /
+--------> C <--------+
This diagram says that given two vector spaces A and B, their direct sum
is a vector space A(+)B equipped with embeddings i_A:A->A(+)B and
i_B:A->A(+)B such that linear maps f_A:A->C and f_B:B->C into any space C
define a unique linear map f:A(+)B->C which commutes with the
embeddings. This definition is indeed quite abstract and at first hard
to wrap ones brain around, but ane example with matrices clears
everything up.
Namely, if A = R^n and B = R^m, with f_A and f_B being k x n and k x m
matrices respectively (i.e. they map A and B into C=R^k). Then the
desired linear map f from R^(n+m) = A(+)B into C is defined by writing
its matrix in block form:
[a_1] \
[...] <- n components |
[ | ] [a_n] |f = [ f_A | f_B ] acting on --- > n+m components
[ | ] [b_1] |^ ^ [...] <- m components |
| | [b_m] /k x n k x m
\----- -----/
V
k x (n+m)
The embeddings of R^n and R^m into R^(n+m) are quite obviously defined
and existence and uniqueness of f given f_A and f_B is clear and
indisputable.
This illustration of category theoretical concepts can be pushed even
further. The category theoretical definition of the direct sum is in
fact the coproduct in the category of linear spaces. But, by a happy
coincidence, if we look at vector spaces as simple sets instead of
linear spaces, the direct sum in in fact the (cartesian) product of two
spaces in this category (the coproduct in the category of sets is the
disjoint union, while the product in the category of linear spaces is
the tensor product). Perhaps the happy coincidence is a contravariant
functor that's lurking in the background somewhere.
Lets look at the category theoretical definition of the product A x B of
two spaces A and B (just reverse all the arrows from the definition of a
coproduct):
p_A p_B
A <--------- A x B ---------> B
^ ^ ^| | || f_A | f f_B |\ | /\ | /\ | /
+--------- C ---------+
This commutative diagram tells us that the cartesian product of two
sets A and B is another set A x B together with projections p_A:AxB->A
and p_B:AxB->B (think (a,b) goes to a or b respectively) such that given
any function functions f_A:C->A and f_B:C->B there is a unique function
f:C->AxB which commutes with the projections [think f(c)=(f_A(c),f_B(c))].
Once again, this construction can be illustrated by writing matrix
associated with f in block form. Namely, as before, let C=R^k, A=R^n,B=R^m, and AxB=R^(n+m). Consequently, if f_A and f_B are respectively
n x k and m x k matrices, we can write the matrix for the linear map f
as
n x k
|
V
[a_1] [ ]
[...] [ f_A ][a_n] [ ] [c_1]
--- = f c = ----- [...] <-- k components
[b_1] [ ] [c_k]
[...] [ f_B ][b_m] [ ]
^
|
[itex]m x k
Once again, projections the projections are naturally defined, and the
existence and uniquness of f is also beyond dispute.
What is really neat is that we can combine these two pieces of notation
and write a linear map between two direct sums A(+)B and C(+)D in block
form as well. Namely a map f:A(+)B->C(+)D can be represented as
k x n k x m
[c_1] [ | ] [a_1]
[...] [ f_{AC} | f_{BC} ] [...]
[c_k] [ | ] [a_n]
--- = ------ ------ ---
[d_1] [ | ] [b_1]
[...] [ f_{AD} | f_{BD} ] [...]
[d_l] [ | ] [b_m]l x n l x m
To me this seems like a perfect union between the practical and the
abstract. This category theory provides the deep reason why working with
matrices in block form always works (which can appear a little
mysterious when first encountered) while, at the same time, block
decomposition of matrices provides a very hands on example of some
abstract constructions in linear algebra or category theory.
I wonder if there are similar tricks that can be played with the tensor
product of vector spaces. Can the tensor product be represented as a
coproduct in another category? If so, is there an analogous "block
decomposition" of maps between tensor products of spaces in diagramatic
notation?
I wish there was a way to combine column-row notation with diagramatic
notation could somehow be combined so that the respective ease of working
with direct sums and tensor products could be harnessed at the same
time.
If you got this far, I hope you enjoyed reading this post as much as I
enjoyed writing it. :-)
Igor