What is the purpose of the transpose?

[daviddoria]

Every book I've seen starts out with "to find the transpose, make B_ij = A_ji . However, they don't explain exactly why would would want to do this.

Ie. they tell you the inverse is useful because if you have Ax = b, you can find x by writing b = A^{-1} x.

The only thing I can think of to do with the transpose is visualize the row space by plotting A^T x where x is a bunch of vectors from a unit circle.

Does anyone have anything better to say about transposes?

Thanks,
Dave

[statdad]

In multiple regression (for one case), the estimates of the unknown regression coefficients are the solutions to the system of equations


X \widehat \beta = Y


where X is not a square matrix


X \text{ is } n \times p, \quad \widehat \beta \text{ is } p \times 1, \quad Y \text{ is } n \times 1



The classical solution assumes that X is full-rank, so the solutions can be written as


(X' X) \widehat \beta = X' Y \Rightarrow \widehat \beta = (X' X)^{-1} X'Y


- here the transpose of a matrix is used to obtain a system of equations that can be solved with the method of matrix inverses.
The transpose of X also plays an important role in estimating variances and covariances in regression.

I'm not sure this answers your question entirely, but it is a start.

[morphism]

Another reason is that the transpose (and more importantly the conjugate transpose) comes up quite a bit in the study of the 'structure' of matrices. It turns out we can say a lot about a matrix if we know that it's equal to its transpose (i.e. A=A^T) or even if it merely commutes with it (i.e. AA^T=A^TA). Two buzzwords here are "symmetric matrices" and "normal matrices."

A small elaboration: the process of taking the conjugate transpose of a matrix is somewhat analogous to the process of taking the conjugate of a complex number. This analogy has surprisingly far-reaching outcomes.

[daviddoria]

statdad, I'm familiar with the pseudo inverse. The derivation is from assuming (correctly) that the error in the least squared solution is orthogonal to the best solution. But that seems to introduce the transpose as a side effect, rather than explain what it actually does.

morphism, i guess the question is then WHY is it special if A = A^T ? I think that means the column space is the same as the row space? But why is that so nice?

I've always thought about the "action" of a matrix by looking that the result of applying the matrix to every point on a unit sphere. I guess I'm not sure if its useful to do the same with A^T?

Dave

[morphism]

Do a google search to see why symmetric matrices are special.

[mathwonk]

a vector space has a dual space, and map of vectors spaces induces an opposite directional map between their duals. if you know the matrix of the first map in some basis, then in the dual bases, the second map's matrix is the transpose of the first one.

Some people avoid dual spaces by looking at spaces with inner products on them. Then to every map T, there is another map T* such that Tv.w = v.T*w.

this map T* is called the adjoint of the map T, and in an orthonormal basis, the matrices of T and T* are transposes of each other. you are invited to read my linear algebra notes for math 4050 on my webpage, especially the section on inner products, duals, and adjoints, including spectral theorems.

so the point is to stop focusing on the matrices themselves and think about what they represent. A matrix represents a linear map in some basis. so ask what map is represented by the transpose of the matrix of a given map.