Transforming a matrix to orthogonal one

[onako]

Suppose a matrix X of size n x p is given, n>p, with p linearly independent columns. Can it be guaranteed that there exists a matrix A of size p x p that converts columns of X to orthonormal columns. In other words, is there an A, such that Y=XA, and Y^TY=I, where I is an p x p identity matrix.

 

[HallsofIvy]

Yes. Since the columns of X are independent, they for a basis for Rⁿ. We can then use the "Gram-Schmidt orthogonalization" process to construct an orthonormal basis from them. A will be the "change of basis" matrix that changes representation of a vector in the original basis to representation in the orthonormal basis.

 

[onako]

Thanks. Just one note: I suppose you've taken into account that there are p columns in X (which is an n x p matrix). If I'm not wrong, only n linearly independent columns of dimensionality R^n define a basis in R^n.

So, given an input X, with linearly independent columns, such columns could be transformed by GS processing to yield Y, such that Y^TY=I, and there exists A, such that Y=XA. How could one calculate such A a priori?

 

[Hawkeye18]

Yes Onako, it is true, but for a different reason, than stated by HallsofIvy.

If the columns of X are linearly independent, X^*X is an invertible p\times p matrix (here X^* is the Hermitian conjugate of X, i.e. the conjugate transpose of X ; if X is real then X^*=X^T (I use X^* only because what I say works for complex matrices as well).

Matrix X^*X is positive semidefinite for all X , and since X^*X is invertible, X^*X is positive definite (all eigenvalues are positive). Since the matrix X^*X is Hermitian (symmetric if X is real), it is can be diagonalized, i.e. it can be represented as a diagonal matrix in some orthonormal basis, or equivalently, it can be written as X^*X =U^* D U, where U is a unitary matrix (U^{-1}=U^*) and D is a diagonal matrix with eigenvalues of X^*X on the diagonal.

We can take a square root of X^*X, namely B = U^* D^{1/2} U, where D^{1/2} is obtained by taking square roots of diagonal entries of D (recall that D is a diagonal matrix). Then B^*=B, and X^*X = B^2, and A=B^{-1} is the matrix you want.

Indeed, if Y=XA, then Y^*Y = A^* X^*X A = A^* B^2 A =A B^2 A =I.

 

[onako]

Thank your for such a good explanation.