Transforming a matrix to orthogonal one

[onako]

Suppose a matrix X of size n x p is given, n>p, with p linearly independent columns. Can it be guaranteed that there exists a matrix A of size p x p that converts columns of X to orthonormal columns. In other words, is there an A, such that Y=XA, and Y^TY=I, where I is an p x p identity matrix.

 

[HallsofIvy]

Yes. Since the columns of X are independent, they for a basis for Rⁿ. We can then use the "Gram-Schmidt orthogonalization" process to construct an orthonormal basis from them. A will be the "change of basis" matrix that changes representation of a vector in the original basis to representation in the orthonormal basis.

 

[onako]

Thanks. Just one note: I suppose you've taken into account that there are p columns in X (which is an n x p matrix). If I'm not wrong, only n linearly independent columns of dimensionality R^n define a basis in R^n.

So, given an input X, with linearly independent columns, such columns could be transformed by GS processing to yield Y, such that Y^TY=I, and there exists A, such that Y=XA. How could one calculate such A a priori?

 

[Hawkeye18]

Yes Onako, it is true, but for a different reason, than stated by HallsofIvy.

If the columns of $X$ are linearly independent, $X^*X$ is an invertible $p\times p$ matrix (here $X^*$ is the Hermitian conjugate of $X$, i.e. the conjugate transpose of $X$ ; if $X$ is real then $X^*=X^T$ (I use $X^*$ only because what I say works for complex matrices as well).

Matrix $X^*X$ is positive semidefinite for all $X$ , and since $X^*X$ is invertible, $X^*X$ is positive definite (all eigenvalues are positive). Since the matrix $X^*X$ is Hermitian (symmetric if $X$ is real), it is can be diagonalized, i.e. it can be represented as a diagonal matrix in some orthonormal basis, or equivalently, it can be written as $X^*X =U^* D U$, where $U$ is a unitary matrix ($U^{-1}=U^*$) and $D$ is a diagonal matrix with eigenvalues of $X^*X$ on the diagonal.

We can take a square root of $X^*X$, namely $B = U^* D^{1/2} U$, where $D^{1/2}$ is obtained by taking square roots of diagonal entries of $D$ (recall that $D$ is a diagonal matrix). Then $B^*=B$, and $X^*X = B^2$, and $A=B^{-1}$ is the matrix you want.

Indeed, if $Y=XA$, then $Y^*Y = A^* X^*X A = A^* B^2 A =A B^2 A =I$.

 

[onako]

Thank your for such a good explanation.