Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Transforming a matrix to orthogonal one

  1. Oct 4, 2012 #1
    Suppose a matrix X of size n x p is given, n>p, with p linearly independent columns. Can it be guaranteed that there exists a matrix A of size p x p that converts columns of X to orthonormal columns. In other words, is there an A, such that Y=XA, and Y^TY=I, where I is an p x p identity matrix.
  2. jcsd
  3. Oct 4, 2012 #2


    User Avatar
    Science Advisor

    Yes. Since the columns of X are independent, they for a basis for Rn. We can then use the "Gram-Schmidt orthogonalization" process to construct an orthonormal basis from them. A will be the "change of basis" matrix that changes representation of a vector in the original basis to representation in the orthonormal basis.
  4. Oct 4, 2012 #3
    Thanks. Just one note: I suppose you've taken into account that there are p columns in X (which is an n x p matrix). If I'm not wrong, only n linearly independent columns of dimensionality R^n define a basis in R^n.

    So, given an input X, with linearly independent columns, such columns could be transformed by GS processing to yield Y, such that Y^TY=I, and there exists A, such that Y=XA. How could one calculate such A a priori?
  5. Oct 4, 2012 #4
    Yes Onako, it is true, but for a different reason, than stated by HallsofIvy.

    If the columns of [itex]X[/itex] are linearly independent, [itex]X^*X[/itex] is an invertible [itex]p\times p[/itex] matrix (here [itex]X^*[/itex] is the Hermitian conjugate of [itex]X[/itex], i.e. the conjugate transpose of [itex]X[/itex] ; if [itex]X[/itex] is real then [itex]X^*=X^T[/itex] (I use [itex]X^*[/itex] only because what I say works for complex matrices as well).

    Matrix [itex]X^*X[/itex] is positive semidefinite for all [itex]X[/itex] , and since [itex]X^*X[/itex] is invertible, [itex]X^*X[/itex] is positive definite (all eigenvalues are positive). Since the matrix [itex]X^*X[/itex] is Hermitian (symmetric if [itex]X[/itex] is real), it is can be diagonalized, i.e. it can be represented as a diagonal matrix in some orthonormal basis, or equivalently, it can be written as [itex]X^*X =U^* D U[/itex], where [itex]U[/itex] is a unitary matrix ([itex]U^{-1}=U^*[/itex]) and [itex]D[/itex] is a diagonal matrix with eigenvalues of [itex]X^*X[/itex] on the diagonal.

    We can take a square root of [itex]X^*X[/itex], namely [itex]B = U^* D^{1/2} U[/itex], where [itex]D^{1/2}[/itex] is obtained by taking square roots of diagonal entries of [itex]D[/itex] (recall that [itex]D[/itex] is a diagonal matrix). Then [itex]B^*=B[/itex], and [itex]X^*X = B^2[/itex], and [itex]A=B^{-1}[/itex] is the matrix you want.

    Indeed, if [itex]Y=XA[/itex], then [itex]Y^*Y = A^* X^*X A = A^* B^2 A =A B^2 A =I[/itex].
  6. Oct 5, 2012 #5
    Thank your for such a good explanation.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook