Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Linear Algebra Question (Trace)

  1. Nov 6, 2008 #1
    I am trying to simplify the following, so that I can differentiate it (with respect to X). Ideally I'll have everything in terms of ([tex]XX^T - YY^T[/tex]).

    [tex]
    \mathrm{trace}[(AX(AX)^T)((BX)(BX)^T)^{-1}] -
    \mathrm{trace}[(AY(AY)^T)((BY)(BY)^T)^{-1}]
    [/tex]

    Where X and Y are 3 x N and A and B are N x N. A is symmetrical, B is anti-symmetrical (or skew symmetrical).

    Some useful properties:
    [tex]
    \mathrm{trace}(UV) = \mathrm{trace}(VU)
    [/tex]
    [tex]
    \mathrm{trace}(U)+\mathrm{trace}(V) = \mathrm{trace}(U+V)
    [/tex]

    I can't figure this out and have spent a long time working on it.
    A is a band diagonal matrix where each row is a shifted version of (1 -2 1) and B is similar with a stencil of (-1 0 1).

    Any ideas?
     
  2. jcsd
  3. Nov 7, 2008 #2

    tiny-tim

    User Avatar
    Science Advisor
    Homework Helper

    Welcome to PF!

    Hi GeoffO! Welcome to PF! :smile:

    Hint: (PQ)T = QTPT.

    And XXT is an n x n matrix.

    So just change the order of everything. :wink:
     
  4. Nov 7, 2008 #3
    Thanks for the reply. I have used this property and end up with
    [tex]
    \mathrm{trace}[
    (B^{-1}A)^T
    (XX^T(B^{-1}A)(X^TX)^{-1} -
    YY^T(B^{-1}A)(Y^TY)^{-1}
    ]
    [/tex]

    Which is okay, but I'd really like to get that common [tex](B^{-1}A)[/tex] term out of there.

    We can rewrite this as
    [tex]
    (Q^T)(R^T Q R^{-1} - S^T Q S^{-1})
    [/tex]
    Where R and S are symmetrical (because they're squared X and Y from above). This looks so simple, but I can not find a way to further simplify.
     
  5. Nov 7, 2008 #4

    tiny-tim

    User Avatar
    Science Advisor
    Homework Helper

    Now change the order of everything …

    put all the As and Bs on the left, and the XXT or YYT on the right. :smile:
     
  6. Nov 7, 2008 #5
    What property allows me to reorder like that? I only know of the property
    [tex]\mathrm{trace}(AB) = \mathrm{trace}(BA)[/tex]
    (Again, thank you.)
     
  7. Nov 7, 2008 #6

    tiny-tim

    User Avatar
    Science Advisor
    Homework Helper

    Yes, and it works for any square matrices …

    and XXT is square, so you can shove it off to one end. :smile:
     
  8. Nov 7, 2008 #7
    I see. You mean the following, right?
    [tex]
    \mathrm{trace}[
    (B^{-1}A)(X^TX)^{-1}(B^{-1}A)^T (XX^T) -
    (B^{-1}A)(Y^TY)^{-1}(B^{-1}A)^T (YY^T)
    ]
    [/tex]

    I would like to factor out the [tex](B^{-1}A)[/tex] terms, if possible. (Thanks!)
     
  9. Nov 7, 2008 #8

    tiny-tim

    User Avatar
    Science Advisor
    Homework Helper

    No, I mean
    [tex]Tr(B^{-1}A)(B^{-1}A)^T Tr(XX^T)(X^TX)^{-1} -
    Tr(B^{-1}A)(B^{-1}A)^T Tr(YY^T)(Y^TY)^{-1}[/tex]
     
  10. Nov 7, 2008 #9
    Hmm... somewhere I have made an error. [tex](X^TX)^{-1}[/tex] is a 3x3, so this doesn't make sense... back in a minute after I figure out my error.

    In the meantime how did you introduce multiplication of traces? What property am I missing? This looks promising!
     
  11. Nov 7, 2008 #10

    tiny-tim

    User Avatar
    Science Advisor
    Homework Helper

    oops … I got carried away :redface: … I meant

    [tex]Tr(B^{-1}A)(B^{-1}A)^T (XX^T)(X^TX)^{-1} -
    Tr(B^{-1}A)(B^{-1}A)^T (YY^T)(Y^TY)^{-1}[/tex]
     
  12. Nov 7, 2008 #11
    Yeah, sorry the inverted terms should have been [tex](XX^T)^{-1}[/tex] and [tex](YY^T)^{-1}[/tex]. This would result in all of the X and Y terms canceling out in your reduction. I'm excited to learn what property allows for this step!
     
  13. Nov 7, 2008 #12
    Even still, what allows you to go from [tex]Tr(M N M^T N^{-1})[/tex] to [tex]Tr(M M^T N N^{-1})[/tex]?

    This is not just a rotation, it's a reordering, right?
     
  14. Nov 7, 2008 #13
    ** solved **

    I see how you did it, for any three square **SYMMETRIC** (or anti-symmetric) matrices you can reorder at will because of these properties.

    The first [tex]Tr(AB) = Tr(BA)[/tex] allows us to write [tex]Tr(ABC) = Tr(BCA)[/tex] and the like.

    The second is [tex]Tr(A) = Tr(A^T)[/tex].

    With these we can permute any three square **SYMMETRIC** (or anti-symmetric) matrices. Here is a proof:
    [tex]Tr(ABC) = Tr(A^T B^T C^T) = Tr((CBA)^T) = Tr(CBA) = Tr(ACB) = Tr(BAC)[/tex]
    Where the last two steps are just an application of the first property.

    So, we can say [tex]A= MN[/tex], [tex]B=M^T[/tex] and [tex]C=N^{-1}[/tex] which allows us to get the result you indicate.

    Thank you very much, you have solved by problem!

    As a side note X and Y are not invertible, but using the pseudoinverse allows me to approximate the least squares solution that I am after.

    Thanks, I hope to give back a bit to the forum in the future.
     
    Last edited: Nov 7, 2008
  15. Nov 7, 2008 #14

    tiny-tim

    User Avatar
    Science Advisor
    Homework Helper

    That's it! :biggrin:

    Except that I think it only works for three symmetric and/or anti-symmetric square matrices, because of the step Tr(ABC) = Tr(AT BT CT) …

    which is ok in this case because XXT is symmetric. :wink:
     
  16. Nov 7, 2008 #15
    Good point.... It's careful surgery. I'll have to double check, not everything here is symmetrical, so being sure to associate carefully is very important.

    (I edited the post above lest I confuse any future reader.)
     
  17. Nov 7, 2008 #16

    tiny-tim

    User Avatar
    Science Advisor
    Homework Helper

    You can put and/or anti-symmetric in also, even for an odd number …

    if A ad C are symmetric and B is anti-symmetric, then ABC is anti-symmetric, so you get a -1 for BT and for (ABC)T, so it's still +1 in the end. :smile:

    EDIT: oooh no, that's rubbish … if A and B are both symmetric, then AB needn't be, although AB + BA will be … and if A is symmetric and B is anti-symmetric, then AB needn't be anti-symmetric, although AB - BA will be.

    But the traces still work for a symmetric A and C and an anti-symmetric B …

    Tr(ABC) = -Tr(BAC)

    though not for three symmetric and one anti-symmetric …

    Tr(ABCD) = -Tr(BADC), not (BACD)

    And B-1A isn't symmetric or anti-symmetric anyway, so I think it's back to square one … :frown:
     
    Last edited: Nov 8, 2008
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?



Similar Discussions: Linear Algebra Question (Trace)
  1. Linear Algebra? (Replies: 4)

Loading...