1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Pseudo inverse very inaccurate

  1. Jan 13, 2013 #1

    I've a very trivial numerical problem where I'm currently stuck. In MATLAB the matrix Hf:

    Code (Text):

    >> Hf
    Hf =

      1.0e+003 *

        1.6443    1.6516    1.6583
        4.8373    4.8349    4.8334
        4.6385    4.6418    4.6445
       -9.6014   -9.6084   -9.6154
    And the following vectors which are very close:

    Code (Text):

    >> [yl1 , yl3 , yl1 - yl3 ]
    ans =

      1.0e+006 *

        0.2966    0.2972   -0.0006
        0.8705    0.8703    0.0002
        0.8352    0.8355   -0.0003
       -1.7288   -1.7295    0.0006
    yl1 is my result as it should be:

    Code (Text):

    >> Hf \ yl1
    ans =

    yl3 is obtained in a different way but is very close to the original. But sill:

    Code (Text):

    >> Hf \ yl3
    ans =

    The result is not just a little bit away, it is terrible, unuseable!

    I have much redundancy in the data, so I can ad much lines to the matrix Hf. However, it does not matter how much, the result is always the same ... unuseable.

    Can anyone explain why the least squares is so terrible in this case? I'm a bit confused because least squares should be pretty robust ...

  2. jcsd
  3. Jan 13, 2013 #2
    Hmm, sorry I think I know: The columns of Hf are too close, right?
  4. Jan 13, 2013 #3
    Likely, yes. The columns are very nearly linearly dependent, which can make calculations like the pseudoinverse numerically unstable.
  5. Jan 13, 2013 #4
    Hmm, thanks. Is there a good way to stabilize such a system? E.g. I have y=Hf*h; I want to find h and may modify y and Hf
  6. Jan 13, 2013 #5


    User Avatar
    Science Advisor

    It's been a while since I did this myself, but have you tried looking at the various numeric iterative schemes to solve systems with a high condition number?
  7. Jan 13, 2013 #6
    Hmm, I wonder why MATLAB would not implement these itself?
    Anycase, I get similar results when the condition number is not so high, e.g. cond(Hf) = 183. Starting from which condition number could one say that the LS becomes unstable?

    I am currently trying to reformulate the problem to avoid these similar columns. In my current setup I sum over a series where each column differs only with one sample of the series. Clearly, this yields very similar numbers in each column per row.

    However, I am currently confused about the linear dependency of the rows/columns in such a linear system. Can you clearify that? I thought for the solvability, only the linear dependency of the rows (rather than the cols) plays a role?

    E.g. when there are fewer lin. independent rows than unknowns, then the system cannot be solved.

    So does this mean that the rows AND columns must be lin. independent?

    I found no clear explanation.
  8. Jan 13, 2013 #7


    User Avatar
    Science Advisor

    That is an extremely high condition number: 183 is very big.
  9. Jan 13, 2013 #8


    User Avatar
    Science Advisor
    Homework Helper

    Instead of Hy \ y1, try pinv(Hf)*y1. That may give you more "consistent" results.

    Judging by http://www.mathworks.co.uk/help/matlab/ref/arithmeticoperators.html, \ uses a fairly simple minded (but quick) algorithm to solve badly conditioned problems. The bottom line is that the solution isn't unique, because there is a nonzero vector v such that Hf.v = 0, and you can add any multiple of v to the "solution" to get another solution. And the algorithm that \ uses doesn't care what that multiple is, but the algorithm for pinv() does care - it also tries to make the components of the "answer" as small as possible.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook