Pseudo inverse very inaccurate

  • Thread starter divB
  • Start date
  • Tags
    Inverse
In summary, the conversation discusses a numerical problem in MATLAB involving a matrix and vectors that are very close. The result of using the least squares method is not accurate due to the high condition number of the matrix. Suggestions are made to use the pinv() function instead, which takes into account the linear dependency of both rows and columns in the matrix. There is also a discussion about the different algorithms used by \ and pinv() to solve such problems.
  • #1
divB
87
0
Hi,

I've a very trivial numerical problem where I'm currently stuck. In MATLAB the matrix Hf:

Code: 
>> Hf
Hf =

  1.0e+003 *

    1.6443    1.6516    1.6583
    4.8373    4.8349    4.8334
    4.6385    4.6418    4.6445
   -9.6014   -9.6084   -9.6154

And the following vectors which are very close:

Code: 
>> [yl1 , yl3 , yl1 - yl3 ]
ans =

  1.0e+006 *

    0.2966    0.2972   -0.0006
    0.8705    0.8703    0.0002
    0.8352    0.8355   -0.0003
   -1.7288   -1.7295    0.0006

yl1 is my result as it should be:

Code: 
>> Hf \ yl1
ans =

  100.0000
   75.0000
    5.0000

yl3 is obtained in a different way but is very close to the original. But sill:

Code: 
>> Hf \ yl3
ans =

   56.0412
   72.5578
   51.4007

The result is not just a little bit away, it is terrible, unuseable!

I have much redundancy in the data, so I can ad much lines to the matrix Hf. However, it does not matter how much, the result is always the same ... unuseable.

Can anyone explain why the least squares is so terrible in this case? I'm a bit confused because least squares should be pretty robust ...

Thanks,
divB
 
Mathematics news on Phys.org
  • #2
Hmm, sorry I think I know: The columns of Hf are too close, right?
 
  • #3
divB said:
Hmm, sorry I think I know: The columns of Hf are too close, right?

Likely, yes. The columns are very nearly linearly dependent, which can make calculations like the pseudoinverse numerically unstable.
 
  • #4
Hmm, thanks. Is there a good way to stabilize such a system? E.g. I have y=Hf*h; I want to find h and may modify y and Hf
 
  • #5
It's been a while since I did this myself, but have you tried looking at the various numeric iterative schemes to solve systems with a high condition number?
 
  • #6
Hmm, I wonder why MATLAB would not implement these itself?
Anycase, I get similar results when the condition number is not so high, e.g. cond(Hf) = 183. Starting from which condition number could one say that the LS becomes unstable?

I am currently trying to reformulate the problem to avoid these similar columns. In my current setup I sum over a series where each column differs only with one sample of the series. Clearly, this yields very similar numbers in each column per row.

However, I am currently confused about the linear dependency of the rows/columns in such a linear system. Can you clearify that? I thought for the solvability, only the linear dependency of the rows (rather than the cols) plays a role?

E.g. when there are fewer lin. independent rows than unknowns, then the system cannot be solved.

So does this mean that the rows AND columns must be lin. independent?

I found no clear explanation.
 
  • #7
That is an extremely high condition number: 183 is very big.
 
  • #8
Instead of Hy \ y1, try pinv(Hf)*y1. That may give you more "consistent" results.

Judging by http://www.mathworks.co.uk/help/matlab/ref/arithmeticoperators.html, \ uses a fairly simple minded (but quick) algorithm to solve badly conditioned problems. The bottom line is that the solution isn't unique, because there is a nonzero vector v such that Hf.v = 0, and you can add any multiple of v to the "solution" to get another solution. And the algorithm that \ uses doesn't care what that multiple is, but the algorithm for pinv() does care - it also tries to make the components of the "answer" as small as possible.
 

1. What is a pseudo inverse and why is it used in science?

A pseudo inverse is a mathematical tool used in science to find an approximate solution to a system of equations that does not have a unique solution. It is useful in situations where the equations are over-determined or when there is no exact solution.

2. Why is the pseudo inverse often considered to be inaccurate?

The pseudo inverse is considered to be inaccurate because it is an approximation and not an exact solution. It may also introduce errors due to rounding or truncation during the calculation process.

3. How does the accuracy of a pseudo inverse compare to other methods of solving equations?

The accuracy of a pseudo inverse can vary depending on the specific problem and the method used to calculate it. In some cases, it may provide a more accurate solution compared to other methods, while in other cases it may be less accurate. It is important to evaluate the accuracy of the pseudo inverse for each specific problem.

4. What factors can affect the accuracy of a pseudo inverse?

The accuracy of a pseudo inverse can be affected by several factors, such as the number of equations in the system, the condition of the equations (e.g. if they are nearly linearly dependent), and the method used to calculate the pseudo inverse.

5. Are there any ways to improve the accuracy of a pseudo inverse?

Yes, there are some ways to improve the accuracy of a pseudo inverse. One approach is to use a more sophisticated method for calculating the pseudo inverse, such as the singular value decomposition method. Additionally, using a higher precision arithmetic or reducing the rounding and truncation errors during the calculation process can also improve the accuracy.

Hot Threads

Recent Insights

Back
Top