Over constrained least squares analysis

[benzun_1999]

Hi,

I have a over constrained least squares problem. I also have the correct solution to the problem. Now I need to determine which of the vectors are contributing how much information that is close to the correct solution. I am sure there should be some methodology for this analysis, but I don't know. Maybe I can do a eigen decomposition to find which all vectors are unique? But would that say anything to if these vectors are helping me find the solution closer to the correct solution? I would be grateful if someone can give me some intuition into this problem.

Thanks,
Benzun

 

[haruspex]

Expressing the problem as AX=Y, A and Y given, what are these vectors? Are they the rows of A?
If so, it will not generally happen that a subset of these vectors is linearly dependent.
The row which produces the least deviation from the Y value could be considered the most reliable.
Or maybe you want the row which, in some sense, is 'most independent' of the other rows. Something like, that vector which subtends the greatest angle to the subspace generated by the rest?

 

[benzun_1999]

Most independent vector can be an outliers. yes the vectors I am talking about are the rows of A matrix. I also know vector X and Y. I know the correct solution to the problem. I could have found X with a minimal definition of A since I have A over-defined, I want to find out how much contribution each vector had to my final solution. This analysis will help me identify the important vectors.

 

[haruspex]

Most independent vector can be an outliers. yes the vectors I am talking about are the rows of A matrix. I also know vector X and Y. I know the correct solution to the problem. I could have found X with a minimal definition of A since I have A over-defined, I want to find out how much contribution each vector had to my final solution. This analysis will help me identify the important vectors.

Ok, but my point is that this is not a clearly defined function. What does it mean to say that one vector contributes more to the answer than another? As far as the least squares formula is concerned, they are all treated as equally important. What will you use the answer for? That might help clarify the concept.
Note that if you were to double a row and double the corresponding Y element then this would double the influence of the row on the answer. Is that consistent with your intuitive concept here?
Here's a possibility: consider small variations in one vector (counting the Y element as part of the vector) and observe the variation in the answer. You'd get a Jacobian in general, so you'd then need a way to collapse that into a scalar.