Over constrained least squares analysis

benzun_1999

Hi,

I have a over constrained least squares problem. I also have the correct solution to the problem. Now I need to determine which of the vectors are contributing how much information that is close to the correct solution. I am sure there should be some methodology for this analysis, but I dont know. Maybe I can do a eigen decomposition to find which all vectors are unique? But would that say anything to if these vectors are helping me find the solution closer to the correct solution? I would be grateful if someone can give me some intuition into this problem.

Thanks,
Benzun

haruspex

Homework Helper
Gold Member
2018 Award
Expressing the problem as AX=Y, A and Y given, what are these vectors? Are they the rows of A?
If so, it will not generally happen that a subset of these vectors is linearly dependent.
The row which produces the least deviation from the Y value could be considered the most reliable.
Or maybe you want the row which, in some sense, is 'most independent' of the other rows. Something like, that vector which subtends the greatest angle to the subspace generated by the rest?

benzun_1999

Most independent vector can be an outliers. yes the vectors I am talking about are the rows of A matrix. I also know vector X and Y. I know the correct solution to the problem. I could have found X with a minimal definition of A since I have A over-defined, I want to find out how much contribution each vector had to my final solution. This analysis will help me identify the important vectors.

haruspex

Homework Helper
Gold Member
2018 Award
Most independent vector can be an outliers. yes the vectors I am talking about are the rows of A matrix. I also know vector X and Y. I know the correct solution to the problem. I could have found X with a minimal definition of A since I have A over-defined, I want to find out how much contribution each vector had to my final solution. This analysis will help me identify the important vectors.
Ok, but my point is that this is not a clearly defined function. What does it mean to say that one vector contributes more to the answer than another? As far as the least squares formula is concerned, they are all treated as equally important. What will you use the answer for? That might help clarify the concept.
Note that if you were to double a row and double the corresponding Y element then this would double the influence of the row on the answer. Is that consistent with your intuitive concept here?
Here's a possibility: consider small variations in one vector (counting the Y element as part of the vector) and observe the variation in the answer. You'd get a Jacobian in general, so you'd then need a way to collapse that into a scalar.

Physics Forums Values

We Value Quality
• Topics based on mainstream science
• Proper English grammar and spelling
We Value Civility
• Positive and compassionate attitudes
• Patience while debating
We Value Productivity
• Disciplined to remain on-topic
• Recognition of own weaknesses
• Solo and co-op problem solving