Over constrained least squares analysis

In summary, the conversation discusses a problem of determining the contribution of each vector in an over constrained least squares problem to the correct solution. The question arises of how to measure the importance of each vector in finding the solution, and different methods are suggested, such as finding the row with the least deviation from the Y value or the most independent vector. However, there is no clear definition of what it means for a vector to contribute more to the solution than another. It is suggested to consider small variations in each vector and observe the resulting variation in the solution.
  • #1
benzun_1999
260
0
Hi,

I have a over constrained least squares problem. I also have the correct solution to the problem. Now I need to determine which of the vectors are contributing how much information that is close to the correct solution. I am sure there should be some methodology for this analysis, but I don't know. Maybe I can do a eigen decomposition to find which all vectors are unique? But would that say anything to if these vectors are helping me find the solution closer to the correct solution? I would be grateful if someone can give me some intuition into this problem.

Thanks,
Benzun
 
Mathematics news on Phys.org
  • #2
Expressing the problem as AX=Y, A and Y given, what are these vectors? Are they the rows of A?
If so, it will not generally happen that a subset of these vectors is linearly dependent.
The row which produces the least deviation from the Y value could be considered the most reliable.
Or maybe you want the row which, in some sense, is 'most independent' of the other rows. Something like, that vector which subtends the greatest angle to the subspace generated by the rest?
 
  • #3
Most independent vector can be an outliers. yes the vectors I am talking about are the rows of A matrix. I also know vector X and Y. I know the correct solution to the problem. I could have found X with a minimal definition of A since I have A over-defined, I want to find out how much contribution each vector had to my final solution. This analysis will help me identify the important vectors.
 
  • #4
benzun_1999 said:
Most independent vector can be an outliers. yes the vectors I am talking about are the rows of A matrix. I also know vector X and Y. I know the correct solution to the problem. I could have found X with a minimal definition of A since I have A over-defined, I want to find out how much contribution each vector had to my final solution. This analysis will help me identify the important vectors.
Ok, but my point is that this is not a clearly defined function. What does it mean to say that one vector contributes more to the answer than another? As far as the least squares formula is concerned, they are all treated as equally important. What will you use the answer for? That might help clarify the concept.
Note that if you were to double a row and double the corresponding Y element then this would double the influence of the row on the answer. Is that consistent with your intuitive concept here?
Here's a possibility: consider small variations in one vector (counting the Y element as part of the vector) and observe the variation in the answer. You'd get a Jacobian in general, so you'd then need a way to collapse that into a scalar.
 

1. What is over constrained least squares analysis?

Over constrained least squares analysis is a statistical method used to estimate the parameters of a linear regression model. It involves finding the best fit line that minimizes the sum of the squared differences between the observed data points and the predicted values. It is considered "over constrained" when there are more independent variables than data points, making it impossible to find a unique solution.

2. How does over constrained least squares analysis differ from regular least squares analysis?

Regular least squares analysis assumes that there are at least as many data points as independent variables, allowing for a unique solution to be found. Over constrained least squares analysis, on the other hand, deals with situations where there are more independent variables than data points, making it necessary to use additional constraints or assumptions to find a solution.

3. What are the potential challenges of using over constrained least squares analysis?

One challenge of over constrained least squares analysis is that it can be difficult to determine the appropriate constraints or assumptions to use in order to find a solution. Additionally, the results of the analysis may be sensitive to these constraints and may not accurately reflect the underlying data. It is important to carefully consider the data and the assumptions being used in order to avoid misleading results.

4. In what situations would over constrained least squares analysis be useful?

Over constrained least squares analysis can be useful in situations where there are more independent variables than data points, such as in the case of highly correlated variables or when dealing with missing data. It can also be used to test the significance of individual variables in a model and to identify potential outliers.

5. Are there any alternative methods to over constrained least squares analysis?

Yes, there are alternative methods to over constrained least squares analysis, such as ridge regression and principal component regression. These methods are designed to handle situations where there are more independent variables than data points and may be more appropriate depending on the specific data and research question being addressed.

Similar threads

I Exploring Nonlinear Least Squares for Regression Analysis
    • Set Theory, Logic, Probability, Statistics
Replies
4
Views
1K
I Modeling Asteroid Rotation Using Quaternions: Seeking Guidance on Init
    • General Math
Replies
3
Views
724
I Linear least-squares method and row multiplication of matrix
    • General Math
Replies
2
Views
1K
A Convergence issue in this Least Squares calculation
    • Set Theory, Logic, Probability, Statistics
Replies
2
Views
975
MHB Least squares method : approximation of a cubic polynomial
    • General Math
Replies
3
Views
2K
MHB Ratio of areas of squares - Challenging problem
    • General Math
Replies
1
Views
1K
Using Least Squares to find Orthogonal Projection
    • Calculus and Beyond Homework Help
Replies
1
Views
1K
I Excellent 3D Graphics thing at math3d.org
    • General Math
Replies
7
Views
1K
Modified least squares fitting?
    • General Math
Replies
2
Views
2K
MHB Not a "help need" but a question about "a self made square root formula"
    • General Math
Replies
2
Views
2K

Hot Threads

Recent Insights

Back
Top