Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Over constrained least squares analysis

  1. Dec 17, 2015 #1
    Hi,

    I have a over constrained least squares problem. I also have the correct solution to the problem. Now I need to determine which of the vectors are contributing how much information that is close to the correct solution. I am sure there should be some methodology for this analysis, but I dont know. Maybe I can do a eigen decomposition to find which all vectors are unique? But would that say anything to if these vectors are helping me find the solution closer to the correct solution? I would be grateful if someone can give me some intuition into this problem.

    Thanks,
    Benzun
     
  2. jcsd
  3. Dec 17, 2015 #2

    haruspex

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member
    2016 Award

    Expressing the problem as AX=Y, A and Y given, what are these vectors? Are they the rows of A?
    If so, it will not generally happen that a subset of these vectors is linearly dependent.
    The row which produces the least deviation from the Y value could be considered the most reliable.
    Or maybe you want the row which, in some sense, is 'most independent' of the other rows. Something like, that vector which subtends the greatest angle to the subspace generated by the rest?
     
  4. Dec 18, 2015 #3
    Most independent vector can be an outliers. yes the vectors I am talking about are the rows of A matrix. I also know vector X and Y. I know the correct solution to the problem. I could have found X with a minimal definition of A since I have A over-defined, I want to find out how much contribution each vector had to my final solution. This analysis will help me identify the important vectors.
     
  5. Dec 18, 2015 #4

    haruspex

    User Avatar
    Science Advisor
    Homework Helper
    Gold Member
    2016 Award

    Ok, but my point is that this is not a clearly defined function. What does it mean to say that one vector contributes more to the answer than another? As far as the least squares formula is concerned, they are all treated as equally important. What will you use the answer for? That might help clarify the concept.
    Note that if you were to double a row and double the corresponding Y element then this would double the influence of the row on the answer. Is that consistent with your intuitive concept here?
    Here's a possibility: consider small variations in one vector (counting the Y element as part of the vector) and observe the variation in the answer. You'd get a Jacobian in general, so you'd then need a way to collapse that into a scalar.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Over constrained least squares analysis
Loading...