Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Weighted Least Squares for coefficients

  1. Aug 1, 2014 #1
    Hi,

    I have an ordinary least squares setup y = Ac where A is an NxM (N>>M) matrix, c the unknown coefficients and y the measurements.

    Now WEIGHTED least squares allows to weight the MEASUREMENTS if, for example, some measurements are more important or contain a lower variance.

    However, I am looking for a solution of putting weights on the coefficient vector. Pictorially speaking, some values in my coefficient vector c are more important than others and I would like emphasize some more than others using my limited set of N measurements.

    Just adding a diagonal weighting matrix w as follows does not work:

    [tex]
    \min_c \| y - Awc \|_2
    [/tex]
     
  2. jcsd
  3. Aug 2, 2014 #2

    mfb

    User Avatar
    2016 Award

    Staff: Mentor

    How can some unknown coefficients be "more important than others"? I don't think that is a meaningful concept.
     
  4. Aug 2, 2014 #3
    But there are cases where it makes sense. Not everything is a linear system. In my case I clearly see that perturbing coefficients with the same noise gives different results, depending on which I perturb. So some are more important than others.

    It's difficult to explain but I tried to explain the setup already some time ago in a different context (https://www.physicsforums.com/showpost.php?p=4533702&postcount=7 ff.).
     
  5. Aug 2, 2014 #4

    Stephen Tashi

    User Avatar
    Science Advisor

    You need to explain why you are asking about the system of linears equations given by y = Ac. I suggest you explain the problem you are trying to solve.
     
  6. Aug 2, 2014 #5
    Ok, it is a Volterra series. So it is linear in its coefficients but a non-linear system. But I think it does not matter. Anyway, since for a practical system the coefficients decay very fast, lower-order terms dominate the total error. But if you are interested in certain non-linear behavior you want to put more emphasis on those terms.
     
  7. Aug 3, 2014 #6

    mfb

    User Avatar
    2016 Award

    Staff: Mentor

    So you are talking about correlations?
    Sure, calculate the correlation matrix for your parameters.
     
  8. Aug 3, 2014 #7
    Am I? I am not sure ... at least I do not see how.

    Can you provide me with an idea how this relates to the described setup and how to obtain the correlation matrix then?

    The one thing I know is that for my application, some coefficients are more important for me - I do not know how they are cross-correlated with themselves! Therefore I only know the structure but generally I do not know their statistics (although it could be useful to leverage this too).

    Indenpendently from that, how would I add this information to solve for the coefficients? Does it still work for Least Squares? Are you referring to MAP estimation?
     
  9. Aug 4, 2014 #8

    mfb

    User Avatar
    2016 Award

    Staff: Mentor

    It's hard to understand the problem with pieces of information scattered across multiple posts like this.

    TThe fit does not care (and does not have to care) which parameters are more interesting for you. It gives you a description which parameters work best, and their corresponding uncertainties and correlations.
     
  10. Aug 21, 2014 #9
    And how could I make it care?
     
  11. Aug 21, 2014 #10

    mfb

    User Avatar
    2016 Award

    Staff: Mentor

    The fit must not and cannot care about that. This is then your interpretation of the fit result.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Weighted Least Squares for coefficients
  1. Least Square Solution (Replies: 2)

Loading...