1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Linear regression with least squares for quadratic function

  1. Jul 19, 2014 #1

    Maylis

    User Avatar
    Gold Member

    1. The problem statement, all variables and given/known data
    We want to determine the coefficients of a polynomial of the form:

    ##p(x)=c_{1}x^2 +c_{2}x+c_{3}##


    The polynomial ##p(x)## must satisfy the constraint ##p(1)=1##.

    We would also like ##p(x)## to satisfy the following 4 constraints:

    ##p(−1)=5##

    ##p(0)=−1##

    ##p(2)=6##

    ##p(3)=12##


    However, this is not possible, since the system of equations is over-determined. Instead, we wish to minimize the error, ##E##, as shown below, using least squares.

    ##E = (p(-1)-5)^2 +(p(0)+1)^2 +(p(2)-6)^2 +(p(3)-12)^2##

    part 1
    Solve for ##c_{3}## in terms of ##c_{1}## and ##c_{2}## such that the constraint, ##p(1)=1##, holds. Which of the following expressions is ##c_{3}## equal to?

    part 2
    If you substitute the expression for ##c_{3}## into the polynomial, ##p(x)##, what is the new polynomial?

    part 3
    Put the new polynomial and the 4 constraints you would like to satisfy into the ##Az=b## form where ##x = \left[ \begin{array}{c} c_1 \\ c_2 \end{array} \right]## . Using Matlab find the least squares solution of this system of equations. What is the value for ##x##?


    2. Relevant equations



    3. The attempt at a solution
    I am having a lot of difficulty parsing this problem. I don't really understand what to do. The equation must satisfy ##p(1) = 1## and we attempt to satisfy the following

    ##p(-1) = 5##
    ##p(0) = -1##
    ##p(2) = 6##
    ##p(3) = 12##

    Since one of the equations must be satisfied, and the other ones are ''try our best to satisfy'', I don't know how to set up a least square to solve this. Do I not include the one that must be satisfied, or include all? How do I make it so that it guarantees that specific one to be satisfied, but the other ones don't have to be. Very confused about this. Even how to set up ## c_{3}## in terms of ##c_{1}## and ##c_{2}##.
    ##E =min \underset x \in \mathbb R] \hspace{0.05 in} \|\left[ \begin{array}{x} 1 & -1 & 1 \\ 0 & 0 & 1 \\ 4 & 2 & 1 \\ 9 & 3 & 1 \end{array} \right] \left[\begin{array}{c} c_{1} \\ c_{2} \\ c_{3} \end{array} \right] - \left[\begin{array}{y} 5 \\ -1 \\ 6 \\ 12 \end{array} \right]\|##

    how do I get the x is an element of R to be under the min?? And taller || to show that it is the norm
     
    Last edited: Jul 19, 2014
  2. jcsd
  3. Jul 19, 2014 #2

    SammyS

    User Avatar
    Staff Emeritus
    Science Advisor
    Homework Helper
    Gold Member

    I take it there's a typo in the above line & it should read:
    ##E = (p(-1)-5)^2 +(p(0)+1)^2 +(p(2)-6)^2 +(p(3)-12)^2 \ .##​
    Have you answered parts 1 & 2?

    They seem pretty straight forward.

    What is p(1) ? Substitute 1 in for x in ##c_1x^2+c_2x+c_3\ .##
     
  4. Jul 19, 2014 #3

    Maylis

    User Avatar
    Gold Member

    Okay, when I saw all that business about one having to be satisfied and the other not, it got me way confused. I now did part 1 and 2. It temporarily disoriented me until I saw your post about it being trivial.

    I got my function into the form with the substitution

    ##p(x) = c_{1}(x^2-1) + c_{2}(x-1) + 1##

    Here is part 3. The way I thought about it was that there should be an array in the form ##Ax = b##

    Then I set it up, keeping in mind that I moved the 1 to the right hand side of the equation,
    ##\left [\begin{array}{x} x_{1}^2 -1 & x_{1}-1 \\ x_{2}^2 -1 & x_{2}-1 \\ x_{3}^2 -1 & x_{3}-1 \\ x_{4}^2 -1 & x_{4}-1 \end{array} \right] \left[\begin{array}{c} c_{1} \\ c_{2} \end{array} \right] = b - 1##


    Code (Text):
     A = [0 -2; -1 -1; 3 1; 8 2]
    b = [5; -1; 6; 12];
    x = A\(b-1);

     
    I got it now, the wording totally disoriented me though so I didn't know what to do.
     
    Last edited: Jul 19, 2014
  5. Jul 19, 2014 #4

    Maylis

    User Avatar
    Gold Member

    What I don't get is how is this linear regression? All I did for this problem was what I did in the last module, least squares. With matlab, does it automatically calculate the least squares when I do A\b? Also, why was the equation for E never used??
     
    Last edited: Jul 19, 2014
  6. Jul 19, 2014 #5

    AlephZero

    User Avatar
    Science Advisor
    Homework Helper

    Yes, that's exactly what the \ operator does.

    If A is a square nonsingular matrix, the "least squares" solution is the same as an "ordinary" equation solution, so you can also use the \ operator to solve problems that don't have anything to do with least squares.

    If you want to minimize ##E = ||Ax - b||##, one way to do it is explicitly form the equations ##(A^TA)x = (A^Tb)## and solve them. If you solved least squares problems by hand, that's probably how you did it. Conceptually that is what the Matlab \ operator does, but it actually uses a different numerical method which is faster and avoids problems with rounding errors, etc.
     
  7. Jul 19, 2014 #6

    SammyS

    User Avatar
    Staff Emeritus
    Science Advisor
    Homework Helper
    Gold Member

    It's called linear because the expression you're fitting is linear in the parameters that you determine, in this case, the c1 and c2 .
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted



Similar Discussions: Linear regression with least squares for quadratic function
Loading...