Linear regression with least squares for quadratic function

1. Jul 19, 2014

Maylis

1. The problem statement, all variables and given/known data
We want to determine the coefficients of a polynomial of the form:

$p(x)=c_{1}x^2 +c_{2}x+c_{3}$

The polynomial $p(x)$ must satisfy the constraint $p(1)=1$.

We would also like $p(x)$ to satisfy the following 4 constraints:

$p(−1)=5$

$p(0)=−1$

$p(2)=6$

$p(3)=12$

However, this is not possible, since the system of equations is over-determined. Instead, we wish to minimize the error, $E$, as shown below, using least squares.

$E = (p(-1)-5)^2 +(p(0)+1)^2 +(p(2)-6)^2 +(p(3)-12)^2$

part 1
Solve for $c_{3}$ in terms of $c_{1}$ and $c_{2}$ such that the constraint, $p(1)=1$, holds. Which of the following expressions is $c_{3}$ equal to?

part 2
If you substitute the expression for $c_{3}$ into the polynomial, $p(x)$, what is the new polynomial?

part 3
Put the new polynomial and the 4 constraints you would like to satisfy into the $Az=b$ form where $x = \left[ \begin{array}{c} c_1 \\ c_2 \end{array} \right]$ . Using Matlab find the least squares solution of this system of equations. What is the value for $x$?

2. Relevant equations

3. The attempt at a solution
I am having a lot of difficulty parsing this problem. I don't really understand what to do. The equation must satisfy $p(1) = 1$ and we attempt to satisfy the following

$p(-1) = 5$
$p(0) = -1$
$p(2) = 6$
$p(3) = 12$

Since one of the equations must be satisfied, and the other ones are ''try our best to satisfy'', I don't know how to set up a least square to solve this. Do I not include the one that must be satisfied, or include all? How do I make it so that it guarantees that specific one to be satisfied, but the other ones don't have to be. Very confused about this. Even how to set up $c_{3}$ in terms of $c_{1}$ and $c_{2}$.
$E =min \underset x \in \mathbb R] \hspace{0.05 in} \|\left[ \begin{array}{x} 1 & -1 & 1 \\ 0 & 0 & 1 \\ 4 & 2 & 1 \\ 9 & 3 & 1 \end{array} \right] \left[\begin{array}{c} c_{1} \\ c_{2} \\ c_{3} \end{array} \right] - \left[\begin{array}{y} 5 \\ -1 \\ 6 \\ 12 \end{array} \right]\|$

how do I get the x is an element of R to be under the min?? And taller || to show that it is the norm

Attached Files:

• Quiz_ Linear regression as least squares_ quadratic functions of one variable.pdf
File size:
87.1 KB
Views:
103
Last edited: Jul 19, 2014
2. Jul 19, 2014

SammyS

Staff Emeritus
I take it there's a typo in the above line & it should read:
$E = (p(-1)-5)^2 +(p(0)+1)^2 +(p(2)-6)^2 +(p(3)-12)^2 \ .$​
Have you answered parts 1 & 2?

They seem pretty straight forward.

What is p(1) ? Substitute 1 in for x in $c_1x^2+c_2x+c_3\ .$

3. Jul 19, 2014

Maylis

Okay, when I saw all that business about one having to be satisfied and the other not, it got me way confused. I now did part 1 and 2. It temporarily disoriented me until I saw your post about it being trivial.

I got my function into the form with the substitution

$p(x) = c_{1}(x^2-1) + c_{2}(x-1) + 1$

Here is part 3. The way I thought about it was that there should be an array in the form $Ax = b$

Then I set it up, keeping in mind that I moved the 1 to the right hand side of the equation,
$\left [\begin{array}{x} x_{1}^2 -1 & x_{1}-1 \\ x_{2}^2 -1 & x_{2}-1 \\ x_{3}^2 -1 & x_{3}-1 \\ x_{4}^2 -1 & x_{4}-1 \end{array} \right] \left[\begin{array}{c} c_{1} \\ c_{2} \end{array} \right] = b - 1$

Code (Text):
A = [0 -2; -1 -1; 3 1; 8 2]
b = [5; -1; 6; 12];
x = A\(b-1);

I got it now, the wording totally disoriented me though so I didn't know what to do.

Last edited: Jul 19, 2014
4. Jul 19, 2014

Maylis

What I don't get is how is this linear regression? All I did for this problem was what I did in the last module, least squares. With matlab, does it automatically calculate the least squares when I do A\b? Also, why was the equation for E never used??

Last edited: Jul 19, 2014
5. Jul 19, 2014

AlephZero

Yes, that's exactly what the \ operator does.

If A is a square nonsingular matrix, the "least squares" solution is the same as an "ordinary" equation solution, so you can also use the \ operator to solve problems that don't have anything to do with least squares.

If you want to minimize $E = ||Ax - b||$, one way to do it is explicitly form the equations $(A^TA)x = (A^Tb)$ and solve them. If you solved least squares problems by hand, that's probably how you did it. Conceptually that is what the Matlab \ operator does, but it actually uses a different numerical method which is faster and avoids problems with rounding errors, etc.

6. Jul 19, 2014

SammyS

Staff Emeritus
It's called linear because the expression you're fitting is linear in the parameters that you determine, in this case, the c1 and c2 .