Linear regression with least squares for quadratic function

[gfd43tg]

Homework Statement

We want to determine the coefficients of a polynomial of the form:

$p(x)=c_{1}x^2 +c_{2}x+c_{3}$The polynomial $p(x)$ must satisfy the constraint $p(1)=1$.

We would also like $p(x)$ to satisfy the following 4 constraints:

$p(−1)=5$

$p(0)=−1$

$p(2)=6$

$p(3)=12$However, this is not possible, since the system of equations is over-determined. Instead, we wish to minimize the error, $E$, as shown below, using least squares.

$E = (p(-1)-5)^2 +(p(0)+1)^2 +(p(2)-6)^2 +(p(3)-12)^2$

part 1
Solve for $c_{3}$ in terms of $c_{1}$ and $c_{2}$ such that the constraint, $p(1)=1$, holds. Which of the following expressions is $c_{3}$ equal to?

part 2
If you substitute the expression for $c_{3}$ into the polynomial, $p(x)$, what is the new polynomial?

part 3
Put the new polynomial and the 4 constraints you would like to satisfy into the $Az=b$ form where $x = \left[ \begin{array}{c} c_1 \\ c_2 \end{array} \right]$ . Using Matlab find the least squares solution of this system of equations. What is the value for $x$?

Homework Equations

The Attempt at a Solution

I am having a lot of difficulty parsing this problem. I don't really understand what to do. The equation must satisfy $p(1) = 1$ and we attempt to satisfy the following

$p(-1) = 5$
$p(0) = -1$
$p(2) = 6$
$p(3) = 12$

Since one of the equations must be satisfied, and the other ones are ''try our best to satisfy'', I don't know how to set up a least square to solve this. Do I not include the one that must be satisfied, or include all? How do I make it so that it guarantees that specific one to be satisfied, but the other ones don't have to be. Very confused about this. Even how to set up $c_{3}$ in terms of $c_{1}$ and $c_{2}$.
$E =min \underset x \in \mathbb R] \hspace{0.05 in} \|\left[ \begin{array}{x} 1 & -1 & 1 \\ 0 & 0 & 1 \\ 4 & 2 & 1 \\ 9 & 3 & 1 \end{array} \right] \left[\begin{array}{c} c_{1} \\ c_{2} \\ c_{3} \end{array} \right] - \left[\begin{array}{y} 5 \\ -1 \\ 6 \\ 12 \end{array} \right]\|$

how do I get the x is an element of R to be under the min?? And taller || to show that it is the norm

 

[SammyS]

Homework Statement

We want to determine the coefficients of a polynomial of the form:

$p(x)=c_{1}x^2 +c_{2}x+c_{3}$

The polynomial $p(x)$ must satisfy the constraint $p(1)=1$.

We would also like $p(x)$ to satisfy the following 4 constraints:

$p(−1)=5$

$p(0)=−1$

$p(2)=6$

$p(3)=12$


However, this is not possible, since the system of equations is over-determined. Instead, we wish to minimize the error, $E$, as shown below, using least squares.

$E = (p(-1)-5)^2 +(p(0)+1)^2 +(p(2)-6)^2 +(p(3)-12)^2] E=(p(−1)−5)2+(p(0)+1)2+(p(2)−6)2+(p(3)−12)2$

I take it there's a typo in the above line & it should read:

$E = (p(-1)-5)^2 +(p(0)+1)^2 +(p(2)-6)^2 +(p(3)-12)^2 \ .$​

part 1
Solve for $c_{3}$ in terms of $c_{1}$ and $c_{2}$ such that the constraint, $p(1)=1$, holds. Which of the following expressions is $c_{3}$ equal to?

part 2
If you substitute the expression for $c_{3}$ into the polynomial, $p(x)$, what is the new polynomial?

part 3
Put the new polynomial and the 4 constraints you would like to satisfy into the $Az=b$ form where $x = \left[ \begin{array}{c} c_1 \\ c_2 \end{array} \right]$ . Using Matlab find the least squares solution of this system of equations. What is the value for$x$?

Homework Equations

The Attempt at a Solution

I am having a lot of difficulty parsing this problem. I don't really understand what to do. The equation must satisfy $p(1) = 1$ and we attempt to satisfy the following

$p(-1) = 5$
$p(0) = -1$
$p(2) = 6$
$p(3) = 12$

Since one of the equations must be satisfied, and the other ones are ''try our best to satisfy'', I don't know how to set up a least square to solve this. Do I not include the one that must be satisfied, or include all? How do I make it so that it guarantees that specific one to be satisfied, but the other ones don't have to be. Very confused about this. Even how to set up $c_{3}$ in terms of $c_{1}$ and $c_{2}$.
$E =min \underset x \in \mathbb R] \hspace{0.05 in} \|\left[ \begin{array}{x} 1 & -1 & 1 \\ 0 & 0 & 1 \\ 4 & 2 & 1 \\ 9 & 3 & 1 \end{array} \right] \left[\begin{array}{c} c_{1} \\ c_{2} \\ c_{3} \end{array} \right] - \left[\begin{array}{y} 5 \\ -1 \\ 6 \\ 12 \end{array} \right]\|$

how do I get the x is an element of R to be under the min?? And taller || to show that it is the norm

Have you answered parts 1 & 2?

They seem pretty straight forward.

What is p(1) ? Substitute 1 in for x in $c_1x^2+c_2x+c_3\ .$

 

[gfd43tg]

Okay, when I saw all that business about one having to be satisfied and the other not, it got me way confused. I now did part 1 and 2. It temporarily disoriented me until I saw your post about it being trivial.

I got my function into the form with the substitution

$p(x) = c_{1}(x^2-1) + c_{2}(x-1) + 1$

Here is part 3. The way I thought about it was that there should be an array in the form $Ax = b$

Then I set it up, keeping in mind that I moved the 1 to the right hand side of the equation,
$\left [\begin{array}{x} x_{1}^2 -1 & x_{1}-1 \\ x_{2}^2 -1 & x_{2}-1 \\ x_{3}^2 -1 & x_{3}-1 \\ x_{4}^2 -1 & x_{4}-1 \end{array} \right] \left[\begin{array}{c} c_{1} \\ c_{2} \end{array} \right] = b - 1$

 A = [0 -2; -1 -1; 3 1; 8 2]
b = [5; -1; 6; 12];
x = A\(b-1);

I got it now, the wording totally disoriented me though so I didn't know what to do.

 

[gfd43tg]

What I don't get is how is this linear regression? All I did for this problem was what I did in the last module, least squares. With matlab, does it automatically calculate the least squares when I do A\b? Also, why was the equation for E never used??

 

[AlephZero]

With matlab, does it automatically calculate the least squares when I do A\b?

Yes, that's exactly what the \ operator does.

If A is a square nonsingular matrix, the "least squares" solution is the same as an "ordinary" equation solution, so you can also use the \ operator to solve problems that don't have anything to do with least squares.

If you want to minimize $E = ||Ax - b||$, one way to do it is explicitly form the equations $(A^TA)x = (A^Tb)$ and solve them. If you solved least squares problems by hand, that's probably how you did it. Conceptually that is what the Matlab \ operator does, but it actually uses a different numerical method which is faster and avoids problems with rounding errors, etc.

 

[SammyS]

What I don't get is how is this linear regression? All I did for this problem was what I did in the last module, least squares. With matlab, does it automatically calculate the least squares when I do A\b? Also, why was the equation for E never used??

It's called linear because the expression you're fitting is linear in the parameters that you determine, in this case, the c₁ and c₂ .