MHB Least squares method : approximation of a cubic polynomial

mathmari
Gold Member
MHB
Messages
4,984
Reaction score
7
Hey! :o

I want to determine an approximation of a cubic polynomial that has at the points $$x_0=-2, \ x_1=-1, \ x_2=0 , \ x_3=3, \ x_4=3.5$$ the values $$y_0=-33, \ y_1=-20, \ y_2=-20.1, \ y_3=-4.3 , \ y_4=32.5$$ using the least squares method.

So we are looking for a cubic polynomial $p(x)$ such that $$\sum_{i=0}^4\left (p(x_i)-y_i\right )^2$$ is minimal, right? (Wondering) Let $p(x)=a_3x^3+a_2x^2+a_1x+a_0$. Then we get the following sum:
\begin{align*}\sum_{i=0}^4\left (p(x_i)-y_i\right )^2=&\left (-8a_3+4a_2-2a_1+a_0+33\right )^2+\left (-a_3+a_2-a_1+a_0+20\right )^2+\left (a_0+20.1\right )^2 \\ & +\left (27a_3+9a_2+3a_1+a_0+4.3\right )^2 +\left (42.875a_3+12.25a_2+3.5a_1+a_0-32.5\right )^2\end{align*}

Now we calculate the partial derivatives in respect to each $a_i$ and then we set these equal to $0$. In that we get a system of equations.

Let this sum be $S$.

We get the following partial derivatives:

\begin{align*}\frac{\partial{S}}{\partial{a_0}}=&2\left (-8a_3+4a_2-2a_1+a_0+33\right )+2\left (-a_3+a_2-a_1+a_0+20\right )\\ & +2\left (a_0+20.1\right ) +2\left (27a_3+9a_2+3a_1+a_0+4.3\right ) +2\left (42.875a_3+12.25a_2+3.5a_1+a_0-32.5\right )\\ = &89.8 + 10 a_0 + 7 a_1+ 52.5 a_2 + 121.75 a_3\end{align*}

\begin{align*}\frac{\partial{S}}{\partial{a_1}}=&2\left (-8a_3+4a_2-2a_1+a_0+33\right )\cdot 2+2\left (-a_3+a_2-a_1+a_0+20\right )\cdot (-1) \\ & +2\left (27a_3+9a_2+3a_1+a_0+4.3\right )\cdot 3 +2\left (42.875a_3+12.25a_2+3.5a_1+a_0-32.5\right )\cdot 3.5 \\ = & -109.7 + 15 a_0 + 36.5 a_1 + 153.75 a_2 + 432.125 a_3\end{align*}

\begin{align*}\frac{\partial{S}}{\partial{a_2}}=&2\left (-8a_3+4a_2-2a_1+a_0+33\right )\cdot 4+2\left (-a_3+a_2-a_1+a_0+20\right ) \\ & +2\left (27a_3+9a_2+3a_1+a_0+4.3\right )\cdot 9 +2\left (42.875a_3+12.25a_2+3.5a_1+a_0-32.5\right )\cdot 12.25\\ = & -414.85 + 52.5 a_0 + 121.75 a_1 + 496.125 a_2 + 1470.44 a_3 \end{align*}

\begin{align*}\frac{\partial{S}}{\partial{a_3}}=&2\left (-8a_3+4a_2-2a_1+a_0+33\right )\cdot (-8)+2\left (-a_3+a_2-a_1+a_0+20\right )\cdot (-1) \\ & +2\left (27a_3+9a_2+3a_1+a_0+4.3\right )\cdot 27 +2\left (42.875a_3+12.25a_2+3.5a_1+a_0-32.5\right )\cdot 42.875\\ = & -3122.68 + 121.75 a_0 + 496.125 a_1 + 1470.44 a_2 + 5264.53 a_3\end{align*}

So, we get the system:
\begin{align*}& \ \ \ \ 89.8 + 10 a_0 + 7 a_1+ 52.5 a_2 + 121.75 a_3=0 \\ &-109.7 + 15 a_0 + 36.5 a_1 + 153.75 a_2 + 432.125 a_3=0 \\ &-414.85 + 52.5 a_0 + 121.75 a_1 + 496.125 a_2 + 1470.44 a_3=0 \\ &-3122.68 + 121.75 a_0 + 496.125 a_1 + 1470.44 a_2 + 5264.53 a_3=0\end{align*}

Then we get the solution: \begin{equation*}a_0\approx -21.6611, \ a_1\approx -7.08875, \ a_2\approx -2.0.6057, \ a_3\approx 2.33768\end{equation*}

The cubic polynomial that we are looking for is \begin{equation*}p(x)=2.33768x^3-2.0.6057x^2-7.08875x-21.6611 \end{equation*}
Doing the above in an other way I got an other result, but I don't know why and which is the correct one. (Wondering)

Using the least square method and the normal equation we get that the solution that we are looking for is the vector $\vec{a}$ such that $A \vec{a}=b \Rightarrow A^TA\vec{a}=A^Tb$.

So we get the following:
\begin{equation*}
\begin{bmatrix}
5 & 3.5 & 26.25 & 60.875 \\
3.5 & 26.25 & 60.875 & 248.0625 \\
26.25 & 60.875 & 248.0625 & 735.21875 \\
60.875 & 248.0625 & 735.21875 & 2632.265625
\end{bmatrix}
\begin{bmatrix}
a_0 \\
a_1 \\
a_2 \\
a_3 \\
\end{bmatrix}=
\begin{bmatrix}
-44.9 \\
186 \\
207.425 \\
1561.3375 \\
\end{bmatrix}
\end{equation*}

Solving this one we get the polynomial $$p(x)=-23.0087-12.2424x-2.7837x^2+3.0565x^3$$ This is not the same polynomial as the one that we get at the first method. And for the second polynomial the condition $p(0)=-20.1$ is not true. (Wondering)
 
Mathematics news on Phys.org
Hey mathmari!

I checked the solution $a=(A^T A)^{-1} A^T y$, and found the same solution that you did.
If I graph it, I get:
View attachment 8788
https://octave-online.net/
xx=[-2 -1 0 3 3.5]';
yy=[-33 -20 -20.1 -4.3 32.5]';
A=[ones(5,1) xx xx.^2 xx.^3];
ATA = A'*A
ATy = A'*yy
a = ATA\ATy
plot(xx,yy)
hold on
plot(xx, A*a)
That looks pretty close doesn't it, even though $p(0)$ is slightly off? (Wondering)

And yes, your solution with the derivatives set to zero should have the same result. (Worried)EDIT: Now I've also plotted it with the $a$ you found with the derivatives:
View attachment 8789
a_diff = [-21.6611, -7.08875, -2.06057, 2.33768]'
plot(xx, A*a_diff)

Hmm... that's not the same is it?
The first one does look better though.
Perhaps there is a mistake? (Wondering)EDIT 2:
If I calculate the residu $\|Aa-y\|$ in both cases, I get:
Code:
>> norm(A*a-yy)
ans =  5.2292
>> norm(A*a_diff-yy)
ans =  7.5340

In other words, the solution with the derivatives did not find the lowest possible residu.
It confirms that there must be a calculation mistake. (Worried)
 

Attachments

  • cubic_lsq.png
    cubic_lsq.png
    2.9 KB · Views: 107
  • cubic_lsq_diff.png
    cubic_lsq_diff.png
    3.1 KB · Views: 125
I checked my calculations again and I saw that I had a wrong sign. Now I get \begin{equation*}p(x)=3.05677x^3-2.78419x^2-12.2439x-23.0084 \end{equation*} which is approximately the same as the second one. Do we have have to get the exactly same polynomial or is it correct that we get approximately the same one? But what about the value of the polynomial at $x=0$ ? Shouldn't it be as the given value? (Wondering)
 
mathmari said:
I checked my calculations again and I saw that I had a wrong sign. Now I get \begin{equation*}p(x)=3.05677x^3-2.78419x^2-12.2439x-23.0084 \end{equation*} which is approximately the same as the second one. Do we have have to get the exactly same polynomial or is it correct that we get approximately the same one? But what about the value of the polynomial at $x=0$ ? Shouldn't it be as the given value?

Good!
The first 4 significant digits of each coefficient are the same.
I think it means that intermediate results of the numbers have been rounded to perhaps 5 or 6 significant digits.
The results should be exactly the same, but of cause they are sensitive to rounding.
Anyway, they are close enough that I believe that the calculations are correct now. (Thinking)And no, the value of polynomial is not expected to be the same as the given value at $x=0$.
It is an approximation after all.
We can only fit a cubic polynomial perfectly through 4 points.
Since 5 points were given, it is normal that every point is slightly off from the given value.
The squared sum of the differences is merely as low as possible. (Nerd)
 
Insights auto threads is broken atm, so I'm manually creating these for new Insight articles. In Dirac’s Principles of Quantum Mechanics published in 1930 he introduced a “convenient notation” he referred to as a “delta function” which he treated as a continuum analog to the discrete Kronecker delta. The Kronecker delta is simply the indexed components of the identity operator in matrix algebra Source: https://www.physicsforums.com/insights/what-exactly-is-diracs-delta-function/ by...
Fermat's Last Theorem has long been one of the most famous mathematical problems, and is now one of the most famous theorems. It simply states that the equation $$ a^n+b^n=c^n $$ has no solutions with positive integers if ##n>2.## It was named after Pierre de Fermat (1607-1665). The problem itself stems from the book Arithmetica by Diophantus of Alexandria. It gained popularity because Fermat noted in his copy "Cubum autem in duos cubos, aut quadratoquadratum in duos quadratoquadratos, et...
I'm interested to know whether the equation $$1 = 2 - \frac{1}{2 - \frac{1}{2 - \cdots}}$$ is true or not. It can be shown easily that if the continued fraction converges, it cannot converge to anything else than 1. It seems that if the continued fraction converges, the convergence is very slow. The apparent slowness of the convergence makes it difficult to estimate the presence of true convergence numerically. At the moment I don't know whether this converges or not.
Back
Top