I Why Is the First Derivative Zero in Least Squares Optimization?

  • I
  • Thread starter Thread starter Amany Gouda
  • Start date Start date
  • Tags Tags
    Method Square
AI Thread Summary
In least squares optimization, the first derivative of the error summation is zero at the minimum point, indicating that the slope of the error function is flat at this extremum. This is because the method aims to minimize the sum of squared errors, and at a minimum, the derivative must equal zero. While participants agree on this principle, there is a request for a formal proof to support the claim. One participant acknowledges having encountered a proof but struggles to understand its logic. The discussion highlights the connection between derivatives and optimization in least squares theory.
Amany Gouda
Messages
29
Reaction score
0
Hello Sir,

I would studying the theory of least square and I find that the derivative of the error summation between the predicated line points and the true data is equal zero. Why the first derivative is equal zero?
 
Mathematics news on Phys.org
Amany Gouda said:
Hello Sir,

I would studying the theory of least square and I find that the derivative of the error summation between the predicated line points and the true data is equal zero. Why the first derivative is equal zero?
I'm partly guessing exactly what you did, but I suggest it is because the method finds the line that minimises the sum square of errors, and when a smooth function is at a maximum or minimum the slope (derivative) of the function is zero.
 
You are right, I have the same opinion regarding the answer. But is there a prove to this fact?
 
Amany Gouda said:
You are right, I have the same opinion regarding the answer. But is there a prove to this fact?
A proof of which fact? That at an extremum the derivative is zero?
 
  • Like
Likes Amany Gouda
Unfortunately, there is a prove but I didn't reach to it.
 
Amany Gouda said:
Unfortunately, there is a prove but I didn't reach to it.
What does this mean? Did you find a proof but were unable to follow the logic of it?
 

Thread 'What Exactly is Dirac’s Delta Function? - Insight'

Insights auto threads is broken atm, so I'm manually creating these for new Insight articles. In Dirac’s Principles of Quantum Mechanics published in 1930 he introduced a “convenient notation” he referred to as a “delta function” which he treated as a continuum analog to the discrete Kronecker delta. The Kronecker delta is simply the indexed components of the identity operator in matrix algebra Source: https://www.physicsforums.com/insights/what-exactly-is-diracs-delta-function/ by...
View full post »

Thread 'Fermat's Last Theorem'

Fermat's Last Theorem has long been one of the most famous mathematical problems, and is now one of the most famous theorems. It simply states that the equation $$ a^n+b^n=c^n $$ has no solutions with positive integers if ##n>2.## It was named after Pierre de Fermat (1607-1665). The problem itself stems from the book Arithmetica by Diophantus of Alexandria. It gained popularity because Fermat noted in his copy "Cubum autem in duos cubos, aut quadratoquadratum in duos quadratoquadratos, et...
View full post »
Thread 'Imaginary Pythagorus'

Thread 'Imaginary Pythagorus'

I posted this in the Lame Math thread, but it's got me thinking. Is there any validity to this? Or is it really just a mathematical trick? Naively, I see that i2 + plus 12 does equal zero2. But does this have a meaning? I know one can treat the imaginary number line as just another axis like the reals, but does that mean this does represent a triangle in the complex plane with a hypotenuse of length zero? Ibix offered a rendering of the diagram using what I assume is matrix* notation...
View full post »

Similar threads

I Gradient descent algorithm and optimizers
Replies
5
Views
2K
I Can I Use a Natural Log Function for Least Square Fitting?
Replies
4
Views
1K
I Fitting Data to Grafted Distribution
Replies
6
Views
1K
I Discussion about least squares method
Replies
2
Views
2K
I What is Nonlinear Least Squares in Regression Analysis?
Replies
4
Views
2K
I Linear least-squares method and row multiplication of matrix
Replies
2
Views
2K
I Feynman's derivation of average square distance variation in Brownian motion
Replies
3
Views
201
MHB Least squares method : approximation of a cubic polynomial
Replies
3
Views
3K
I Nonlinear Least Squares or OLS for Nonlinear Models?
Replies
7
Views
2K
I Why can't I reach every cell in a 3x3 square?
Replies
8
Views
3K

Hot Threads

Recent Insights

Back
Top