I Why Is the First Derivative Zero in Least Squares Optimization?

  • I
  • Thread starter Thread starter Amany Gouda
  • Start date Start date
  • Tags Tags
    Method Square
AI Thread Summary
In least squares optimization, the first derivative of the error summation is zero at the minimum point, indicating that the slope of the error function is flat at this extremum. This is because the method aims to minimize the sum of squared errors, and at a minimum, the derivative must equal zero. While participants agree on this principle, there is a request for a formal proof to support the claim. One participant acknowledges having encountered a proof but struggles to understand its logic. The discussion highlights the connection between derivatives and optimization in least squares theory.
Amany Gouda
Messages
29
Reaction score
0
Hello Sir,

I would studying the theory of least square and I find that the derivative of the error summation between the predicated line points and the true data is equal zero. Why the first derivative is equal zero?
 
Mathematics news on Phys.org
Amany Gouda said:
Hello Sir,

I would studying the theory of least square and I find that the derivative of the error summation between the predicated line points and the true data is equal zero. Why the first derivative is equal zero?
I'm partly guessing exactly what you did, but I suggest it is because the method finds the line that minimises the sum square of errors, and when a smooth function is at a maximum or minimum the slope (derivative) of the function is zero.
 
You are right, I have the same opinion regarding the answer. But is there a prove to this fact?
 
Amany Gouda said:
You are right, I have the same opinion regarding the answer. But is there a prove to this fact?
A proof of which fact? That at an extremum the derivative is zero?
 
  • Like
Likes Amany Gouda
Unfortunately, there is a prove but I didn't reach to it.
 
Amany Gouda said:
Unfortunately, there is a prove but I didn't reach to it.
What does this mean? Did you find a proof but were unable to follow the logic of it?
 

Thread 'Non-orthogonal bases'

Suppose ,instead of the usual x,y coordinate system with an I basis vector along the x -axis and a corresponding j basis vector along the y-axis we instead have a different pair of basis vectors ,call them e and f along their respective axes. I have seen that this is an important subject in maths My question is what physical applications does such a model apply to? I am asking here because I have devoted quite a lot of time in the past to understanding convectors and the dual...
View full post »

Thread 'What Exactly is Dirac’s Delta Function? - Insight'

Insights auto threads is broken atm, so I'm manually creating these for new Insight articles. In Dirac’s Principles of Quantum Mechanics published in 1930 he introduced a “convenient notation” he referred to as a “delta function” which he treated as a continuum analog to the discrete Kronecker delta. The Kronecker delta is simply the indexed components of the identity operator in matrix algebra Source: https://www.physicsforums.com/insights/what-exactly-is-diracs-delta-function/ by...
View full post »

Similar threads

I Gradient descent algorithm and optimizers
Replies
5
Views
2K
I Can I Use a Natural Log Function for Least Square Fitting?
Replies
4
Views
1K
I Fitting Data to Grafted Distribution
Replies
6
Views
1K
I Discussion about least squares method
Replies
2
Views
2K
I What is Nonlinear Least Squares in Regression Analysis?
Replies
4
Views
2K
I Linear least-squares method and row multiplication of matrix
Replies
2
Views
2K
I Feynman's derivation of average square distance variation in Brownian motion
Replies
3
Views
256
MHB Least squares method : approximation of a cubic polynomial
Replies
3
Views
3K
I Nonlinear Least Squares or OLS for Nonlinear Models?
Replies
7
Views
2K
I Why can't I reach every cell in a 3x3 square?
Replies
8
Views
3K

Hot Threads

Recent Insights

Back
Top