I Why Is the First Derivative Zero in Least Squares Optimization?

  • I
  • Thread starter Thread starter Amany Gouda
  • Start date Start date
  • Tags Tags
    Method Square
AI Thread Summary
In least squares optimization, the first derivative of the error summation is zero at the minimum point, indicating that the slope of the error function is flat at this extremum. This is because the method aims to minimize the sum of squared errors, and at a minimum, the derivative must equal zero. While participants agree on this principle, there is a request for a formal proof to support the claim. One participant acknowledges having encountered a proof but struggles to understand its logic. The discussion highlights the connection between derivatives and optimization in least squares theory.
Amany Gouda
Messages
29
Reaction score
0
Hello Sir,

I would studying the theory of least square and I find that the derivative of the error summation between the predicated line points and the true data is equal zero. Why the first derivative is equal zero?
 
Mathematics news on Phys.org
Amany Gouda said:
Hello Sir,

I would studying the theory of least square and I find that the derivative of the error summation between the predicated line points and the true data is equal zero. Why the first derivative is equal zero?
I'm partly guessing exactly what you did, but I suggest it is because the method finds the line that minimises the sum square of errors, and when a smooth function is at a maximum or minimum the slope (derivative) of the function is zero.
 
You are right, I have the same opinion regarding the answer. But is there a prove to this fact?
 
Amany Gouda said:
You are right, I have the same opinion regarding the answer. But is there a prove to this fact?
A proof of which fact? That at an extremum the derivative is zero?
 
  • Like
Likes Amany Gouda
Unfortunately, there is a prove but I didn't reach to it.
 
Amany Gouda said:
Unfortunately, there is a prove but I didn't reach to it.
What does this mean? Did you find a proof but were unable to follow the logic of it?
 

Thread 'Trigonometry problem of interest'

Seemingly by some mathematical coincidence, a hexagon of sides 2,2,7,7, 11, and 11 can be inscribed in a circle of radius 7. The other day I saw a math problem on line, which they said came from a Polish Olympiad, where you compute the length x of the 3rd side which is the same as the radius, so that the sides of length 2,x, and 11 are inscribed on the arc of a semi-circle. The law of cosines applied twice gives the answer for x of exactly 7, but the arithmetic is so complex that the...
View full post »
Thread 'Video on imaginary numbers and some queries'

Thread 'Video on imaginary numbers and some queries'

Hi, I was watching the following video. I found some points confusing. Could you please help me to understand the gaps? Thanks, in advance! Question 1: Around 4:22, the video says the following. So for those mathematicians, negative numbers didn't exist. You could subtract, that is find the difference between two positive quantities, but you couldn't have a negative answer or negative coefficients. Mathematicians were so averse to negative numbers that there was no single quadratic...
View full post »
Thread 'Unit Circle Double Angle Derivations'

Thread 'Unit Circle Double Angle Derivations'

Here I made a terrible mistake of assuming this to be an equilateral triangle and set 2sinx=1 => x=pi/6. Although this did derive the double angle formulas it also led into a terrible mess trying to find all the combinations of sides. I must have been tired and just assumed 6x=180 and 2sinx=1. By that time, I was so mindset that I nearly scolded a person for even saying 90-x. I wonder if this is a case of biased observation that seeks to dis credit me like Jesus of Nazareth since in reality...
View full post »

Similar threads

I Gradient descent algorithm and optimizers
Replies
5
Views
2K
I Can I Use a Natural Log Function for Least Square Fitting?
Replies
4
Views
1K
I Fitting Data to Grafted Distribution
Replies
6
Views
1K
I Discussion about least squares method
Replies
2
Views
2K
I What is Nonlinear Least Squares in Regression Analysis?
Replies
4
Views
2K
I Linear least-squares method and row multiplication of matrix
Replies
2
Views
2K
I Feynman's derivation of average square distance variation in Brownian motion
Replies
3
Views
294
MHB Least squares method : approximation of a cubic polynomial
Replies
3
Views
3K
I Nonlinear Least Squares or OLS for Nonlinear Models?
Replies
7
Views
2K
I Why can't I reach every cell in a 3x3 square?
Replies
8
Views
3K

Hot Threads

Recent Insights

Back
Top