Undergrad Why Is the First Derivative Zero in Least Squares Optimization?

  • Thread starter Thread starter Amany Gouda
  • Start date Start date
  • Tags Tags
    Method Square
Click For Summary
In least squares optimization, the first derivative of the error summation is zero at the minimum point, indicating that the slope of the error function is flat at this extremum. This is because the method aims to minimize the sum of squared errors, and at a minimum, the derivative must equal zero. While participants agree on this principle, there is a request for a formal proof to support the claim. One participant acknowledges having encountered a proof but struggles to understand its logic. The discussion highlights the connection between derivatives and optimization in least squares theory.
Amany Gouda
Messages
29
Reaction score
0
Hello Sir,

I would studying the theory of least square and I find that the derivative of the error summation between the predicated line points and the true data is equal zero. Why the first derivative is equal zero?
 
Mathematics news on Phys.org
Amany Gouda said:
Hello Sir,

I would studying the theory of least square and I find that the derivative of the error summation between the predicated line points and the true data is equal zero. Why the first derivative is equal zero?
I'm partly guessing exactly what you did, but I suggest it is because the method finds the line that minimises the sum square of errors, and when a smooth function is at a maximum or minimum the slope (derivative) of the function is zero.
 
You are right, I have the same opinion regarding the answer. But is there a prove to this fact?
 
Amany Gouda said:
You are right, I have the same opinion regarding the answer. But is there a prove to this fact?
A proof of which fact? That at an extremum the derivative is zero?
 
  • Like
Likes Amany Gouda
Unfortunately, there is a prove but I didn't reach to it.
 
Amany Gouda said:
Unfortunately, there is a prove but I didn't reach to it.
What does this mean? Did you find a proof but were unable to follow the logic of it?
 

Thread 'Area of Overlapping Squares'

Here is a little puzzle from the book 100 Geometric Games by Pierre Berloquin. The side of a small square is one meter long and the side of a larger square one and a half meters long. One vertex of the large square is at the center of the small square. The side of the large square cuts two sides of the small square into one- third parts and two-thirds parts. What is the area where the squares overlap?
View full post »

Similar threads

Undergrad Gradient descent algorithm and optimizers
  • · Replies 5 ·
Replies
5
Views
2K
Undergrad Fitting Data to Grafted Distribution
  • · Replies 6 ·
Replies
6
Views
1K
Undergrad What is Nonlinear Least Squares in Regression Analysis?
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad Discussion about least squares method
  • · Replies 2 ·
Replies
2
Views
2K
Undergrad Linear least-squares method and row multiplication of matrix
  • · Replies 2 ·
Replies
2
Views
2K
Undergrad Feynman's derivation of average square distance variation in Brownian motion
  • · Replies 3 ·
Replies
3
Views
692
MHB Least squares method : approximation of a cubic polynomial
  • · Replies 3 ·
Replies
3
Views
3K
Undergrad Nonlinear Least Squares or OLS for Nonlinear Models?
  • · Replies 7 ·
Replies
7
Views
2K
Python Why backpropagation dominates neural networks instead of interpolation
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad Can Alternative Least Squares Methods Be Used in Linear Regression?
  • · Replies 6 ·
Replies
6
Views
3K