Least Squares Derivation—Simple Algebraic Simplification

Click For Summary
SUMMARY

The discussion focuses on the derivation of the least squares estimator, specifically addressing the transition from the equation on page 7 to equation (11) on page 8 of the referenced document. The key point of contention is the denominator's structure, which is identified as a difference of two terms rather than a sum. The error is attributed to a misinterpretation during the multiplication of terms, where the sign of the first term in the denominator was incorrectly altered. This clarification is essential for accurately understanding the least squares method.

PREREQUISITES
  • Understanding of least squares estimation
  • Familiarity with algebraic manipulation of equations
  • Basic knowledge of statistical concepts
  • Ability to interpret mathematical notation
NEXT STEPS
  • Review the derivation of least squares estimators in "Introduction to Linear Regression Analysis" by Douglas C. Montgomery
  • Study the properties of estimators in statistics, focusing on unbiasedness and efficiency
  • Learn about matrix representations of least squares problems using R or Python
  • Explore common pitfalls in algebraic simplifications in statistical derivations
USEFUL FOR

Statisticians, data analysts, and students studying regression analysis who seek to deepen their understanding of least squares derivation and its mathematical foundations.

END
Messages
36
Reaction score
4
Hey, PF

I'm reading the following derivation of least squares, and I'm trying to figure out how the author went from the last step at the bottom of pg. 7 to the final equation (11) at the top of pg. 8.

[http://isites.harvard.edu/fs/docs/icb.topic515975.files/OLSDerivation.pdf]

More specifically, why is the denominator a difference of two terms? Aren't the terms in the denominator summed in the prior step?

I would expect the answer to be

$$
b_1=\dfrac{\displaystyle \sum_{\textrm{i=1}}^{n}y_ix_{i}-\left(\frac{1}{n}\right)\left(\sum_{\textrm{i=1}}^{n}y_i\sum_{\textrm{i=1}}^{n}x_{i}\right)}{\displaystyle\sum_{\textrm{i=1}}^{n}x_{i}^2+\left(\frac{1}{n}\right)\left(\sum_{\textrm{i=1}}^{n}x_{i}\right)^{2}}
$$

Note: I'm no statistician, but I thought you guys might be more familiar with this derivation.
 
Physics news on Phys.org
I think you are right. The step only "divides both sides of the equation by the quantity in the large brackets on the left side" as the text states.

So, the sign doesn't change and will be the sum of the two terms in the denominator as you wrote out above.
 
But the the mistake is a few steps before that where the text reads "Multiplying out the last term on the right". The writer removes the brackets but only changes the sign for the first term of the brackets. It is supposed to be a minus sign in the denominator (as in a difference of the two terms).
 
  • Like
Likes END
titasB said:
But the the mistake is a few steps before that where the text reads "Multiplying out the last term on the right". The writer removes the brackets but only changes the sign for the first term of the brackets. It is supposed to be a minus sign in the denominator (as in a difference of the two terms).

Thank you, TitasB!
 
  • Like
Likes titasB
If there are an infinite number of natural numbers, and an infinite number of fractions in between any two natural numbers, and an infinite number of fractions in between any two of those fractions, and an infinite number of fractions in between any two of those fractions, and an infinite number of fractions in between any two of those fractions, and... then that must mean that there are not only infinite infinities, but an infinite number of those infinities. and an infinite number of those...

Similar threads

Replies
1
Views
3K
Replies
1
Views
4K
Replies
6
Views
2K
  • · Replies 12 ·
Replies
12
Views
2K
  • · Replies 1 ·
Replies
1
Views
1K
  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 3 ·
Replies
3
Views
1K
  • · Replies 42 ·
2
Replies
42
Views
5K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K