Linear Regression: Pros and cons of Normal vs. simplified methods?

Click For Summary
SUMMARY

This discussion focuses on the comparison between two methods for calculating linear regression equations: the Normal method and the simplified method. The Normal method, represented by the equation y = a + bx, provides precise coefficients a and b, while the simplified method, y = Bx + (ȳ - Bx), yields slightly different results primarily due to roundoff errors. The consensus is that while both methods are mathematically identical, the Normal method is preferred for its accuracy, especially when model assumptions are satisfied, as it offers the best linear unbiased estimates.

PREREQUISITES
  • Understanding of linear regression concepts
  • Familiarity with the Normal equation for linear regression
  • Knowledge of least squares estimation
  • Basic statistical principles and model assumptions
NEXT STEPS
  • Research the derivation of the Normal equation in linear regression
  • Explore the implications of roundoff error in numerical methods
  • Learn about the assumptions underlying linear regression models
  • Investigate alternative regression techniques and their applications
USEFUL FOR

Statisticians, data analysts, and students studying linear regression who seek to understand the nuances between different calculation methods and their implications for accuracy in statistical modeling.

BrownishMonst
Messages
1
Reaction score
0
I'm currently looking at a linear regression handout from Uni and there are two methods to calculate the equation. The Normal one is to find a and b for y=a+bx, the equations for a and b are given in the handout but I'll assume you're familiar with them. The simplified one is using

[itex]y = Bx + (\overline{y} − B\overline{x}[/itex])​

The two produce slightly different values and I assume the normal one is more precise than the simplified.

What I'd like to know is which one would be best to use when? Is the difference negligible?

Thanks, and sorry if I've posted this in the wrong forum.
 
Physics news on Phys.org
The two are mathematically identical, so the difference is likely roundoff error due to the difference in computation.

[itex]y = Bx + (y - Bx)[/itex]
[itex]y = Bx + A[/itex]
[itex]y = A + Bx[/itex]

Note that the standard least squares estimates are the best linear unbiased estimates (as well as the maximum likelihood estimates), provided that the model assumptions are met. In this case, you would rarely want to use anything else.
 

Similar threads

  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 30 ·
2
Replies
30
Views
5K
Replies
3
Views
3K
  • · Replies 6 ·
Replies
6
Views
2K
  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 64 ·
3
Replies
64
Views
6K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 8 ·
Replies
8
Views
3K