Linear Regression: Pros and cons of Normal vs. simplified methods?

In summary, there are two methods for calculating the equation in linear regression: the Normal method and the simplified method. The normal method is more precise and produces slightly different values compared to the simplified method. However, both methods are mathematically identical and the difference is likely due to roundoff error. It is generally recommended to use the normal method, as it provides the best linear unbiased estimates, assuming all model assumptions are met.
  • #1
BrownishMonst
1
0
I'm currently looking at a linear regression handout from Uni and there are two methods to calculate the equation. The Normal one is to find a and b for y=a+bx, the equations for a and b are given in the handout but I'll assume you're familiar with them. The simplified one is using

[itex]y = Bx + (\overline{y} − B\overline{x}[/itex])​

The two produce slightly different values and I assume the normal one is more precise than the simplified.

What I'd like to know is which one would be best to use when? Is the difference negligible?

Thanks, and sorry if I've posted this in the wrong forum.
 
Mathematics news on Phys.org
  • #2
The two are mathematically identical, so the difference is likely roundoff error due to the difference in computation.

[itex]y = Bx + (y - Bx)[/itex]
[itex]y = Bx + A[/itex]
[itex]y = A + Bx[/itex]

Note that the standard least squares estimates are the best linear unbiased estimates (as well as the maximum likelihood estimates), provided that the model assumptions are met. In this case, you would rarely want to use anything else.
 

1. What is linear regression?

Linear regression is a statistical method used to model the relationship between two or more variables. It is used to predict the value of one variable based on the values of other variables.

2. What are the pros of using the normal method for linear regression?

The normal method, also known as the Ordinary Least Squares (OLS) method, is the most commonly used approach for linear regression. Its pros include simplicity, interpretability, and the ability to handle a large number of variables.

3. What are the cons of using the normal method for linear regression?

Some potential cons of using the normal method for linear regression include its sensitivity to outliers, the assumption of linear relationships between variables, and potential multicollinearity issues when dealing with highly correlated variables.

4. What are the pros of using simplified methods for linear regression?

Simplified methods, such as the Ridge and Lasso regression, have the advantage of being able to handle multicollinearity issues and reduce the impact of outliers. They also allow for the inclusion of non-linear relationships between variables.

5. What are the cons of using simplified methods for linear regression?

Some cons of using simplified methods for linear regression include the potential loss of interpretability, as well as the need for tuning parameters in order to achieve optimal results. These methods also may not perform well with small or unbalanced datasets.

Similar threads

Replies
8
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
3
Views
476
  • Set Theory, Logic, Probability, Statistics
Replies
3
Views
841
  • Set Theory, Logic, Probability, Statistics
Replies
30
Views
2K
  • STEM Educators and Teaching
Replies
11
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
494
  • Set Theory, Logic, Probability, Statistics
2
Replies
64
Views
3K
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
8
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
894
Back
Top