Regression least squares (boring question? or more to it)

Click For Summary

Discussion Overview

The discussion revolves around the assumptions required for least squares fitting in linear regression, the derivation of least squares estimators, and the interpretation of an unbiased estimator involving linear combinations of parameters. Participants explore whether the questions posed in an exam context require specific calculations or if they can be answered with theoretical knowledge alone.

Discussion Character

  • Homework-related
  • Technical explanation
  • Debate/contested

Main Points Raised

  • One participant questions the necessary assumptions on the errors \(e_i\) for justifying a least squares fit, suggesting that \(E(e_i)=0\) might be the only requirement, while also pondering the role of normal distribution.
  • Another participant proposes that the assumptions include normal distribution of errors, constant variance (homoscedasticity), and independence of errors, asserting that these are essential for the validity of linear regression.
  • A participant confirms that the unbiased estimator of \(a + 10b\) is simply \(a^* + 10b^*\), citing the linearity of expectation, and expresses frustration over the perceived simplicity of the question.
  • There is a discussion about whether to provide just the formulas for the least squares estimators \(b^* = S_{xy}/S_{xx}\) and \(a^* = \bar{y} - b^*\bar{x}\) or to elaborate further in an exam setting.
  • One participant argues that while there are no minimal assumptions required for least squares fitting, certain assumptions about the errors lead to additional theoretical insights regarding residuals and goodness of fit.

Areas of Agreement / Disagreement

Participants express differing views on the necessary assumptions for least squares fitting and whether the exam questions require detailed calculations or can be answered with theoretical knowledge. There is no consensus on the minimal assumptions needed or the depth of response expected in an exam context.

Contextual Notes

Some participants highlight that the assumptions about error distributions may not be strictly necessary for the application of least squares fitting, but they do affect the interpretation of results and the validity of certain statistical inferences.

GreenGoblin
Messages
68
Reaction score
0
yi = a + bxi + ei is the simple liner regression model as per is usual

"state the assumptions on the errors ei to justify a least squares fit"

? So is this just that E(ei)=0, i can't see what else is a 'must' for this? what about that they are normally distrubited? i know the properties of the errors but what does it mean state the assumptions?

"obtain the least squars estimators a* and b*"

right but there is no data set? so does this just mean give the formulae for the estimators? is this just simple write it down book work or is there something to do here? do you think an examiner must give full mark if you say just a* = yBAR - b*xBAR, b* = Sxy/Sxx?. I don't know what else they can be asking. i have no official solution to it.
t
"suggest an unbiased estimato of a + 10b".

now this one just annoys me the most since we have E(a*)=a and E(b*) = b, with the linearity of exectation, this is just going to be a* + 10b*? so there really is nothing to do in the whole question?

i am just lost if there is some calculations to actually do here since this is a pretty valuable question on exam paper and there appears to be no work to do..
 
Physics news on Phys.org
Regarding the assumptions, aren't they that the errors should be normally distributed, have constant variance (homoscedasticity) and they should be independent?

As far as I know (and stats isn't really my forte), the assumptions need to be met in order for a linear regression to be valid.
 
Can someone help/confirm my answers for these questions?

The assumptions on the errors are just the expectation of 0, variance of sigma^2, normally distributed, independenced

the unbiased estimator of a + 10b is JUST a* + 10b* (since Expectation of the estimators is just the estimators themselves, 10 is just a constant.. a nd expectation is linear.. this is a bum question really? are they just testing that you know these basic ideas?)

"obtain the least squares estimators"
do you think i can just say sxy/sxx for b* or do i give the formula? is this all they want and a* = yBAR - b*xBAR

thanks dave mk.!
 
There is no minimal set of assumptions needed to justify the use of least squares fitting, but you will not get any marks for that in an exam answer. For the model to be coherent you assume that \(E(\varepsilon_i)=0\), but that is an assumption of a linear model.

The normal equation work, and give a least squared fit line independent of any assumptions about the error distributions

That the \(\varepsilon_i\) are homoeostatic independent and normally distributed allow the use of additional theory that tells you about the distribution of residual, goodness of the fit, and optimality etc.

CB
 

Similar threads

Undergrad What is Nonlinear Least Squares in Regression Analysis?
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad Can Alternative Least Squares Methods Be Used in Linear Regression?
  • · Replies 6 ·
Replies
6
Views
3K
Undergrad What are the commonly used estimators in regression models?
  • · Replies 23 ·
Replies
23
Views
4K
Undergrad Least squares line - understanding formulas
  • · Replies 5 ·
Replies
5
Views
2K
Undergrad Bias in Linear Regression (x-intercept) vs Statistics
  • · Replies 2 ·
Replies
2
Views
2K
Undergrad Ridge Regression Cross Validation
  • · Replies 4 ·
Replies
4
Views
2K
Undergrad Desirable estimator properties...
  • · Replies 7 ·
Replies
7
Views
3K
Undergrad Linear regression with error term
  • · Replies 8 ·
Replies
8
Views
3K
Linear Least-Squares Regression: a_0 = ?
  • · Replies 3 ·
Replies
3
Views
3K
Graduate Does the Error Distribution Affect Least Squares Estimation in Simple Models?
  • · Replies 1 ·
Replies
1
Views
2K