Least squares assumptions: finite and nonzero 4th moments

In summary, according to the textbook, one of the least squares assumptions is that large outliers are unlikely. If this assumption is not met, then the results of the regression are very sensitive to the presence of outliers.
  • #1
slakedlime
76
2
This isn't a homework problem - I'm just confused by something in a textbook that I'm reading (not for a class, either). I'd appreciate an intuitive clarification, or a link to a good explanation (can't seem to find anything useful on Google or in my textbook).

My book states that one of the least squares assumptions (e.g. for ordinary least squares, OLS, estimation) is that large outliers are unlikely.

That is, for the following equation:
[itex]Y_{i}[/itex] = [itex]β_{0}+β_{1}X_i+u_{i}[/itex]

It must be that ([itex]X_{i}[/itex], [itex]Y_{i}[/itex]), i = 1, ..., n have nonzero finite fourth moments.

Why is this significant? What is the relationship between large outliers and nonzero finite fourth moments? I don't intuitively see the mathematical explanation. Any help and/or direction is much appreciated.
 
Physics news on Phys.org
  • #2
The real importance of the fourth moment statement is that with it in place the underlying ideas needed for consistent estimation of variances are easily verified.
The argument that links the finite fourth moments to outliers can be intuitively stated as: if the fourth moments are finite, then the tails of the distribution are relatively short, so the PROBABILITY of unusually large observations occurring is small. In that regard it's an assumption made to try to account for the fact that least squares regression (least squares methods in general) are non-robust and results are very sensitive to the presence of outliers.
The better notion is: if you believe outliers could be an issue, use a method for robust regression.

The notion you ask about is discussed in this article.
http://www.aw-bc.com/info/stock_watson/Chapter4.pdf
 
  • #3
Thank you so much!
 
  • #4
thank you
 
  • #5
statdad said:
The real importance of the fourth moment statement is that with it in place the underlying ideas needed for consistent estimation of variances are easily verified.
The argument that links the finite fourth moments to outliers can be intuitively stated as: if the fourth moments are finite, then the tails of the distribution are relatively short, so the PROBABILITY of unusually large observations occurring is small. In that regard it's an assumption made to try to account for the fact that least squares regression (least squares methods in general) are non-robust and results are very sensitive to the presence of outliers.
The better notion is: if you believe outliers could be an issue, use a method for robust regression.

The notion you ask about is discussed in this article.
http://www.aw-bc.com/info/stock_watson/Chapter4.pdf

Access is forbidden!
 

1. What are the least squares assumptions?

The least squares assumptions are a set of conditions that must be met in order to use the least squares method for linear regression. These assumptions include linearity, independence of errors, homoscedasticity, normality of errors, and finite and nonzero 4th moments.

2. What is the significance of finite and nonzero 4th moments?

The finite and nonzero 4th moments assumption states that the fourth moment of the error term in a linear regression model must be finite and nonzero. This ensures that the distribution of the error term is not too heavily skewed or has outliers, which can affect the accuracy of the regression results.

3. How can we check for finite and nonzero 4th moments?

To check for finite and nonzero 4th moments, we can use statistical tests such as the Jarque-Bera test or the Shapiro-Wilk test. These tests assess the normality of the distribution of the error term, and a p-value greater than 0.05 indicates that the assumption is met.

4. What happens if the finite and nonzero 4th moments assumption is violated?

If the finite and nonzero 4th moments assumption is violated, it means that the distribution of the error term is not normal. This can lead to biased and unreliable regression results, as the least squares method assumes a normal distribution of errors.

5. Are there any alternatives to using the least squares method if the finite and nonzero 4th moments assumption is violated?

Yes, there are alternative methods such as generalized least squares or robust regression which can be used when the assumption of normality is violated. These methods are more robust to deviations from normality and can provide more accurate results in such cases.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
7
Views
474
  • Calculus and Beyond Homework Help
Replies
1
Views
2K
  • Calculus and Beyond Homework Help
Replies
1
Views
3K
Replies
2
Views
2K
  • Engineering and Comp Sci Homework Help
Replies
8
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
2K
  • Differential Equations
Replies
3
Views
1K
  • Engineering and Comp Sci Homework Help
Replies
1
Views
1K
Replies
5
Views
1K
  • General Math
Replies
2
Views
2K
Back
Top