A hideous Linear Regression/confidence set question

  • Context: Graduate 
  • Thread starter Thread starter Phillips101
  • Start date Start date
  • Tags Tags
    Linear Set
Click For Summary

Discussion Overview

The discussion revolves around the statistical properties of linear regression, specifically focusing on the distribution of a quadratic form involving the estimated coefficients and the construction of confidence sets for these coefficients. The scope includes theoretical aspects of statistical inference and distributional properties in the context of linear models.

Discussion Character

  • Technical explanation
  • Mathematical reasoning
  • Debate/contested

Main Points Raised

  • One participant proposes that the distribution of the quadratic form \((\hat{\beta}-\beta)' X'X (\hat{\beta}-\beta)\) is a \(\sigma^2 \chi^2(n-p)\) distribution.
  • Another participant notes that \(\frac{\hat{\beta}' X'X \hat{\beta}}{\sigma^2}\) has a \(\chi^2\) distribution but highlights the challenge of estimating the variance since it is unknown.
  • A third participant suggests using the maximum likelihood estimator \(\hat{\sigma}^2 = \frac{1}{n} ||Y - X\hat{\beta}||^2\) for estimating the variance, claiming it follows a chi-squared distribution with \(n-1\) degrees of freedom.
  • A later reply questions whether substituting \(\hat{\beta}\) with \(\hat{\beta}-\beta\) affects the distribution being discussed.

Areas of Agreement / Disagreement

Participants express uncertainty regarding the estimation of variance and the implications of different substitutions in the distribution. There is no consensus on the confidence set construction or the implications of the proposed substitutions.

Contextual Notes

Limitations include the dependence on the correct estimation of variance and the assumptions regarding the distribution of the residuals. The discussion does not resolve the mathematical steps necessary for constructing the confidence set.

Phillips101
Messages
33
Reaction score
0
Take the linear model Y=X*beta+e, where e~Nn(0, sigma^2 * I), and it has MLE beta.hat

First, find the distribution of (beta.hat-beta)' * X'*X * (beta.hat-beta), where t' is t transpose. I think I've done this. I think it's a sigma^2 chi-squared (n-p) distribution.

Next, Hence find a (1-a)-level confidence set for beta based on a root with an F distribution. I can't do this to save my life. I'm aware that an F distribution is the ratio of two chi-squareds, but where the hell I'm going to get another chi squared from I have no idea. Also, we're dealing in -vectors- and I don't know how,what,why any confidence set is going to be or even look like, and I've no idea how to even try to get one.

-Any- help would be appreciated. Thanks
 
Physics news on Phys.org
Notice that

[tex] \frac{\hat{\beta}' X'X \hat{\beta}}{\sigma^2}[/tex]

has a [tex]\Chi^2[/tex] distribution. however, the variance is unknown, so you need to estimate it (with another expression from the regression). What would you use for the estimate, and what is its distribution?
 
Use the MLE sigma2.hat=(1/n)*||Y-Xbeta.hat||^2 ? This is distributed as a chi-squared n-1 variable if I remember correctly...
 
If that's correct, then the thing you posted is distributed as an F distribution, which is what I need? And would swapping beta.hat for beta.hat-beta make any difference to this?
 

Similar threads

Replies
3
Views
3K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 30 ·
2
Replies
30
Views
5K
  • · Replies 1 ·
Replies
1
Views
3K
  • · Replies 1 ·
Replies
1
Views
3K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 3 ·
Replies
3
Views
1K
  • · Replies 8 ·
Replies
8
Views
2K