A hideous Linear Regression/confidence set question

  • Thread starter Thread starter Phillips101
  • Start date Start date
  • Tags Tags
    Linear Set
Click For Summary
SUMMARY

The discussion centers on deriving the distribution of the expression \((\hat{\beta} - \beta)' X'X (\hat{\beta} - \beta)\) in the context of linear regression, specifically under the assumption that \(e \sim N_n(0, \sigma^2 I)\). The user concludes that this expression follows a \(\sigma^2 \chi^2(n-p)\) distribution. Furthermore, they seek assistance in constructing a \((1-\alpha)\)-level confidence set for \(\beta\) using an F distribution, acknowledging the need for an additional chi-squared variable to facilitate this process. The discussion highlights the challenge of estimating variance and the implications of using \(\hat{\sigma}^2 = \frac{1}{n} ||Y - X\hat{\beta}||^2\) for constructing the confidence set.

PREREQUISITES
  • Understanding of linear regression models and maximum likelihood estimation (MLE).
  • Familiarity with chi-squared and F distributions in statistical inference.
  • Knowledge of matrix algebra, particularly transposes and vector operations.
  • Experience with confidence intervals and sets in the context of regression analysis.
NEXT STEPS
  • Study the derivation of the chi-squared distribution in the context of linear regression.
  • Learn how to construct confidence sets for regression coefficients using F distributions.
  • Explore the properties of maximum likelihood estimators (MLE) in linear models.
  • Investigate the implications of variance estimation in regression analysis.
USEFUL FOR

Statisticians, data analysts, and researchers involved in regression analysis and statistical inference, particularly those working with linear models and confidence intervals.

Phillips101
Messages
33
Reaction score
0
Take the linear model Y=X*beta+e, where e~Nn(0, sigma^2 * I), and it has MLE beta.hat

First, find the distribution of (beta.hat-beta)' * X'*X * (beta.hat-beta), where t' is t transpose. I think I've done this. I think it's a sigma^2 chi-squared (n-p) distribution.

Next, Hence find a (1-a)-level confidence set for beta based on a root with an F distribution. I can't do this to save my life. I'm aware that an F distribution is the ratio of two chi-squareds, but where the hell I'm going to get another chi squared from I have no idea. Also, we're dealing in -vectors- and I don't know how,what,why any confidence set is going to be or even look like, and I've no idea how to even try to get one.

-Any- help would be appreciated. Thanks
 
Physics news on Phys.org
Notice that

<br /> \frac{\hat{\beta}&#039; X&#039;X \hat{\beta}}{\sigma^2}<br />

has a \Chi^2 distribution. however, the variance is unknown, so you need to estimate it (with another expression from the regression). What would you use for the estimate, and what is its distribution?
 
Use the MLE sigma2.hat=(1/n)*||Y-Xbeta.hat||^2 ? This is distributed as a chi-squared n-1 variable if I remember correctly...
 
If that's correct, then the thing you posted is distributed as an F distribution, which is what I need? And would swapping beta.hat for beta.hat-beta make any difference to this?
 

Similar threads

Undergrad What Are the Key Differences Between OLS and GLM Models in Statistical Analysis?
  • · Replies 3 ·
Replies
3
Views
3K
Undergrad Nonlinear Least Squares or OLS for Nonlinear Models?
  • · Replies 7 ·
Replies
7
Views
2K
Undergrad Linear regression and random variables
  • · Replies 30 ·
2
Replies
30
Views
4K
Undergrad Does the generalized gamma distribution have a mgf?
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad Proof request for best linear predictor
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad Desirable estimator properties...
  • · Replies 7 ·
Replies
7
Views
3K
Undergrad Linear regression with error term
  • · Replies 8 ·
Replies
8
Views
2K
MHB Determine the confidence interval
  • · Replies 3 ·
Replies
3
Views
1K
Undergrad Auto Regressive Moving Average Model (ARMA) Ljung-Box Test
  • · Replies 8 ·
Replies
8
Views
2K
Undergrad Data Analaysis -- How to choose the best statistical model to use?
  • · Replies 5 ·
Replies
5
Views
2K