Estimating Parameters in Multivariate Regression

Click For Summary
SUMMARY

The discussion centers on the estimation of parameters in multivariate regression, specifically addressing the relationship between the estimated coefficient of a single explanatory variable and the true multiple regression coefficients. It is established that the equation proposed for estimating beta_1 is only valid under specific conditions, primarily when accounting for correlations among all explanatory variables. The need for additional terms to capture these correlations is emphasized, particularly the influence of correlations between x2 and x3 on the estimation of beta_1.

PREREQUISITES
  • Understanding of multivariate regression analysis
  • Familiarity with correlation coefficients and their implications
  • Knowledge of regression coefficients and their estimation
  • Proficiency in statistical notation and equations
NEXT STEPS
  • Study the implications of multicollinearity in regression models
  • Learn about the Generalized Least Squares (GLS) method for parameter estimation
  • Explore the concept of partial correlation in multivariate statistics
  • Investigate the use of software tools like R or Python for multivariate regression analysis
USEFUL FOR

Statisticians, data analysts, and students studying regression analysis who seek to deepen their understanding of parameter estimation in multivariate contexts.

TranscendArcu
Messages
277
Reaction score
0

Homework Statement



Skjermbilde_2012_07_02_kl_11_37_49_AM.png


The Attempt at a Solution



So I was wondering whether or not, in an instance of n observations and k explanatory variables, where the following is an accurate statement:
Code_Cogs_Eqn.gif


That is, the estimate of beta_1 found by only regressing y on x_1 is equal to the the true multiple regression, beta_1_hat plus all the effects of the x_j on y (the beta_j's) times the slope estimate found by regressing x_j with j≠1 on x_1.

Apparently this is not true, and the following explanation was offered:

"[This equation] turns out to only be correct in some very specific circumstances. The problem is that we have to account for correlations between all the x's. So it's not just the correlation of x1 with x2 and the corr of x1 with x3 that matters, but also the corr of x2 with x3 will play a part. So there would have to be additional terms that allow for that. "

I'm just wondering if these additional terms with be positive if x2 and x3 are highly correlated, and zero if they are uncorrelated. Can anybody help me with understanding this?
 
Physics news on Phys.org
I'm just thinking out loud here, but might it be conceivable to write the following equation for the general case:

\tilde{\beta}_1 = \hat{\beta}_1 + \Sigma_{j=2}^k (\hat{\beta}_j) \frac{\Sigma_{i=1}^n x_{1i}(x_ji - \bar{x}_j)}{\Sigma_{i=1}^n x_{1i}(x_1i - \bar{x}_1)} + \Sigma_{j=2} ^k Err[corr(x_1,x_j)]

Thus, we would have an "error term" (maybe error is the wrong word to use) that accounts for the correlation of x1 and xj. What do you guys think?
 

Similar threads

  • · Replies 23 ·
Replies
23
Views
4K
  • · Replies 3 ·
Replies
3
Views
2K
Replies
7
Views
10K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 3 ·
Replies
3
Views
3K
Replies
7
Views
7K
  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 2 ·
Replies
2
Views
1K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 6 ·
Replies
6
Views
10K