Estimating Parameters in Multivariate Regression

1 reply · 2K views · last reply
TranscendArcu
Messages
277
Reaction score
0

Homework Statement



Homework problem 3 text: estimating the relationship between y and x1 with data on x2 and x3; asks whether simple regression estimate of beta1 would be similar to multiple regression estimate if x1 is highly correlated with x2 and x3 and x2 and x3 have large partial effects on y, explained in words.


The Attempt at a Solution



So I was wondering whether or not, in an instance of n observations and k explanatory variables, where the following is an accurate statement:
Equation: beta1 estimate from regressing y on x1 alone equals beta1_hat plus the sum over j from 2 to k of beta_j_hat times a ratio of sums of x1 times (xj minus its mean) over sums of x1 times (x1 minus its mean).


That is, the estimate of beta_1 found by only regressing y on x_1 is equal to the the true multiple regression, beta_1_hat plus all the effects of the x_j on y (the beta_j's) times the slope estimate found by regressing x_j with j≠1 on x_1.

Apparently this is not true, and the following explanation was offered:

"[This equation] turns out to only be correct in some very specific circumstances. The problem is that we have to account for correlations between all the x's. So it's not just the correlation of x1 with x2 and the corr of x1 with x3 that matters, but also the corr of x2 with x3 will play a part. So there would have to be additional terms that allow for that. "

I'm just wondering if these additional terms with be positive if x2 and x3 are highly correlated, and zero if they are uncorrelated. Can anybody help me with understanding this?
 
I'm just thinking out loud here, but might it be conceivable to write the following equation for the general case:

[tex]\tilde{\beta}_1 = \hat{\beta}_1 + \Sigma_{j=2}^k (\hat{\beta}_j) \frac{\Sigma_{i=1}^n x_{1i}(x_ji - \bar{x}_j)}{\Sigma_{i=1}^n x_{1i}(x_1i - \bar{x}_1)} + \Sigma_{j=2} ^k Err[corr(x_1,x_j)][/tex]

Thus, we would have an "error term" (maybe error is the wrong word to use) that accounts for the correlation of x1 and xj. What do you guys think?