- #1

- 1,538

- 107

- TL;DR Summary
- understand the expected coefficient change (magnitude and sign) from simple to multiple linear regression

Hello forum,

I have created some linear regression models based on a simple dataset with 4 variables (columns). The first models simply involve one predictor variable: $$Y=\beta_1 X_1+\beta_0$$ and $$Y=\beta_2 X_2+ \beta_0$$

The 3rd model is multiple linear regression model involving the 3 predictors: $$Y= \beta_3 X_3 + \beta_2 X_2 + \beta_1 X_1 + \beta_0$$

I believe that the coefficient ##\beta_1## or ##\beta_2## for the predictors ##X_1## and ##X_2## change in magnitude when the two predictors are included together in the multivariate model (model 3), correct? What about the sign of those coefficients? Should the sign stay the same or can it possibly change?

I would think that the sign should remain the same to indicate that the variable ##Y## and ##X_1## (or ##X_2##) vary in the same direction in both the simple and multiple linear regression models...

Now, if multicollinearity is present, the coefficients for each predictor would certainly change in magnitude and sign from the coefficients in the simple linear regression model but not in the correct way due to the inter-variable correlation...

Thanks

I have created some linear regression models based on a simple dataset with 4 variables (columns). The first models simply involve one predictor variable: $$Y=\beta_1 X_1+\beta_0$$ and $$Y=\beta_2 X_2+ \beta_0$$

The 3rd model is multiple linear regression model involving the 3 predictors: $$Y= \beta_3 X_3 + \beta_2 X_2 + \beta_1 X_1 + \beta_0$$

I believe that the coefficient ##\beta_1## or ##\beta_2## for the predictors ##X_1## and ##X_2## change in magnitude when the two predictors are included together in the multivariate model (model 3), correct? What about the sign of those coefficients? Should the sign stay the same or can it possibly change?

I would think that the sign should remain the same to indicate that the variable ##Y## and ##X_1## (or ##X_2##) vary in the same direction in both the simple and multiple linear regression models...

Now, if multicollinearity is present, the coefficients for each predictor would certainly change in magnitude and sign from the coefficients in the simple linear regression model but not in the correct way due to the inter-variable correlation...

Thanks