Does centering variables for regression always result in unchanged coefficients?

In summary, when centering variables for a regression analysis, the coefficients of non-interaction terms do not change. However, when there is an interaction between variables, the coefficients of the non-interaction terms do change due to the non-linear nature of the interaction. This is because linearity preserves translation, but non-linear interactions do not.
  • #1
monsmatglad
76
0
I am studying mean-centering for multiple linear regression (ols).
Specifically I'm talking about the situation when there is interaction.
When centering variables for a regression analysis, my literature tells me that the coefficients do not change? But when there is some sort of interaction between the variables, the coefficients of the non-interaction terms (the variables that take part in the interaction, but are also represented individually) of the variables do in fact change.

When it is said that when centering the variables, "the coefficients do not change", does that only apply to the non-integrated variables?
 
Physics news on Phys.org
  • #2
monsmatglad said:
When it is said that when centering the variables, "the coefficients do not change", does that only apply to the non-integrated variables?
What do you mean by 'non-integrated variables'?
 
  • #3
oops.. Was supposed to be "non-interaction"
 
  • #4
In that case, yes. Consider the model
$$y_j = a_0 + a_1x_1 + a_2x_2 +a_12x_1x_2 + a_3 x_3+\epsilon_j$$
in which there is an interaction of $x_1,x_2$ but no interactions for $x_3$.
Now centring each variable we get
$$y_j = a'_0 + a'_1(x_1-\bar x_1) + a'_2(x_2-\bar x_2) +a'_{12}(x_1-\bar x_1)(x_2-\bar x_2) + a'_3 (x_3-\bar x_3)+\epsilon_j$$
Rearranging this and matching coefficients to the first equation, we get:
  • ##a_0=a'_0-a'_1\bar x_1-a'_2\bar x_2-a'_3\bar x_3 +a'_12\bar x_1\bar x_2##
  • ##a_1=a'_1 - a'_{12}\bar x_2##
  • ##a_2=a'_2 - a'_{12}\bar x_1##
  • ##a_3=a'_3## [no change]
  • ##a_{12}=a'_{12}## [no change]
So the only coefficients that remain unchanged are those of any variables with no interactions, plus those of any interaction terms.
 
  • #5
I think this is just a property of linearity, which I believe is equivalent with a lack of interaction between variables, i.e., linearity "preserves translation" , but non-linear interactions do not.
 

1. What is interaction in Ordinary Least Squares (OLS)?

Interaction in OLS refers to the statistical technique of examining the effect of one independent variable on the relationship between two other independent variables. It allows for the testing of whether the relationship between two variables is dependent on the level of a third variable.

2. How is interaction represented in OLS?

Interaction in OLS is typically represented by including an interaction term in the regression model. This term is created by multiplying the two independent variables of interest together. For example, if we are examining the effect of education and gender on income, the interaction term would be education * gender. This allows us to estimate the effect of education on income for different levels of gender.

3. What is centering in OLS?

Centering in OLS refers to the process of subtracting the mean value of a variable from each individual data point. This is done in order to make the interpretation of the regression coefficients more meaningful and to reduce problems with multicollinearity.

4. Why is centering important in interaction analysis?

Centering is important in interaction analysis because it allows for a more accurate interpretation of the interaction term. By centering the variables, we are able to interpret the interaction effect as the change in the dependent variable when the independent variables are at their mean values, rather than at a specific value. This can help to avoid misleading interpretations of the interaction effect.

5. How do I interpret the results of a regression model with interaction and centered variables?

The interpretation of a regression model with interaction and centered variables is similar to that of a regular regression model. The coefficients for the independent variables represent the effect of that variable on the dependent variable when all other variables are held constant at their mean values. The interaction term represents the change in the effect of one independent variable on the dependent variable for a one unit change in the other independent variable. To interpret the interaction term, it is important to also consider the individual coefficients for each independent variable and their respective means.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
3
Views
698
  • Set Theory, Logic, Probability, Statistics
Replies
8
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
23
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
13
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
30
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
7
Views
335
Back
Top