R2 Addition in OLS Regression: Unrelated Variables?

In summary, the conversation discusses the relationship between independent variables and their explanatory power in a regression analysis. It is noted that having two independent variables with a correlation does not necessarily mean they will also be correlated in the data. Additionally, it is mentioned that drawing conclusions about the correlation between two variables based on their individual relationships with a third variable may not be accurate. Ultimately, the question is raised about whether the sum of the R-squared values from two regressions would equal the R-squared value from a third regression if the independent variables were uncorrelated in the data.
  • #1
monsmatglad
76
0
Hey. I am working with OLS regression. First I run 2 regression operations with each having just one independent variable. Then I run another regression using both the independent variables from the first two regressions. If the explanatory "power" (R^2) in the third regression was to be the sum of the R^2 from the two first regressions, would this require the independent variables to be completely unrelated?
 
Physics news on Phys.org
  • #2
The short answer is no.
You need to distinguish between the independent variables versus the sample data. Consider these two possibilities:
  • Two independent uncorrelated variables are unlikely to be represented in the data as though they are exactly uncorrelated.
  • Two correlated variables may be in a designed experiment where they appear in the data as though they are uncorrelated.
 
  • #3
yes, but what if the sample shows that the independent variables are in fact uncorrelated, regardless of how likely it is that this represents the population, would then R1^2 + R2^2 = R3^2?

Mons
 
  • #4
I think it is a mistake to draw conclusions about two variables from their individual relationships to a third variable.
Suppose you have three uncorrelated variables, ε1, ε2, and ε3.
Consider X1 = ε1 + ε3, X2 = ε2 + ε3, and Y = ε1 + ε2

X1 and X2 are correlated through ε3, which is uncorrelated with Y.
The individual correlations of Y with X1 and X2 are through the uncorrelated variables ε1 and ε2, respectively.
 
Last edited:
  • #5
monsmatglad said:
yes, but what if the sample shows that the independent variables are in fact uncorrelated, regardless of how likely it is that this represents the population, would then R1^2 + R2^2 = R3^2?
This is not the same question as the original post. I think this may be true.
 
Last edited:

Related to R2 Addition in OLS Regression: Unrelated Variables?

1. What is the purpose of adding R2 in OLS regression?

The addition of R2 in OLS regression allows for the evaluation of how well the regression model fits the data. It measures the proportion of variation in the dependent variable that can be explained by the independent variables. A higher R2 indicates a better fit of the model.

2. How is R2 calculated in OLS regression?

R2 is calculated by dividing the explained sum of squares by the total sum of squares. The explained sum of squares is the sum of squared differences between the predicted values and the mean of the dependent variable. The total sum of squares is the sum of squared differences between the actual values and the mean of the dependent variable.

3. Can R2 be negative in OLS regression?

No, R2 cannot be negative in OLS regression. It can range from 0 to 1, with 0 indicating no relationship between the independent and dependent variables and 1 indicating a perfect relationship.

4. How do unrelated variables affect the R2 value in OLS regression?

Unrelated variables will not significantly affect the R2 value in OLS regression. This is because R2 measures the relationship between the independent and dependent variables, not the relationship between the independent variables themselves. However, adding unrelated variables to the model can decrease the interpretability of the results and lead to overfitting.

5. Is R2 the only measure of model fit in OLS regression?

No, R2 is not the only measure of model fit in OLS regression. Other commonly used measures include the adjusted R2, which takes into account the number of independent variables in the model, and the root mean squared error, which measures the average difference between the predicted and actual values. It is important to consider multiple measures of model fit when evaluating the performance of a regression model.

Similar threads

I Exploring OLS and GLM Models: Understanding the Link Function and Coefficients
    • Set Theory, Logic, Probability, Statistics
Replies
3
Views
884
I Linear regression, feature scaling, and regression coefficients
    • Set Theory, Logic, Probability, Statistics
Replies
8
Views
1K
I Can Alternative Least Squares Methods Be Used in Linear Regression?
    • Set Theory, Logic, Probability, Statistics
Replies
6
Views
1K
I What are the commonly used estimators in regression models?
    • Set Theory, Logic, Probability, Statistics
Replies
23
Views
2K
I Coefficient sign flip in linear regression with correlated predictors
    • Set Theory, Logic, Probability, Statistics
Replies
13
Views
2K
I Nonlinear least squares vs OLS
    • Set Theory, Logic, Probability, Statistics
Replies
7
Views
509
I Linear regression and random variables
    • Set Theory, Logic, Probability, Statistics
Replies
30
Views
2K
I Expected coefficient change from simple to multiple linear regression
    • Set Theory, Logic, Probability, Statistics
Replies
4
Views
1K
I Multicollinearity and Interactions
    • Set Theory, Logic, Probability, Statistics
Replies
5
Views
1K
B Limitations of Multivariate Linear Regression
    • Set Theory, Logic, Probability, Statistics
Replies
6
Views
1K

Hot Threads

Recent Insights

Back
Top