R2 Addition in OLS Regression: Unrelated Variables?

  • Context: Graduate 
  • Thread starter Thread starter monsmatglad
  • Start date Start date
  • Tags Tags
    linear regression
Click For Summary

Discussion Overview

The discussion revolves around the relationship between R^2 values in Ordinary Least Squares (OLS) regression when using independent variables that may or may not be correlated. Participants explore whether the sum of R^2 values from separate regressions can equal the R^2 value from a combined regression, particularly focusing on the conditions under which this holds true.

Discussion Character

  • Technical explanation
  • Debate/contested

Main Points Raised

  • One participant questions if the sum of R^2 values from two separate regressions requires the independent variables to be completely unrelated.
  • Another participant argues that the relationship between independent variables and sample data complicates the interpretation of R^2 values, suggesting that uncorrelated variables may not be represented as such in the data.
  • A different participant posits that if the sample indicates the independent variables are uncorrelated, it raises the question of whether R1^2 + R2^2 would equal R3^2.
  • Another participant cautions against drawing conclusions about the relationship between two variables based solely on their individual relationships to a third variable, providing a hypothetical scenario involving uncorrelated variables.
  • One participant reiterates the question about the sum of R^2 values, suggesting that it may be true under certain conditions, indicating a potential agreement with the previous point but emphasizing the complexity of the situation.

Areas of Agreement / Disagreement

Participants express differing views on the relationship between the independence of variables and the summation of R^2 values. There is no consensus on whether the sum of R^2 values can be equated in the presence of uncorrelated independent variables, indicating ongoing debate and uncertainty.

Contextual Notes

The discussion highlights the complexities involved in interpreting R^2 values, particularly regarding the correlation of independent variables and the implications for regression analysis. Assumptions about the nature of the data and the relationships between variables remain unresolved.

monsmatglad
Messages
75
Reaction score
0
Hey. I am working with OLS regression. First I run 2 regression operations with each having just one independent variable. Then I run another regression using both the independent variables from the first two regressions. If the explanatory "power" (R^2) in the third regression was to be the sum of the R^2 from the two first regressions, would this require the independent variables to be completely unrelated?
 
Physics news on Phys.org
The short answer is no.
You need to distinguish between the independent variables versus the sample data. Consider these two possibilities:
  • Two independent uncorrelated variables are unlikely to be represented in the data as though they are exactly uncorrelated.
  • Two correlated variables may be in a designed experiment where they appear in the data as though they are uncorrelated.
 
yes, but what if the sample shows that the independent variables are in fact uncorrelated, regardless of how likely it is that this represents the population, would then R1^2 + R2^2 = R3^2?

Mons
 
I think it is a mistake to draw conclusions about two variables from their individual relationships to a third variable.
Suppose you have three uncorrelated variables, ε1, ε2, and ε3.
Consider X1 = ε1 + ε3, X2 = ε2 + ε3, and Y = ε1 + ε2

X1 and X2 are correlated through ε3, which is uncorrelated with Y.
The individual correlations of Y with X1 and X2 are through the uncorrelated variables ε1 and ε2, respectively.
 
Last edited:
monsmatglad said:
yes, but what if the sample shows that the independent variables are in fact uncorrelated, regardless of how likely it is that this represents the population, would then R1^2 + R2^2 = R3^2?
This is not the same question as the original post. I think this may be true.
 
Last edited:

Similar threads

Undergrad Linear regression, feature scaling, and regression coefficients
  • · Replies 8 ·
Replies
8
Views
3K
Undergrad What are the commonly used estimators in regression models?
  • · Replies 23 ·
Replies
23
Views
4K
Undergrad Linear regression and random variables
  • · Replies 30 ·
2
Replies
30
Views
4K
Undergrad Expected coefficient change from simple to multiple linear regression
  • · Replies 4 ·
Replies
4
Views
2K
High School Limitations of Multivariate Linear Regression
  • · Replies 6 ·
Replies
6
Views
3K
Undergrad Nonlinear Least Squares or OLS for Nonlinear Models?
  • · Replies 7 ·
Replies
7
Views
3K
Graduate Using standard deviation values as independent variables
  • · Replies 1 ·
Replies
1
Views
2K
High School Comparing Approaches: Linear Regression of Y on X vs X on Y
  • · Replies 64 ·
3
Replies
64
Views
6K
Undergrad Linear Model with independent categorical variable
  • · Replies 5 ·
Replies
5
Views
2K
Undergrad How to propagate errors through a regression & non-linear model?
  • · Replies 8 ·
Replies
8
Views
3K