Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Linear Regression Models (1)

  1. May 14, 2009 #1
    1) "In regression models, there are two types of variables:
    X = independent variable
    Y = dependent variable
    Y is modeled as random.
    X is sometimes modeled as random and sometimes it has fixed value for each observation."

    I don't understand the meaning of the last line. When is X random? When is X fixed? Can anyone illustrate each case with a quick example?

    2) "Simple linear regression model: Y = β0 + β1X + ε
    If X is random, E(Y|X) = β0 + β1X
    If X is fixed, E(Y|X=x) = β0 + β1x"

    Now what's the difference between E(Y|X) and E(Y|X=x)? The above is suuposed to be dealing with 2 separate cases (X random and X fixed), but I don't see any difference...
    Most of the time, I am seeing E(Y) = β0 + β1X instead, how come??? This is inconsistent with the above. E(Y) is not the same as E(Y|X=x) and I don't think they can ever be equal.

    Thanks for explaining!
    Last edited: May 14, 2009
  2. jcsd
  3. May 18, 2009 #2
    In many cases the question whether X is random is theoretical. A clear-cut case for nonrandom X is the time trend (e.g., seconds into the experiment, or years into the Obama administration, etc.). Two clear cases of random X is (a) when X is co-determined with Y; and (b) when X is measured with random error.

    E(Y|X) implies that the random variable X is not assumed to take on a particular value; E(Y|X=x) implies X is assumed to equal the predetermined, nonrandom value x. E[Y] is being used as a shorthand for "E[Y|X] if X is random, E[Y|x] otherwise."
  4. May 20, 2009 #3
    For example, if we have height v.s. age (Y v.s. X), is X fixed or random?

    Also, what does it mean for X to be FIXED? If we have five data points, x1,x2...,x5, and NOT all of them have the same value of X (e.g. x1≠x2), is X fixed in this case?

    Thank you!
  5. May 20, 2009 #4
    "Fixed vs. random" usually depends on your goal. In your example, height vs. age, there may be at least two different contexts:

    1. Heights of 10 children are measured at ages 1 through 10. We would like to determine the relationship between height and age for these 10 children.

    2. 100 children are selected at random from a population of 10,000; their ages are recorded and their heights are measured. We would like to determine a general relationship between height and age for the entire population, based on this sample.

    In case 1, age is fixed. In case 2, it is random.
  6. May 20, 2009 #5
    Thanks for the concrete examples. Things make a lot more sense now!
  7. May 21, 2009 #6
    2) By definitions,

    ∫ y f(y) dy


    ∫ y f(y|x) dy

    If X is FIXED, does this ALWAYS imply that X and Y are INDEPENDENT and E(Y)=E(Y|X=x)?? Why or why not?

    For simple linear regression model, my textbook typically write
    Y= β0 + β1*X + ε as
    E(Y) = β0 + β1*X

    However, I have seen occasionally that
    Y= β0 + β1*X + ε is written as
    E(Y|X) = β0 + β1*X which looks a bit inconsistent to the above...how come? The definitions of E(Y) and E(Y|X) are clearly different as I outlined above, but here it seems like they are equal? How come?

    Thanks for explaining!
    Last edited: May 22, 2009
  8. May 22, 2009 #7
    E(Y) (using shorthand notation) is a function of X: E(Y) = b0 + b1 X. That means E(Y) is never indep. of X; the question is whether it's a dependence on a nonrandom variable ("x"), or a random variable ("X"). As I explained above, E(Y) is a shorthand notation.
  9. May 29, 2009 #8
    In their Econometric Foundations, Mittelhammer, Judge & Miller hold "E[Y] = E[Y|X] whenever X = x," (i.e. always). [Not an exact quotation.]
    Last edited: May 29, 2009
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Similar Discussions: Linear Regression Models (1)
  1. Linear regression (Replies: 7)