Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Linear regression and bivariate normal, is there a relationship?

  1. Aug 27, 2011 #1
    Hi everyone,

    This is not a homework question. I just want to understand an aspect of linear regression better. The book "Applied Linear Models" by Kutchner et al, states that a linear regression model is of the form

    [tex] Y_i = B_0 + B_1 X_i + \epsilon_i [/tex]

    [itex] Y_i [/itex] is the value of the response variable in the ith trial
    [itex] B_0, B_1 [/itex] are parameters
    [itex] X_i [/itex] is a known constant
    [itex] \epsilon_i [/itex] is a random variable, normally distributed.
    Therefore, [itex]Y_i [/itex] is also a random variable, normally distributed but [itex]X_i [/itex] is a constant.

    This confused me a bit because I always associated linear regression with the bivariate normal distribution. That is, the underlying assumption of linear regression is the data [itex]\{(x_1,y_1), (x_2,y_2),...,(x_n,y_x) \} [/itex] is sampled from a bivariate normal distribution. In which case, both X and Y are random variables. But in the formulation above, X is a known constant, while [itex]\epsilon[/itex] and therefore [itex]Y[/itex] are the random variables.

    So in summary, what is the connection (if any) is between linear regression as formulated by Kutner and the bivariate normal.
  2. jcsd
  3. Aug 27, 2011 #2

    Stephen Tashi

    User Avatar
    Science Advisor

    I've never seen a treatment of regression that made that assumption. Are you confusing linear regession with some sort of "total least squares" regression?
  4. Aug 27, 2011 #3
    Thanks for responding, Stephen.

    Yea, that was my own confusion for making that assumption. Thanks for clearing that up.

    By the way, total least squares is just a generalization of linear regression in that the curve you're fitting the data points to can be polynomials with degrees higher than 1, right? Or is there more to total least squares?
  5. Aug 27, 2011 #4

    Stephen Tashi

    User Avatar
    Science Advisor

    Total least squares treats both X and Y as random variables.
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook