Linear regression and bivariate normal, is there a relationship?

[CantorSet]

Hi everyone,

This is not a homework question. I just want to understand an aspect of linear regression better. The book "Applied Linear Models" by Kutchner et al, states that a linear regression model is of the form

$$Y_i = B_0 + B_1 X_i + \epsilon_i$$

where
$Y_i$ is the value of the response variable in the ith trial
$B_0, B_1$ are parameters
$X_i$ is a known constant
$\epsilon_i$ is a random variable, normally distributed.
Therefore, $Y_i$ is also a random variable, normally distributed but $X_i$ is a constant.

This confused me a bit because I always associated linear regression with the bivariate normal distribution. That is, the underlying assumption of linear regression is the data $\{(x_1,y_1), (x_2,y_2),...,(x_n,y_x) \}$ is sampled from a bivariate normal distribution. In which case, both X and Y are random variables. But in the formulation above, X is a known constant, while $\epsilon$ and therefore $Y$ are the random variables.

So in summary, what is the connection (if any) is between linear regression as formulated by Kutner and the bivariate normal.

 

[Stephen Tashi]

the underlying assumption of linear regression is the data $\{(x_1,y_1), (x_2,y_2),...,(x_n,y_x) \}$ is sampled from a bivariate normal distribution. In which case, both X and Y are random variables.

I've never seen a treatment of regression that made that assumption. Are you confusing linear regession with some sort of "total least squares" regression?
http://en.wikipedia.org/wiki/Total_least_squares

 

[CantorSet]

I've never seen a treatment of regression that made that assumption. Are you confusing linear regession with some sort of "total least squares" regression?
http://en.wikipedia.org/wiki/Total_least_squares

Thanks for responding, Stephen.

Yea, that was my own confusion for making that assumption. Thanks for clearing that up.

By the way, total least squares is just a generalization of linear regression in that the curve you're fitting the data points to can be polynomials with degrees higher than 1, right? Or is there more to total least squares?

 

[Stephen Tashi]

Total least squares treats both X and Y as random variables.

 

[blue_raver22]

I can provide some clarification on the relationship between linear regression and the bivariate normal distribution.

First, it is important to understand that linear regression is a statistical method used to model the relationship between a dependent variable (Y) and one or more independent variables (X). The goal of linear regression is to find the best fitting line that describes the relationship between the variables.

On the other hand, the bivariate normal distribution is a probability distribution that describes the joint distribution of two continuous random variables. In the context of linear regression, the bivariate normal distribution is often used to model the relationship between the independent variable (X) and the error term (\epsilon). This is because the error term is assumed to follow a normal distribution in order to make statistical inference about the regression coefficients.

So, while the bivariate normal distribution is not explicitly used in the formulation of linear regression as described by Kutner, it is still an underlying assumption of the model. This means that the data points (x,y) are assumed to be sampled from a bivariate normal distribution, even though X is treated as a known constant.

In summary, there is a strong relationship between linear regression and the bivariate normal distribution, as the latter is often used to model the error term in the former. However, the formulation of linear regression may not explicitly mention the bivariate normal distribution, as it focuses on the relationship between the dependent and independent variables.