1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Linear regression

  1. Sep 17, 2008 #1
    1. The problem statement, all variables and given/known data

    Consider model of linear regression:

    Y_i = \beta_0 + x_i \beta_1 + \epsilon_i

    i = 1, ..., 5, where [itex]\epsilon_i \sim \mathcal{N}(0, \sigma^2)[/itex] are independent. Find expected value and variance of predicted values [tex]\widehat{Y}_i[/tex] considering that observations are obtained in points 1, 2, 3, 4, 5 (ie. [tex]x_i = i[/tex] for i = 1, ..., 5) and [tex]\sigma^2 = 1[/tex]. Hint: remember that

    \widehat{Y} = HY

    2. Relevant equations

    H = X\left(X^T X\right)^{-1}X^T Y

    3. The attempt at a solution

    My attempt is

    E \widehat{Y} = \beta_0 + X\beta_1 = (\beta_0 + \beta_1, \beta_0 + 2\beta_1, \beta_0 + 3\beta_1, \beta_0 + 4\beta_1, \beta_0 + 5\beta_1)

    Is it correct?

    Anyway, even if it is, how do I find the variance and how do I use the hint? :)

    Thank you.
  2. jcsd
  3. Sep 18, 2008 #2


    User Avatar
    Homework Helper

    "Find expected value and variance of predicted values"

    Since you know that

    \widehat Y = X \left(X' X)^{-1}X' Y

    you can find

    E[\widehat Y] = E[X \left(X' X\right)^{-1} X' Y]

    Use the properties of expected value and the expected value of [tex] Y [/tex] (unless I'm totally missing something, I don't see how the form of the [tex] x_i [/tex] applies here).

    As far as finding the variance of [tex] \widehat Y [/tex], you can use the hint. Write out the
    matrix [tex] X [/tex] (first column consists of ones, for the intercept, and you know the values of [tex] x [/tex] to use in the second column), and use the matrix formulas for the covariance matrix in regression to find the variances. Because the [tex] x [/tex] values are consecutive integers, a little algebra in the matrix multiplication will give nice forms for the entries.
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook