Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Why does conditional probability used in mean square error equal zero?

  1. Mar 27, 2014 #1
    Hi guys,

    I am having trouble showing that [itex]\mathbb{E}\left[(Y-\mathbb{E}[Y|X])^{2}\right]=0[/itex].

    I understand the proof of why E[Y|X] minimizes the mean square error, but I cannot understand why it is then equal to zero.

    I tried multiplying out the square to get [itex]\mathbb{E}\left[Y^{2}\right]-2\mathbb{E}\left[Y\mathbb{E}[Y|X]\right]+\mathbb{E}\left[\mathbb{E}[Y|X]\mathbb{E}[Y|X]\right][/itex]
    but have not been able to justify [itex]\mathbb{E}\left[Y\mathbb{E}[Y|X]\right]=\mathbb{E}\left[Y^{2}\right]
    [/itex] or [itex]\mathbb{E}\left[\mathbb{E}[Y|X]\mathbb{E}[Y|X]\right]=\mathbb{E}\left[Y^{2}\right][/itex].

    Thanks in advance.
  2. jcsd
  3. Mar 27, 2014 #2
    Can you tell us your definition of the conditional expectation and what properties you are allowed to use?
  4. Mar 27, 2014 #3

    Stephen Tashi

    User Avatar
    Science Advisor

    It would also be helpful to improve the notation. If [itex] X [/itex] and [itex] Y [/itex] are random variables and [itex] f(X,Y) [/itex] is a function of them then the notation [itex] E f(X,y) [/itex] is ambiguous. It is not clear whether the expectation is being computed with respect to the distribution of [itex] X [/itex] or the distribution of [itex] Y [/itex] - or perhaps with respect to the joint distribution for [itex] (X,Y) [/itex].

    You can use a subscript to denote which distribution is used to compute the expectation. For example, if [itex] Y [/itex] is not a function of [itex] X [/itex] then [itex] E_X ( E_Y ( 3Y + 1) ) [/itex] is the expectation of with respect to the distribution of [itex] X [/itex] of the constant value [itex] E_Y(3Y + 1) [/itex] Hence [itex] E_X (E_Y (3Y+1)) = E_Y (3Y+ 1) [/itex].
  5. Mar 28, 2014 #4


    User Avatar
    Science Advisor
    Gold Member
    2017 Award

    It's hard to prove because it is not true unless (Y-E(Y|X))==0. Are you sure that (Y-E(Y|X)) is supposed to be squared?
  6. Mar 28, 2014 #5

    Stephen Tashi

    User Avatar
    Science Advisor

    And before (Y - E(Y|X)) is equal or not equal to zero, it would have to mean something. How do we interpret Y - E(Y|X) ? Is it a random variable? To realize it , do we realize a value Y = y0 from the distribution of Y and then take the expected value of the constant y0 with respect to the distribution of X ?
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook