Minimizing cost function Statistics

[colstat]

Homework Statement

Let X and Y be two unknown variables with E(Y)=$$\mu$$ and EY² < $$\infty$$.

Homework Equations

a. Show that the constant c that minimizes E(Y-c)² is c=$$\mu$$.
b. Deduce that the random variable f(X) that minimizes E[(Y-f(X))²|X] is f(X)= E[Y|X].
c. Deduce that the random variable f(X) that minimizes E[(Y-f(X))²] is also f(X)= E[Y|X].

The Attempt at a Solution

a. E(Y-c)² = E[Y²-2Y*c+c²]
other ideas: Can I do the derivative w.r.t. Y, set it equal to zero. But there is an expectation operator, how do you take expectation through the operator.
I can't use a posterior mean, the problem did not specify distr.of the r.v.

b, c. no ideas yet

 

[LCKurtz]

Homework Statement

Let X and Y be two unknown variables with E(Y)=$$\mu$$ and EY² < $$\infty$$.

Homework Equations

a. Show that the constant c that minimizes E(Y-c)² is c=$$\mu$$.
b. Deduce that the random variable f(X) that minimizes E[(Y-f(X))²|X] is f(X)= E[Y|X].
c. Deduce that the random variable f(X) that minimizes E[(Y-f(X))²] is also f(X)= E[Y|X].

The Attempt at a Solution

a. E(Y-c)² = E[Y²-2Y*c+c²]
other ideas: Can I do the derivative w.r.t. Y, set it equal to zero. But there is an expectation operator, how do you take expectation through the operator.
I can't use a posterior mean, the problem did not specify distr.of the r.v.

b, c. no ideas yet

For (a) Remember, the variable here is c, not Y. Start by using linearity to expand
E[Y²-2Y*c+c²] and remember that.

 

[colstat]

LCKurtz,
E[Y²-2Y*c+c²]=E[Y²]-2E[Y]E[c]+E[c²]],
is that what you meant by linearity to expand?
But how do you solve for c, really!

correction from my post: "how do you take derivative through operator"

 

[LCKurtz]

LCKurtz,
E[Y²-2Y*c+c²]=E[Y²]-2E[Y]E[c]+E[c²]],
is that what you meant by linearity to expand?
But how do you solve for c, really!

correction from my post: "how do you take derivative through operator"

Close, but the way you have written the middle term leads me to believe you don't quite understand. The expected value operation is linear means that if X and Y are random variables and c is a constant then

1. E(X + Y) = E(X) + E(Y)

and

2. E(cX) = cE(X)

Use those two properties (carefully) on E[Y²-2Y*c+c²].

And remember, E(Y²) and μ = E(Y) are just numbers. You are trying to minimize a function of c, just like you maximized and minimized functions of x in calculus.

 

[colstat]

Yes, I know about the expectation part wait, I still don't get it. Do I use derivative to do it?

 

[LCKurtz]

Yes, I know about the expectation part wait, I still don't get it. Do I use derivative to do it?

Show me the function of c that you got when you expanded it.

 

[colstat]

E[Y²-2Y*c+c²]
=E[Y²]-2E[Y]E[c]+E[c²]]
=E[Y²]-2cE[Y]+c²
=E[Y²]-2c$$\mu$$+c²

 

[statdad]

OK, so, if you have

$$E[(Y-\mu)^2] = E[Y^2-2c\mu + c^2$$

as a function of $c$, how would you minimize it?

 

[LCKurtz]

E[Y²-2Y*c+c²]
=E[Y²]-2E[Y]E[c]+E[c²]]
=E[Y²]-2cE[Y]+c²
=E[Y²]-2c$$\mu$$+c²

While I wouldn't quibble with the result, my objection to your expansion is your writing

E(-2Yc) = -2E(Y)E(c)

with no additional commentary. It looks like you are treating c as a random variable and using the "fact" that E(XY) = E(X)E(Y), which is not generally true and certainly not one of the two linearity properties. Perhaps you understand what you are doing at that step, but you haven't convinced me of it yet.

On an additional note, we have company for the next week so I am going to let StatDad take it from here if he is willing.

 

[colstat]

so, what do you do next?