Minimizing cost function Statistics

  • Thread starter colstat
  • Start date
  • #1
colstat
56
0

Homework Statement


Let X and Y be two unknown variables with E(Y)=[tex]\mu[/tex] and EY2 < [tex]\infty[/tex].


Homework Equations


a. Show that the constant c that minimizes E(Y-c)2 is c=[tex]\mu[/tex].
b. Deduce that the random variable f(X) that minimizes E[(Y-f(X))2|X] is f(X)= E[Y|X].
c. Deduce that the random variable f(X) that minimizes E[(Y-f(X))2] is also f(X)= E[Y|X].

The Attempt at a Solution


a. E(Y-c)2 = E[Y2-2Y*c+c2]
other ideas: Can I do the derivative w.r.t. Y, set it equal to zero. But there is an expectation operator, how do you take expectation through the operator.
I can't use a posterior mean, the problem did not specify distr.of the r.v.

b, c. no ideas yet
 
Last edited:

Answers and Replies

  • #2
LCKurtz
Science Advisor
Homework Helper
Insights Author
Gold Member
9,568
774

Homework Statement


Let X and Y be two unknown variables with E(Y)=[tex]\mu[/tex] and EY2 < [tex]\infty[/tex].


Homework Equations


a. Show that the constant c that minimizes E(Y-c)2 is c=[tex]\mu[/tex].
b. Deduce that the random variable f(X) that minimizes E[(Y-f(X))2|X] is f(X)= E[Y|X].
c. Deduce that the random variable f(X) that minimizes E[(Y-f(X))2] is also f(X)= E[Y|X].

The Attempt at a Solution


a. E(Y-c)2 = E[Y2-2Y*c+c2]
other ideas: Can I do the derivative w.r.t. Y, set it equal to zero. But there is an expectation operator, how do you take expectation through the operator.
I can't use a posterior mean, the problem did not specify distr.of the r.v.

b, c. no ideas yet

For (a) Remember, the variable here is c, not Y. Start by using linearity to expand
E[Y2-2Y*c+c2] and remember that.
 
  • #3
colstat
56
0
LCKurtz,
E[Y2-2Y*c+c2]=E[Y2]-2E[Y]E[c]+E[c2]],
is that what you meant by linearity to expand?
But how do you solve for c, really!

correction from my post: "how do you take derivative through operator"
 
  • #4
LCKurtz
Science Advisor
Homework Helper
Insights Author
Gold Member
9,568
774
LCKurtz,
E[Y2-2Y*c+c2]=E[Y2]-2E[Y]E[c]+E[c2]],
is that what you meant by linearity to expand?
But how do you solve for c, really!

correction from my post: "how do you take derivative through operator"

Close, but the way you have written the middle term leads me to believe you don't quite understand. The expected value operation is linear means that if X and Y are random variables and c is a constant then

1. E(X + Y) = E(X) + E(Y)

and

2. E(cX) = cE(X)

Use those two properties (carefully) on E[Y2-2Y*c+c2].

And remember, E(Y2) and μ = E(Y) are just numbers. You are trying to minimize a function of c, just like you maximized and minimized functions of x in calculus.
 
  • #5
colstat
56
0
Yes, I know about the expectation part wait, I still don't get it. Do I use derivative to do it?
 
  • #6
LCKurtz
Science Advisor
Homework Helper
Insights Author
Gold Member
9,568
774
Yes, I know about the expectation part wait, I still don't get it. Do I use derivative to do it?

Show me the function of c that you got when you expanded it.
 
  • #7
colstat
56
0
E[Y2-2Y*c+c2]
=E[Y2]-2E[Y]E[c]+E[c2]]
=E[Y2]-2cE[Y]+c2
=E[Y2]-2c[tex]\mu[/tex]+c2
 
  • #8
statdad
Homework Helper
1,495
36
OK, so, if you have

[tex]
E[(Y-\mu)^2] = E[Y^2-2c\mu + c^2
[/tex]

as a function of [itex] c [/itex], how would you minimize it?
 
  • #9
LCKurtz
Science Advisor
Homework Helper
Insights Author
Gold Member
9,568
774
E[Y2-2Y*c+c2]
=E[Y2]-2E[Y]E[c]+E[c2]]
=E[Y2]-2cE[Y]+c2
=E[Y2]-2c[tex]\mu[/tex]+c2

While I wouldn't quibble with the result, my objection to your expansion is your writing

E(-2Yc) = -2E(Y)E(c)

with no additional commentary. It looks like you are treating c as a random variable and using the "fact" that E(XY) = E(X)E(Y), which is not generally true and certainly not one of the two linearity properties. Perhaps you understand what you are doing at that step, but you haven't convinced me of it yet.

On an additional note, we have company for the next week so I am going to let StatDad take it from here if he is willing.
 
  • #10
colstat
56
0
so, what do you do next?
 

Suggested for: Minimizing cost function Statistics

  • Last Post
Replies
14
Views
318
  • Last Post
Replies
6
Views
719
Replies
5
Views
414
Replies
8
Views
330
Replies
4
Views
467
Replies
1
Views
108
  • Last Post
Replies
18
Views
1K
  • Last Post
Replies
3
Views
510
Top