Minimizing cost function Statistics

  • Thread starter colstat
  • Start date
  • #1
56
0

Homework Statement


Let X and Y be two unknown variables with E(Y)=[tex]\mu[/tex] and EY2 < [tex]\infty[/tex].


Homework Equations


a. Show that the constant c that minimizes E(Y-c)2 is c=[tex]\mu[/tex].
b. Deduce that the random variable f(X) that minimizes E[(Y-f(X))2|X] is f(X)= E[Y|X].
c. Deduce that the random variable f(X) that minimizes E[(Y-f(X))2] is also f(X)= E[Y|X].

The Attempt at a Solution


a. E(Y-c)2 = E[Y2-2Y*c+c2]
other ideas: Can I do the derivative w.r.t. Y, set it equal to zero. But there is an expectation operator, how do you take expectation through the operator.
I can't use a posterior mean, the problem did not specify distr.of the r.v.

b, c. no ideas yet
 
Last edited:

Answers and Replies

  • #2
LCKurtz
Science Advisor
Homework Helper
Insights Author
Gold Member
9,557
767

Homework Statement


Let X and Y be two unknown variables with E(Y)=[tex]\mu[/tex] and EY2 < [tex]\infty[/tex].


Homework Equations


a. Show that the constant c that minimizes E(Y-c)2 is c=[tex]\mu[/tex].
b. Deduce that the random variable f(X) that minimizes E[(Y-f(X))2|X] is f(X)= E[Y|X].
c. Deduce that the random variable f(X) that minimizes E[(Y-f(X))2] is also f(X)= E[Y|X].

The Attempt at a Solution


a. E(Y-c)2 = E[Y2-2Y*c+c2]
other ideas: Can I do the derivative w.r.t. Y, set it equal to zero. But there is an expectation operator, how do you take expectation through the operator.
I can't use a posterior mean, the problem did not specify distr.of the r.v.

b, c. no ideas yet

For (a) Remember, the variable here is c, not Y. Start by using linearity to expand
E[Y2-2Y*c+c2] and remember that.
 
  • #3
56
0
LCKurtz,
E[Y2-2Y*c+c2]=E[Y2]-2E[Y]E[c]+E[c2]],
is that what you meant by linearity to expand?
But how do you solve for c, really!

correction from my post: "how do you take derivative through operator"
 
  • #4
LCKurtz
Science Advisor
Homework Helper
Insights Author
Gold Member
9,557
767
LCKurtz,
E[Y2-2Y*c+c2]=E[Y2]-2E[Y]E[c]+E[c2]],
is that what you meant by linearity to expand?
But how do you solve for c, really!

correction from my post: "how do you take derivative through operator"

Close, but the way you have written the middle term leads me to believe you don't quite understand. The expected value operation is linear means that if X and Y are random variables and c is a constant then

1. E(X + Y) = E(X) + E(Y)

and

2. E(cX) = cE(X)

Use those two properties (carefully) on E[Y2-2Y*c+c2].

And remember, E(Y2) and μ = E(Y) are just numbers. You are trying to minimize a function of c, just like you maximized and minimized functions of x in calculus.
 
  • #5
56
0
Yes, I know about the expectation part wait, I still don't get it. Do I use derivative to do it?
 
  • #6
LCKurtz
Science Advisor
Homework Helper
Insights Author
Gold Member
9,557
767
Yes, I know about the expectation part wait, I still don't get it. Do I use derivative to do it?

Show me the function of c that you got when you expanded it.
 
  • #7
56
0
E[Y2-2Y*c+c2]
=E[Y2]-2E[Y]E[c]+E[c2]]
=E[Y2]-2cE[Y]+c2
=E[Y2]-2c[tex]\mu[/tex]+c2
 
  • #8
statdad
Homework Helper
1,495
35
OK, so, if you have

[tex]
E[(Y-\mu)^2] = E[Y^2-2c\mu + c^2
[/tex]

as a function of [itex] c [/itex], how would you minimize it?
 
  • #9
LCKurtz
Science Advisor
Homework Helper
Insights Author
Gold Member
9,557
767
E[Y2-2Y*c+c2]
=E[Y2]-2E[Y]E[c]+E[c2]]
=E[Y2]-2cE[Y]+c2
=E[Y2]-2c[tex]\mu[/tex]+c2

While I wouldn't quibble with the result, my objection to your expansion is your writing

E(-2Yc) = -2E(Y)E(c)

with no additional commentary. It looks like you are treating c as a random variable and using the "fact" that E(XY) = E(X)E(Y), which is not generally true and certainly not one of the two linearity properties. Perhaps you understand what you are doing at that step, but you haven't convinced me of it yet.

On an additional note, we have company for the next week so I am going to let StatDad take it from here if he is willing.
 
  • #10
56
0
so, what do you do next?
 

Related Threads on Minimizing cost function Statistics

  • Last Post
Replies
5
Views
4K
Replies
0
Views
2K
  • Last Post
Replies
5
Views
4K
  • Last Post
Replies
1
Views
3K
  • Last Post
Replies
2
Views
994
Replies
3
Views
5K
Replies
1
Views
2K
  • Last Post
Replies
1
Views
10K
Replies
1
Views
6K
Top