Minimum mean square error for two random variables

  • Thread starter Dustinsfl
  • Start date
  • #1
699
5

Homework Statement


Determine the minimum mean square error for the joint PMF. You will need to evaluate ##E_{X, Y}[(Y - 14/11\cdot X - 1/11)^2]##.
oyrcfIM.png


Homework Equations




The Attempt at a Solution


The answer is ##\frac{3}{22}##, but when I work it out, I get ##\frac{203}{484}##. From my values, I have the same expected values, covariance, and variance of X as the book. Therefore, since my numbers are on point, I cant figure out why my solution is so off.

The ##mse(a, b) = E_{X, Y}[(Y - \hat{Y})^2] = E_{X, Y}[(Y - aX - B)^2]## since ##\hat{Y} = aX + b##. Expanding out ##mse(a, b)## and taking the derivatives of ##\frac{\partial mse(a, b)}{\partial a} = 0## and ##\frac{\partial mse(a, b)}{\partial b} = 0##, we get that
\begin{alignat*}{2}
a_{opt} &= \frac{cov(X, Y)}{var(X)} &&={} \frac{14}{11}\\
b_{opt} &= E[Y] - \frac{cov(X, Y)}{var(X)}E[X] &&={} \frac{1}{11}
\end{alignat*}
Then the ##mse(a, b)## is
\begin{align*}
mse(a, b) &= E[Y^2] - 2aE[XY] + a^2E[X^2] - 2bE[Y] + 2abE[X] + b^2\\
&= E[Y^2] - \frac{28}{11}E[XY] + \frac{196}{121}E[X^2] -
\frac{2}{11}E[Y] + \frac{28}{121}E[X] + \frac{1}{121}
\end{align*}
In order to find ##mse(a, b)##, we need to know the expected value of ##X##, ##Y##, ##E[X^2]##, and ##E[X^2]## as well as ##E[XY]##.
\begin{alignat*}{4}
E[X] &= \sum_ix_ip_X[x_i] &\quad &
E[Y] &&={} \sum_jy_jp_Y[y_j]\\
&= 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{2} & &
&&={} 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{4} + 3\frac{1}{4}\\
&= \frac{5}{4} & &
&&={} \frac{3}{2}\\
E[X^2] &= \sum_ix_i^2p_X[x_i] & &
E[Y^2] &&={} \sum_jy_j^2p_Y[y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{2} & &
&&={} 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} + 3^2\frac{1}{4}\\
&= \frac{9}{4} & &
&&={} \frac{15}{4}\\
E[XY] &= \sum_{i, j}x_iy_jp_{X, Y}[x_i, y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} +
2\cdot 3\frac{1}{4}\\
&= \frac{11}{4}
\end{alignat*}
Now, that we have remaining components, we can obtain ##mse(a, b)##.
\begin{align*}
&= \frac{15}{4} - \frac{28}{11}\frac{11}{4} +
\frac{196}{121}\frac{9}{4} - \frac{2}{11}\frac{3}{2} +
\frac{28}{121}\frac{5}{4} + \frac{1}{121}\\
&= \frac{203}{484}
\end{align*}
 
Last edited:

Answers and Replies

  • #2
Ray Vickson
Science Advisor
Homework Helper
Dearly Missed
10,706
1,722

Homework Statement


Determine the minimum mean square error for the joint PMF. You will need to evaluate ##E_{X, Y}[(Y - 14/11\cdot X - 1/11)^2]##.

Homework Equations




The Attempt at a Solution


The answer is ##\frac{3}{22}##, but when I work it out, I get ##\frac{203}{242}##. From my values, I have the same expected values, covariance, and variance of X as the book. Therefore, since my numbers are on point, I cant figure out why my number is so off.

The ##mse(a, b) = E_{X, Y}[(Y - \hat{Y})^2] = E_{X, Y}[(Y - aX - B)^2]## since ##\hat{Y} = aX + b##. Expanding out ##mse(a, b)## and taking the derivatives of ##\frac{\partial mse(a, b)}{\partial a} = 0## and ##\frac{\partial mse(a, b)}{\partial b} = 0##, we get that
\begin{alignat*}{2}
a_{opt} &= \frac{cov(X, Y)}{var(X)} &&={} \frac{14}{11}\\
b_{opt} &= E[Y] - \frac{cov(X, Y)}{var(X)}E[X] &&={} \frac{1}{11}
\end{alignat*}
Then the ##mse(a, b)## is
\begin{align*}
mse(a, b) &= E[Y^2] - 2aE[XY] + a^2E[X^2] - 2bE[Y] + 2abE[X] + b^2\\
&= E[Y^2] - \frac{28}{11}E[XY] + \frac{196}{121}E[X^2] -
\frac{2}{11}E[Y] + \frac{28}{121}E[X] + \frac{1}{121}
\end{align*}
In order to find ##mse(a, b)##, we need to know the expected value of ##X##, ##Y##, ##E[X^2]##, and ##E[X^2]## as well as ##E[XY]##.
\begin{alignat*}{4}
E[X] &= \sum_ix_ip_X[x_i] &\quad &
E[Y] &&={} \sum_jy_jp_Y[y_j]\\
&= 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{2} & &
&&={} 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{4} + 3\frac{1}{4}\\
&= \frac{5}{4} & &
&&={} \frac{3}{2}\\
E[X^2] &= \sum_ix_i^2p_X[x_i] & &
E[Y^2] &&={} \sum_jy_j^2p_Y[y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{2} & &
&&={} 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} + 3^2\frac{1}{4}\\
&= \frac{9}{4} & &
&&={} \frac{15}{4}\\
E[XY] &= \sum_{i, j}x_iy_jp_{X, Y}[x_i, y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} +
2\cdot 3\frac{1}{4}\\
&= \frac{11}{4}
\end{alignat*}
Now, that we have remaining components, we can obtain ##mse(a, b)##.Y - (14/11)X -
\begin{align*}
&= \frac{15}{4} - \frac{28}{11}\frac{11}{4} +
\frac{196}{121}\frac{9}{4} - \frac{2}{11}\frac{3}{2} +
\frac{28}{121}\frac{5}{4} + \frac{1}{121}\\
&= \frac{203}{242}
\end{align*}

I do not fully understand the notation ##(Y - 14/11\cdot X - 1/11)^2##. Do you mean
[tex] \left( Y - \frac{14}{11} X - \frac{1}{11} \right)^2 ? [/tex]
If so, use the "\frac" command in TeX. If you mean something else, try using appropriate parentheses. Or, you could/should even use parentheses without the "\frac" command---for example, write ##(Y - (14/11)X - 1/11)^2## if that is what you mean. Alternatively, you could write ##(Y - aX - b)^2,## where ##a = 14/11, \, b = 1/11##. Personally, I like this last way the best, as it helps keep different factors and influences separate.
 
  • #3
699
5
I do not fully understand the notation ##(Y - 14/11\cdot X - 1/11)^2##. Do you mean
[tex] \left( Y - \frac{14}{11} X - \frac{1}{11} \right)^2 ? [/tex]
If so, use the "\frac" command in TeX. If you mean something else, try using appropriate parentheses. Or, you could/should even use parentheses without the "\frac" command---for example, write ##(Y - (14/11)X - 1/11)^2## if that is what you mean. Alternatively, you could write ##(Y - aX - b)^2,## where ##a = 14/11, \, b = 1/11##. Personally, I like this last way the best, as it helps keep different factors and influences separate.

I have no idea what to say if you dont understand the meaning of ##(Y - 14/11\cdot X - 1/11)^2##. I am not trying to be rude, but I don't need an education in LaTeX.
 
  • #4
34,982
6,734
I have no idea what to say if you dont understand the meaning of ##(Y - 14/11\cdot X - 1/11)^2##. I am not trying to be rude, but I don't need an education in LaTeX.
Dustin, I'm pretty sure what you wrote is exactly what you intended to write. The reason for Ray's comment is that we get so many people who write x - 2/y - 3 when they really mean (x - 2)/(y - 3).
 
  • #5
699
5
Dustin, I'm pretty sure what you wrote is exactly what you intended to write. The reason for Ray's comment is that we get so many people who write x - 2/y - 3 when they really mean (x - 2)/(y - 3).

Even if one is unsure, look past the question into the attempt. If that was done, one would see the proper definition Y - aX - B and realize X is not in the denominator. Critiquing a question in which you didn't fully read is pointless since the reader was premature in their analysis when the clarity was already present and verified later.
 
  • #6
haruspex
Science Advisor
Homework Helper
Insights Author
Gold Member
2020 Award
36,173
6,792
I get a different number for E[Y2], but it still doesn't get me to 3/22.
 
  • #7
699
5
I get a different number for E[Y2], but it still doesn't get me to 3/22.

Did you get 7/2 and then 41/242 for the mean square error? I see my addition error in E[Y^2].
 
  • #8
LCKurtz
Science Advisor
Homework Helper
Insights Author
Gold Member
9,559
770
I have no idea what to say if you dont understand the meaning of ##(Y - 14/11\cdot X - 1/11)^2##. I am not trying to be rude, but I don't need an education in LaTeX.

Even if one is unsure, look past the question into the attempt. If that was done, one would see the proper definition Y - aX - B and realize X is not in the denominator. Critiquing a question in which you didn't fully read is pointless since the reader was premature in their analysis when the clarity was already present and verified later.

Sorry, but I agree with Ray here. It isn't our job to look past what is written to see if we can figure out what was meant. When posters neglect needed parentheses we suggest they use them just as we make other suggestions to help posters solve their problems or how to write their solutions up properly. Often their misuse of parentheses is the cause of later algebra mistakes. Instead of telling us you don't need an education in LaTeX perhaps you could say something like "Thanks, I will try to be more careful." If you aren't "trying to be rude", then don't be rude.
 
  • #9
699
5
Sorry, but I agree with Ray here. It isn't our job to look past what is written to see if we can figure out what was meant. When posters neglect needed parentheses we suggest they use them just as we make other suggestions to help posters solve their problems or how to write their solutions up properly. Often their misuse of parentheses is the cause of later algebra mistakes. Instead of telling us you don't need an education in LaTeX perhaps you could say something like "Thanks, I will try to be more careful." If you aren't "trying to be rude", then don't be rude.

For me, this is the 2nd time Ray hasn't read and made a mistake.

On this post: https://www.physicsforums.com/threads/cov-w-z-where-w-x-and-z-ax-y.775444/ he mentioned and identity to use that I already listed. So he has a recorded with me of not reading the question in full and making a suggestion mistake.

Additionally, I know how to format my questions and wouldnt make the error of saying times when I want to divide that as a quantity. I wasn't being careless and didnt need to be more careful since that format is what I meant and is correct.
 
  • #10
haruspex
Science Advisor
Homework Helper
Insights Author
Gold Member
2020 Award
36,173
6,792
I get a different number for E[Y2], but it still doesn't get me to 3/22.
The other error is in your calculation of B.
As you probably realise, it is very easy to get the 3/22 result once you have A and B. Just calculate Y errors.
 
  • #11
699
5
The other error is in your calculation of B.
As you probably realise, it is very easy to get the 3/22 result once you have A and B. Just calculate Y errors.

Thanks sign error. All is good now.
 

Related Threads on Minimum mean square error for two random variables

  • Last Post
Replies
2
Views
592
Replies
13
Views
2K
  • Last Post
Replies
0
Views
2K
Replies
2
Views
828
Replies
1
Views
943
  • Last Post
Replies
16
Views
868
  • Last Post
Replies
5
Views
1K
  • Last Post
Replies
7
Views
1K
Replies
10
Views
385
M
Top