# Minimum mean square error for two random variables

1. Oct 11, 2014

### Dustinsfl

1. The problem statement, all variables and given/known data
Determine the minimum mean square error for the joint PMF. You will need to evaluate $E_{X, Y}[(Y - 14/11\cdot X - 1/11)^2]$.

2. Relevant equations

3. The attempt at a solution
The answer is $\frac{3}{22}$, but when I work it out, I get $\frac{203}{484}$. From my values, I have the same expected values, covariance, and variance of X as the book. Therefore, since my numbers are on point, I cant figure out why my solution is so off.

The $mse(a, b) = E_{X, Y}[(Y - \hat{Y})^2] = E_{X, Y}[(Y - aX - B)^2]$ since $\hat{Y} = aX + b$. Expanding out $mse(a, b)$ and taking the derivatives of $\frac{\partial mse(a, b)}{\partial a} = 0$ and $\frac{\partial mse(a, b)}{\partial b} = 0$, we get that
\begin{alignat*}{2}
a_{opt} &= \frac{cov(X, Y)}{var(X)} &&={} \frac{14}{11}\\
b_{opt} &= E[Y] - \frac{cov(X, Y)}{var(X)}E[X] &&={} \frac{1}{11}
\end{alignat*}
Then the $mse(a, b)$ is
\begin{align*}
mse(a, b) &= E[Y^2] - 2aE[XY] + a^2E[X^2] - 2bE[Y] + 2abE[X] + b^2\\
&= E[Y^2] - \frac{28}{11}E[XY] + \frac{196}{121}E[X^2] -
\frac{2}{11}E[Y] + \frac{28}{121}E[X] + \frac{1}{121}
\end{align*}
In order to find $mse(a, b)$, we need to know the expected value of $X$, $Y$, $E[X^2]$, and $E[X^2]$ as well as $E[XY]$.
\begin{alignat*}{4}
E[Y] &&={} \sum_jy_jp_Y[y_j]\\
&= 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{2} & &
&&={} 0\frac{1}{4} + 1\frac{1}{4} + 2\frac{1}{4} + 3\frac{1}{4}\\
&= \frac{5}{4} & &
&&={} \frac{3}{2}\\
E[X^2] &= \sum_ix_i^2p_X[x_i] & &
E[Y^2] &&={} \sum_jy_j^2p_Y[y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{2} & &
&&={} 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} + 3^2\frac{1}{4}\\
&= \frac{9}{4} & &
&&={} \frac{15}{4}\\
E[XY] &= \sum_{i, j}x_iy_jp_{X, Y}[x_i, y_j]\\
&= 0^2\frac{1}{4} + 1^2\frac{1}{4} + 2^2\frac{1}{4} +
2\cdot 3\frac{1}{4}\\
&= \frac{11}{4}
\end{alignat*}
Now, that we have remaining components, we can obtain $mse(a, b)$.
\begin{align*}
&= \frac{15}{4} - \frac{28}{11}\frac{11}{4} +
\frac{196}{121}\frac{9}{4} - \frac{2}{11}\frac{3}{2} +
\frac{28}{121}\frac{5}{4} + \frac{1}{121}\\
&= \frac{203}{484}
\end{align*}

Last edited: Oct 11, 2014
2. Oct 11, 2014

### Ray Vickson

I do not fully understand the notation $(Y - 14/11\cdot X - 1/11)^2$. Do you mean
$$\left( Y - \frac{14}{11} X - \frac{1}{11} \right)^2 ?$$
If so, use the "\frac" command in TeX. If you mean something else, try using appropriate parentheses. Or, you could/should even use parentheses without the "\frac" command---for example, write $(Y - (14/11)X - 1/11)^2$ if that is what you mean. Alternatively, you could write $(Y - aX - b)^2,$ where $a = 14/11, \, b = 1/11$. Personally, I like this last way the best, as it helps keep different factors and influences separate.

3. Oct 11, 2014

### Dustinsfl

I have no idea what to say if you dont understand the meaning of $(Y - 14/11\cdot X - 1/11)^2$. I am not trying to be rude, but I don't need an education in LaTeX.

4. Oct 11, 2014

### Staff: Mentor

Dustin, I'm pretty sure what you wrote is exactly what you intended to write. The reason for Ray's comment is that we get so many people who write x - 2/y - 3 when they really mean (x - 2)/(y - 3).

5. Oct 11, 2014

### Dustinsfl

Even if one is unsure, look past the question into the attempt. If that was done, one would see the proper definition Y - aX - B and realize X is not in the denominator. Critiquing a question in which you didn't fully read is pointless since the reader was premature in their analysis when the clarity was already present and verified later.

6. Oct 11, 2014

### haruspex

I get a different number for E[Y2], but it still doesn't get me to 3/22.

7. Oct 11, 2014

### Dustinsfl

Did you get 7/2 and then 41/242 for the mean square error? I see my addition error in E[Y^2].

8. Oct 11, 2014

### LCKurtz

Sorry, but I agree with Ray here. It isn't our job to look past what is written to see if we can figure out what was meant. When posters neglect needed parentheses we suggest they use them just as we make other suggestions to help posters solve their problems or how to write their solutions up properly. Often their misuse of parentheses is the cause of later algebra mistakes. Instead of telling us you don't need an education in LaTeX perhaps you could say something like "Thanks, I will try to be more careful." If you aren't "trying to be rude", then don't be rude.

9. Oct 11, 2014

### Dustinsfl

For me, this is the 2nd time Ray hasn't read and made a mistake.

On this post: https://www.physicsforums.com/threads/cov-w-z-where-w-x-and-z-ax-y.775444/ he mentioned and identity to use that I already listed. So he has a recorded with me of not reading the question in full and making a suggestion mistake.

Additionally, I know how to format my questions and wouldnt make the error of saying times when I want to divide that as a quantity. I wasn't being careless and didnt need to be more careful since that format is what I meant and is correct.

10. Oct 12, 2014

### haruspex

The other error is in your calculation of B.
As you probably realise, it is very easy to get the 3/22 result once you have A and B. Just calculate Y errors.

11. Oct 12, 2014

### Dustinsfl

Thanks sign error. All is good now.