MSE and Estimators for Random Samples

[SavvyAA3]

I would very much appreciate if someone could explain the following:

- What is the use of the MSE (Mean Square Error) i.e. why do we use it?
I understand that MSE(t) = Var(t) + {E(t-&)}^2, but what does this tell us?

- Why/ How does E{A*Sx^2 +b*Sy^2} = a*Var + b*Var

(I am using ^ to denote 'powers' i.e. '^2' means sqaured)

- What is the E(-) operator, what is its use and what are its properties.

I would really, really appreciate help on this as I am having difficulties grasping these concepts of statistical theory.

Thanks

 

[EnumaElish]

What is the use of the MSE (Mean Square Error) i.e. why do we use it?

http://en.wikipedia.org/wiki/Mean_squared_error

What is the E(-) operator, what is its use and what are its properties.

http://en.wikipedia.org/wiki/Expected_value

Why/ How does E{A*Sx^2 +b*Sy^2} = a*Var + b*Var

E{a*Sx^2 +b*Sy^2} = a*E{Sx^2} + b*E{Sy^2} = a Var[x] + b Var[y], assuming "S^2" is an unbiased estimator of Var.

 

[SavvyAA3]

Thanks for the above, I didn't realize the E(-) operator was simply Expected Values of Linear Functions and Random Variables.

For the last part, I am still a little confused. Are you wishing to infer that the expected value of an unbiased estimator is simply its variance?

 

[EnumaElish]

Are you wishing to infer that the expected value of an unbiased estimator is simply its variance?

No, not its variance.

The term variance can refer to either the true variance or an estimated variance, which can be confusing.

The true variance is a population parameter. Just like the population mean, true variance is "hard-wired" into the behavior of the random variable, so to speak. Typically it is an unknown, and it can be estimated from a sample (just like the true mean can be estimated).

Suppose I "invent" a statistic called "EnumaElish variance estimator" (EEVE), as a function of the sample. You give me a sample, EEVE gives you an estimate (of the true variance in the population, based on the sample). If EEVE is an unbiased estimator of the true variance, then its expected value is equal to the population parameter: E[EEVE] = True Variance.

This is different than saying "let Z be any statistic (of the sample). (E.g., Z might be the sample average.) Then Z has its own variance due to the sample variance, which is what you were thinking when you posted the above quotation, I guess. The variance of Z is an estimated variance (like EEVE) and not the true variance of the population.

To sum up, I wasn't proposing E[Z] = Var[Z] for any statistic Z; but this: for a random variable X, if S^2[X] is an unbiased estimator of X's true variance, then E[S^2[X]] = Var[X], which is the true variance of random variable X.

 

[SavvyAA3]

Erm Ok, so you are suggesting that considering random variables, the expected value of an unbiased estimator (used in a sample to estimate the true population parameter, such as the unknown mean or variance etc) has an expected value equal to its true population parameter?

So with what you have given above, does this mean, if I have a sample and have only a sample mean and wish to find the mean of the true population, I can take its expected value and the result that I get will be the true population mean?

Sorry for asking this but I have still not yet quite understood this.

Thanks

 

[EnumaElish]

As a student, I had difficulty understanding the whole expectation concept so you're not alone.

If I have only one sample (which is often what anybody can hope to have), and I compute its average, then that number is my single "best" unbiased estimator of the true mean (which is unknown, and remains unknown, unless either:

1. someone measures the entire population and computes its mean, or:

2. you are working with a theoretical distribution which comes with an assumed true mean (e.g., assume the sample is from a standard normal distribution; then the true mean is given as 0).

Although it may not be exactly equal to the true mean, I know that it is unbiased, so I am not making an error that is decidedly "one way or the other."