Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Bias of an estimator: Can you confirm that I am doing this right?

  1. Aug 11, 2013 #1
    Let [itex]X_{1}, \ldots, X_{n} \; \mathtt{\sim} \; \textrm{Poisson} (\lambda)[/itex] and let [itex]\hat{\lambda} = n^{-1} \sum_{i = 1}^{n} X_{i}[/itex].

    The bias of [itex]\hat{\lambda}[/itex] is [itex]\mathbb{E}_{\lambda} (\hat{\lambda}) - \lambda[/itex]. Since [itex]X_{i} \; \mathtt{\sim} \; \textrm{Poisson} (\lambda)[/itex], and all [itex]X_{i}[/itex] are IID, [itex]\sum_{i = 1}^{n} X_{i} \; \mathtt{\sim} \; \textrm{Poisson} (n \lambda)[/itex].

    Thus, [itex]\mathbb{E} (\hat{\lambda}) = \sum_{nx = 1}^{\infty} x \exp{(-n \lambda)} \frac{(n \lambda)^{nx}}{(nx)!} = \lambda[/itex], and the indicator is unbiased (bias = 0).

    However, I'm using [itex]\mathbb{E}_{\lambda}[/itex] as [itex]\mathbb{E}[/itex], and I don't know if I'm doing it right. I haven't seen any similar examples and this is the first time I'm calculating the bias, so I would really love some insight.
  2. jcsd
  3. Aug 11, 2013 #2


    User Avatar
    Science Advisor

    For any distribution where the mean exists (including Poisson), an average of trials is always an unbiased estimate of the mean. All you need is the law of large numbers.
  4. Aug 11, 2013 #3
    Thanks. However, I still want to know if I calculated this correctly (as I will be doing the same for calculating the standard error and MSE).
  5. Aug 11, 2013 #4

    Stephen Tashi

    User Avatar
    Science Advisor

    What does "using [itex]\mathbb{E}_{\lambda}[/itex] as [itex] \mathbb{E} [/itex]" mean? For the expectation operator to have a definite meaning, you must say what variable [itex] \mathbb{E} [/itex] is being applied to.
  6. Aug 11, 2013 #5
    The textbook defines [itex]E_{\theta} \left( r(X) \right) = \int r(x) f(x; \theta) dx[/itex].

    What I did is just evaluated the expectation of [itex]\hat{\lambda}[/itex].
  7. Aug 11, 2013 #6

    Stephen Tashi

    User Avatar
    Science Advisor

    One would also need to know how the textbook defines the various things involved in that expression. To me that looks like some sort of conditional expectation where the condition is given by the value of the parameter [itex] \theta [/itex] used in the probability density [itex] f [/itex].

    In contrast to that, the notation [itex] E_X Y [/itex] often means "the expected value of the function [itex] Y [/itex] with respect to the random variable [itex] X [/itex]. If the probability density of [itex] X [/itex] is [itex] f(x) [/itex] then this notation means [itex] E_X Y = \int Y(x) f(x) dx [/itex].

    To relate the above notation to your work

    [itex] X = Y = \hat{\lambda}[/itex]
    The possible values of [itex] X [/itex] are denoted by [itex] nx [/itex].
    The probability density of [itex] X [/itex] is [itex] f(nx) = e^{-n\lambda} \frac{ (n \lambda)^{nx}}{(nx)!} [/itex]

    Taking the usual view that a sum is a type of integral, you should compute
    [itex] E_X Y = \int nx\ f(nx) dx = \sum_{nx=0}^\infty nx\ e^{-n\lambda} \frac{ (n \lambda)^{nx}}{(nx)!} [/itex]

    If you did not know the probability density function for [itex] \hat{\lambda} [/itex] then could have used the theorem that the expected value of a sum of random variables is the sum of their expected values and gotten the result in a less direct way.
  8. Aug 13, 2013 #7


    User Avatar
    Homework Helper

    This notation (the [itex] E_{\theta}[\text{ something }][/itex]) is often used when the assumption is the family of distributions is indexed by a (real or vector valued) parameter [itex] \theta [/itex]. In that context there is no possibility of interpreting as a conditional expectation.
    Last edited: Aug 13, 2013
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook