- #1
squenshl
- 479
- 4
Homework Statement
Suppose that ##(Y_1,Y_2,\ldots,Y_n)## are random variables, where ##Y_i## has an exponential distribution with probability density function ##f_Y(y_i|\theta_i) = \theta_i e^{-\theta_i y_i}##, ##y_i > 0##, ##\theta_i > 0## where ##E(Y_i) = \frac{1}{\theta_i}## and ##\text{Var}(Y_i) = \frac{1}{\theta_i^2}##.
Suppose that ##\log{(\theta_i)} = \alpha+\beta(z_i-\bar{z})##, where the values ##z_i## are known for ##i=1,2,\ldots,n##.
We wish to estimate the vector parameter ##\theta = \begin{bmatrix}
\alpha \\
\beta
\end{bmatrix}##.
1. Show that the log-likelihood function
$$\ell(\theta) = n\alpha - \sum_{i=1}^{n} y_ie^{\alpha+\beta(z_i-\bar{z})}$$.
2. Find the two elements of the score statistic ##U(\theta;y)##.
Write down the two equations which must be solved to find the maximum likelihood estimates ##\hat{\alpha}## and ##\hat{\beta}##.
Note: Do not attempt to solve these equations.
3. Show that the information matrix, ##\begin{bmatrix}
n & 0 \\
0 & \sum_{i=1}^{n} (z_i-\bar{z})
\end{bmatrix}##.
Hint: Recall ##E(Y_i) = \frac{1}{\theta_i}## and ##\theta_i = e^{\alpha+\beta(z_i-\bar{z})}##.
4. Find the large sample variances of ##\hat{\alpha}## and ##\hat{\beta}## and show that they are asymptotically uncorrelated.
Homework Equations
The Attempt at a Solution
1. I think this result we are trying to prove is wrong. Firstly, we have
$$\begin{split}
\log{(f_{Y_i}(y_i|\theta_i))} &= \log{\left(\theta_i e^{-\theta_i y_i}\right)} \\
&= \log{(\theta_i)}-y_i\theta_i \\
&= \alpha+\beta(z_i-\bar{z})-y_i\theta_i.
\end{split}$$
Thus,
$$\begin{split}
\ell(\theta) &= \alpha+\beta(z_1-\bar{z})-y_1\theta_1+\alpha+\beta(z_2-\bar{z})-y_2\theta_2+\ldots+\alpha+\beta(z_n-\bar{z})-y_n\theta_n \\
&= \alpha+\beta(z_1-\bar{z})-y_1e^{\alpha+\beta(z_1-\bar{z})}+\alpha+\beta(z_2-\bar{z})-y_2e^{\alpha+\beta(z_2-\bar{z})}+\ldots+\alpha+\beta(z_n-\bar{z})-y_ne^{\alpha+\beta(z_i-\bar{z})} \\
&= n\alpha - \sum_{i=1}^{n} y_ie^{\alpha+\beta(z_i-\bar{z})} - \sum_{i=1}^{n} \beta(z_i-\bar{z}).
\end{split}$$
2. Firstly,
$$\begin{split}
\frac{d\ell}{d\alpha} &= n - \sum_{i=1}^{n} \left(y_ie^{\alpha+\beta(z_i-\bar{z})}\right) \\
&= n - \sum_{i=1}^{n} y_i\theta_i
\end{split}$$
and
$$\begin{split}
\frac{d\ell}{d\beta} &= -\sum_{i=1}^{n} (y_i(z_i-\bar{z})e^{\alpha+\beta(z_i-\bar{z})}) - \sum_{i=1}^{n} (z_i-\bar{z}) \\
&= -\sum_{i=1}^{n} (y_i\theta_i(z_i-\bar{z})) - \sum_{i=1}^{n} (z_i-\bar{z}).
\end{split}$$
So the score statistic is
$$U(\theta;y) = \begin{bmatrix}
n - \sum_{i=1}^{n} y_i\theta_i \\[1em]
-\sum_{i=1}^{n} (y_i\theta_i(z_i-\bar{z})) - \sum_{i=1}^{n} (z_i-\bar{z})
\end{bmatrix}$$
We find ##\hat{\alpha}## and ##\hat{\beta}## by setting ##U(\theta;y) = 0## and solving the two equations. This means solving ##n - \sum_{i=1}^{n} y_i\theta_i = 0## and ##\sum_{i=1}^{n} (y_i\theta_i(z_i-\bar{z})) + \sum_{i=1}^{n} (z_i-\bar{z}) = 0## for ##\hat{\alpha}## and ##\hat{\beta}## respectively.
4. I think this result we are trying to prove is wrong. Firstly, we calculate each partial derivative. Doing so gives
$$\begin{split}
\frac{\partial^2 \ell}{\partial \alpha^2} &= \frac{\partial}{\partial \alpha}\left(n - \sum_{i=1}^{n} y_i\theta_i\right) \\
&= -\sum_{i=1}^{n} y_i\theta_i,
\end{split}$$
$$\begin{split}
\frac{\partial^2 \ell}{\partial \beta \partial \alpha} &= \frac{\partial}{\partial \beta} \left(n - \sum_{i=1}^{n} y_i\theta_i\right) \\
&= -\sum_{i=1}^{n} y_i\theta_i(z_i-\bar{z}) \\
&= \frac{\partial^2 \ell}{\partial \alpha \partial \beta}
\end{split}$$
and
$$\begin{split}
\frac{\partial^2 \ell}{\partial \beta^2} &= \frac{\partial}{\partial \beta} \left(-\sum_{i=1}^{n} (y_i\theta_i(z_i-\bar{z})) - \sum_{i=1}^{n} (z_i-\bar{z})\right) \\
&= -\sum_{i=1}^{n} y_i\theta_i(z_i-\bar{z})^2.
\end{split}$$
Thus,
$$\begin{split}
I(\theta) &= -E\left(\begin{bmatrix}
\frac{\partial^2 \ell}{\partial \alpha^2} & \frac{\partial^2 \ell}{\partial \alpha \partial \beta} \\[1em]
\frac{\partial^2 \ell}{\partial \beta \partial \alpha} & \frac{\partial^2 \ell}{\partial \beta^2}
\end{bmatrix}\right) \\
&= E\left(\begin{bmatrix}
n & \sum_{i=1}^{n} (z_i-\bar{z}) \\[1em]
\sum_{i=1}^{n} (z_i-\bar{z}) & \sum_{i=1}^{n} (z_i-\bar{z})^2
\end{bmatrix}\right) \\
&= \begin{bmatrix}
n & \sum_{i=1}^{n} (z_i-\bar{z}) \\[1em]
\sum_{i=1}^{n} (z_i-\bar{z}) & \sum_{i=1}^{n} (z_i-\bar{z})^2
\end{bmatrix}.
\end{split}$$
I'm not sure if these are right. I reckon that the questions are wrong or is it me.
Please help!