# Bivariate random variables

## Homework Statement

Suppose that ##(Y_1,Y_2,\ldots,Y_n)## are random variables, where ##Y_i## has an exponential distribution with probability density function ##f_Y(y_i|\theta_i) = \theta_i e^{-\theta_i y_i}##, ##y_i > 0##, ##\theta_i > 0## where ##E(Y_i) = \frac{1}{\theta_i}## and ##\text{Var}(Y_i) = \frac{1}{\theta_i^2}##.
Suppose that ##\log{(\theta_i)} = \alpha+\beta(z_i-\bar{z})##, where the values ##z_i## are known for ##i=1,2,\ldots,n##.
We wish to estimate the vector parameter ##\theta = \begin{bmatrix}
\alpha \\
\beta
\end{bmatrix}##.
1. Show that the log-likelihood function
$$\ell(\theta) = n\alpha - \sum_{i=1}^{n} y_ie^{\alpha+\beta(z_i-\bar{z})}$$.

2. Find the two elements of the score statistic ##U(\theta;y)##.
Write down the two equations which must be solved to find the maximum likelihood estimates ##\hat{\alpha}## and ##\hat{\beta}##.
Note: Do not attempt to solve these equations.

3. Show that the information matrix, ##\begin{bmatrix}
n & 0 \\
0 & \sum_{i=1}^{n} (z_i-\bar{z})
\end{bmatrix}##.
Hint: Recall ##E(Y_i) = \frac{1}{\theta_i}## and ##\theta_i = e^{\alpha+\beta(z_i-\bar{z})}##.

4. Find the large sample variances of ##\hat{\alpha}## and ##\hat{\beta}## and show that they are asymptotically uncorrelated.

## The Attempt at a Solution

1. I think this result we are trying to prove is wrong. Firstly, we have
$$\begin{split} \log{(f_{Y_i}(y_i|\theta_i))} &= \log{\left(\theta_i e^{-\theta_i y_i}\right)} \\ &= \log{(\theta_i)}-y_i\theta_i \\ &= \alpha+\beta(z_i-\bar{z})-y_i\theta_i. \end{split}$$
Thus,
$$\begin{split} \ell(\theta) &= \alpha+\beta(z_1-\bar{z})-y_1\theta_1+\alpha+\beta(z_2-\bar{z})-y_2\theta_2+\ldots+\alpha+\beta(z_n-\bar{z})-y_n\theta_n \\ &= \alpha+\beta(z_1-\bar{z})-y_1e^{\alpha+\beta(z_1-\bar{z})}+\alpha+\beta(z_2-\bar{z})-y_2e^{\alpha+\beta(z_2-\bar{z})}+\ldots+\alpha+\beta(z_n-\bar{z})-y_ne^{\alpha+\beta(z_i-\bar{z})} \\ &= n\alpha - \sum_{i=1}^{n} y_ie^{\alpha+\beta(z_i-\bar{z})} - \sum_{i=1}^{n} \beta(z_i-\bar{z}). \end{split}$$

2. Firstly,
$$\begin{split} \frac{d\ell}{d\alpha} &= n - \sum_{i=1}^{n} \left(y_ie^{\alpha+\beta(z_i-\bar{z})}\right) \\ &= n - \sum_{i=1}^{n} y_i\theta_i \end{split}$$
and
$$\begin{split} \frac{d\ell}{d\beta} &= -\sum_{i=1}^{n} (y_i(z_i-\bar{z})e^{\alpha+\beta(z_i-\bar{z})}) - \sum_{i=1}^{n} (z_i-\bar{z}) \\ &= -\sum_{i=1}^{n} (y_i\theta_i(z_i-\bar{z})) - \sum_{i=1}^{n} (z_i-\bar{z}). \end{split}$$
So the score statistic is
$$U(\theta;y) = \begin{bmatrix} n - \sum_{i=1}^{n} y_i\theta_i \\[1em] -\sum_{i=1}^{n} (y_i\theta_i(z_i-\bar{z})) - \sum_{i=1}^{n} (z_i-\bar{z}) \end{bmatrix}$$
We find ##\hat{\alpha}## and ##\hat{\beta}## by setting ##U(\theta;y) = 0## and solving the two equations. This means solving ##n - \sum_{i=1}^{n} y_i\theta_i = 0## and ##\sum_{i=1}^{n} (y_i\theta_i(z_i-\bar{z})) + \sum_{i=1}^{n} (z_i-\bar{z}) = 0## for ##\hat{\alpha}## and ##\hat{\beta}## respectively.

4. I think this result we are trying to prove is wrong. Firstly, we calculate each partial derivative. Doing so gives
$$\begin{split} \frac{\partial^2 \ell}{\partial \alpha^2} &= \frac{\partial}{\partial \alpha}\left(n - \sum_{i=1}^{n} y_i\theta_i\right) \\ &= -\sum_{i=1}^{n} y_i\theta_i, \end{split}$$
$$\begin{split} \frac{\partial^2 \ell}{\partial \beta \partial \alpha} &= \frac{\partial}{\partial \beta} \left(n - \sum_{i=1}^{n} y_i\theta_i\right) \\ &= -\sum_{i=1}^{n} y_i\theta_i(z_i-\bar{z}) \\ &= \frac{\partial^2 \ell}{\partial \alpha \partial \beta} \end{split}$$
and
$$\begin{split} \frac{\partial^2 \ell}{\partial \beta^2} &= \frac{\partial}{\partial \beta} \left(-\sum_{i=1}^{n} (y_i\theta_i(z_i-\bar{z})) - \sum_{i=1}^{n} (z_i-\bar{z})\right) \\ &= -\sum_{i=1}^{n} y_i\theta_i(z_i-\bar{z})^2. \end{split}$$
Thus,
$$\begin{split} I(\theta) &= -E\left(\begin{bmatrix} \frac{\partial^2 \ell}{\partial \alpha^2} & \frac{\partial^2 \ell}{\partial \alpha \partial \beta} \\[1em] \frac{\partial^2 \ell}{\partial \beta \partial \alpha} & \frac{\partial^2 \ell}{\partial \beta^2} \end{bmatrix}\right) \\ &= E\left(\begin{bmatrix} n & \sum_{i=1}^{n} (z_i-\bar{z}) \\[1em] \sum_{i=1}^{n} (z_i-\bar{z}) & \sum_{i=1}^{n} (z_i-\bar{z})^2 \end{bmatrix}\right) \\ &= \begin{bmatrix} n & \sum_{i=1}^{n} (z_i-\bar{z}) \\[1em] \sum_{i=1}^{n} (z_i-\bar{z}) & \sum_{i=1}^{n} (z_i-\bar{z})^2 \end{bmatrix}. \end{split}$$
I'm not sure if these are right. I reckon that the questions are wrong or is it me.