Have you heard about this kind of test of hypothesis

[riemann86]

In one example I saw someone make an argument I haven't seen before. They had 2 confidence intervals for 2 different statistics, and they said that since these two confidence intervals didnt overlap, we could assume that the expected values where different. This made me think of test of hypothesis, and I am wondering the justification behind the argument that if 2 different confidence intervals do not overlap, we can assume the expected value of each is different.

We have two indepent normally distributed random variables, $\hat{X}$ and $\hat{Y}$, with given variance, $\sigma^{2}_{\hat{X}}$ and $\sigma^{2}_{\hat{Y}}$. We also have that $E(\hat{X})=\mu_{X}$, and $E(\hat{Y})=\mu_{Y}$. However we do not know $\mu_{X}$, and $\mu_{Y}$.

We now want to test the hypothesis that $\mu_{X}$ = $\mu_{Y}$.

The standard way of doing is offcourse by
rejecting the hypotesis if

$\left|{\frac{\hat{X}-\hat{Y}}{\sqrt{\sigma^{2}_{\hat{X}}+\sigma^{2}_{\hat{Y}}} } }\right|>z_{\alpha/2}$The alternative way of testing this was constructing 2 confidence intervals, and rejecting the hypothesis if the two intervals do not overlap.
The 2 intervals are:
$(\hat{X}-z_{\alpha/2}*\sigma_{\hat{X}}, \hat{X}+z_{\alpha/2}*\sigma_{\hat{X}})$
and $(\hat{Y}-z_{\alpha/2}*\sigma_{\hat{Y}}, \hat{Y}+z_{\alpha/2}*\sigma_{\hat{Y }})$

However if the two intervals are not to overlap we have either that:
$\hat{Y}+z_{\alpha/2}*\sigma_{\hat{Y}} < \hat{X}-z_{\alpha/2}*\sigma_{\hat{X}}$
or we have that :
$\hat{Y}-z_{\alpha/2}*\sigma_{\hat{Y}} > \hat{X}+z_{\alpha/2}*\sigma_{\hat{X}}$

Together these two inequalities give that we reject the hypothesis if:
$\left|\frac{\hat{X}-\hat{Y}}{\sigma_{\hat{X}}+\sigma_{\hat{Y}}} \right|> z_{\alpha/2}$Now my question is, what is then the justification for using the argument "we can assume that they are different if the 2 confidence intervals do not overlap". In the original test where we have $\sqrt{\sigma^{2}_{\hat{X}}+\sigma^{2}_{\hat{Y}}}$ in the denominator, and we have control on the significance level $\alpha$. When we developed this new test, we just said that we didnt want the 2 confidence interval to overlap. But with algebra we have showed that this new test, is the same as the old test but with the new denominator: $\sigma_{\hat{X}}+\sigma_{\hat{Y}}$. But is there anyting else we can say about this new test?, is it better or worse than the old test? Could it have been derived by starting with the statistic: $\frac{\hat{X}-\hat{Y}}{\sigma_{\hat{X}}+\sigma_{\hat{Y}}}$? I mean, the statistic: $\frac{\hat{X}-\hat{Y}}{\sigma_{\hat{X}}+\sigma_{\hat{Y}}}$ is probably(?) not standard normal distributed, so what do we know about this test?

 

[Stephen Tashi]

what is then the justification for using the argument "we can assume that they are different if the 2 confidence intervals do not overlap".

The two tests are different tests.

$S_2 = \left| \frac { \hat{X} - \hat{Y} } { \sqrt{ \sigma_x^2 + \sigma_y^2}}\right| \ge S_1= \left| \frac{ \hat{X} - \hat{Y} }{ \sigma_x + \sigma_y}\right|$

So it would be correct to say that if a particular observed value of $S_1\ge z_{\alpha/2}$ then there is no need to compute the value of $S_2$ to see if it is $\ge z_{\alpha/2}$ also.

It would not be correct to say that an observed value of $S_1 \le z_{\alpha/2}$ let's us conclude that the null hypothesis would be accepted by the test that uses $S_2$.

 

[riemann86]

The two tests are different tests.

$S_2 = \left| \frac { \hat{X} - \hat{Y} } { \sqrt{ \sigma_x^2 + \sigma_y^2}}\right| \ge S_1= \left| \frac{ \hat{X} - \hat{Y} }{ \sigma_x + \sigma_y}\right|$

So it would be correct to say that if a particular observed value of $S_1\ge z_{\alpha/2}$ then there is no need to compute the value of $S_2$ to see if it is $\ge z_{\alpha/2}$ also.

It would not be correct to say that an observed value of $S_1 \le z_{\alpha/2}$ let's us conclude that the null hypothesis would be accepted by the test that uses $S_2$.

Thank you very much, that is a very good point!