Shortcut for alpha =5% in a two-tailed test

[nomadreid]

If I adopt a level of significance of 5% for data which is normally distributed, and I am adopting a two-tailed test, then I see no reason to mess with standardizing my data and looking at the probability tables, but rather to simply reject anything outside of two standard deviations away from the mean. Am I missing some concept here?

 

[Stephen Tashi]

If I adopt a level of significance of 5% for data which is normally distributed, and I am adopting a two-tailed test, then I see no reason to mess with standardizing my data and looking at the probability tables, but rather to simply reject anything outside of two standard deviations away from the mean. Am I missing some concept here?

You're not expressing your idea correctly. A hypothesis test is based on the distribution of a statistic, not "the data". A statistic (e.g. as used in the F-test, t-test etc.) that is a function of normally distributed data need not be normally distributed itself.

 

[ImaLooser]

If I adopt a level of significance of 5% for data which is normally distributed, and I am adopting a two-tailed test, then I see no reason to mess with standardizing my data and looking at the probability tables, but rather to simply reject anything outside of two standard deviations away from the mean. Am I missing some concept here?

Yes. Reject anything outside of two standard deviations away from the mean does not necessarily imply 5% significance.

 

[nomadreid]

Thank you, Imalooser and Stephen Tashi. On both, I would be grateful for a bit of clarification. First, Stephen Tashi, you are correct, so what I should have written (I think) is: suppose I have a test statistic which is normally distributed, and I perform a two-tailed test, whereby I assume that the level of significance is split symmetrically between the two tails. In this case, may I then reject the hypothesis for values which fall outside of two standard deviations from the mean?
(side note: my background is in pure maths -- pure enough so that I have escaped until now needing any statistics; so although I am a beginner in statistics, replies using mathematical concepts are no problem.)
Now, Imalooser, I understand that the 5% significance level can include values which are within the interval with endpoints being two standard deviations from the mean, but I am a little shaky on the converse as you stated it. If, given the assumptions I mentioned above, I reject the values outside of that interval, that would give me an area of less than 5%. Your reply implies, however, that this wouldn't imply a level of significance of less than 5%. Could you give me a counter-example so that I can clearly see the difference?
Thanks in advance for further replies.

 

[ImaLooser]

Thank you, Imalooser and Stephen Tashi. On both, I would be grateful for a bit of clarification. First, Stephen Tashi, you are correct, so what I should have written (I think) is: suppose I have a test statistic which is normally distributed, and I perform a two-tailed test, whereby I assume that the level of significance is split symmetrically between the two tails. In this case, may I then reject the hypothesis for values which fall outside of two standard deviations from the mean?
(side note: my background is in pure maths -- pure enough so that I have escaped until now needing any statistics; so although I am a beginner in statistics, replies using mathematical concepts are no problem.)
Now, Imalooser, I understand that the 5% significance level can include values which are within the interval with endpoints being two standard deviations from the mean, but I am a little shaky on the converse as you stated it. If, given the assumptions I mentioned above, I reject the values outside of that interval, that would give me an area of less than 5%. Your reply implies, however, that this wouldn't imply a level of significance of less than 5%. Could you give me a counter-example so that I can clearly see the difference?
Thanks in advance for further replies.

If the test statistic is normally distributed then yes you are almost right. It's actually 1.97, not 2.

An example of a test statistic that is not normally distributed is a sample from a normal distribution of size less than thirty. Then the statistic is significantly not normally distributed and it is necessary to use the Student's t correction.

>If, given the assumptions I mentioned above, I reject the values outside of that interval, that would give me an area of less than 5%.

If the accept region has 95% of the area then the reject region must have 5%. The two regions must sum to 100%.

But yes, if you know what you are doing then no one can stop you from taking shortcuts. A teacher may want you to cross all the is and dot all the ts to prove you know what you're doing.

 

[Stephen Tashi]

suppose I have a test statistic which is normally distributed, and I perform a two-tailed test, whereby I assume that the level of significance is split symmetrically between the two tails. In this case, may I then reject the hypothesis for values which fall outside of two standard deviations from the mean?

Yes. By the definition of the "5% significance level", you must. You would also have to do it for the 4.55% significance level.

However, in regard to your original question:

I see no reason to mess with standardizing my data and looking at the probability tables,

The definition of the statistic itself may involve some standardization of the data, such as dividing the sample values by an estimator of the standard deviation. You can't avoid that if you are using the statistic.

I think you are asking is whether the single value of a normally distributed statistic need be "standardized" in order compute various probabiliites. What would this mean in general? You have a sample of N things and you have one statistic computed from those N things. What data would you use to estimate "the standard deviation of the statistic" since you have only one value of the statistic?

 

[nomadreid]

Again, thanks to both Imalooser and Stephen Tashi.
First, Imalooser, your point about the normal distribution being only applicable for a size greater than 30 is understood, and I should have included that in my stated assumptions. Thanks for the correction. (By the way, despite the elementary nature of my questions, I am not a student; I simply wish to figure out a few statistics concepts on my own. Hence I am grateful for this Forum.)
Second: Stephen Tashi: actually, I was assuming (always dangerous) that I had sufficient information from the sampling distribution in order to figure out standard deviations and means, not merely the one statistic. Otherwise put, I assumed that I would have more than 30 samples, each one giving me its value of that statistic. In this case I could figure out whether the given statistic was at least two standard deviations from the mean. But one phrase you used puzzled me: you wrote of a "normally distributed statistic". Since a statistic is a single number, do you mean:
(a) a statistic calculated from a normally distributed sample (from a population which is assumed to be normally distributed), or
(b) a statistic which is one of N>30 statistics, so that the N statistics are themselves normally distributed?
Thanks again for any further help.

 

[Stephen Tashi]

I was assuming (always dangerous) that I had sufficient information from the sampling distribution in order to figure out standard deviations and means, not merely the one statistic. Otherwise put, I assumed that I would have more than 30 samples, each one giving me its value of that statistic. In this case I could figure out whether the given statistic was at least two standard deviations from the mean. But one phrase you used puzzled me: you wrote of a "normally distributed statistic". Since a statistic is a single number,

In statistics, a "statistic" is not a single number. (In layman's terms a "statistic" is a single number as in "2.4 children per family". Even statisticians lapse into this kind of speech, so perhaps we must regard "statistic" as being an ambiguous term. However, it has only one "high class" definition in mathematical statistics.)

Since you have a background in pure math, you should appreciate the need for precise terminology. Technically a statistic is a function, not a single value. For example, the mean of a sample of 60 indpendent realizations of a random variable X is a statistic and it is defined by the formula that adds up the 60 realizations and divides by 60.

You could consider an individual realization of a random variable X as a statistic. However, unless you know that X is normally distributed, you don't know that the statistic defined by one realization is a normal random variable. By contrast, the Central Limit Theorem tells you that the statistic defined by the mean of (say) 60 independent samples of X has an approximately normal distribution even if X doesn't.

do you mean:
(a) a statistic calculated from a normally distributed sample (from a population which is assumed to be normally distributed), or
(b) a statistic which is one of N>30 statistics, so that the N statistics are themselves normally distributed?
Thanks again for any further help.

I mean neither of the above. A statistic may have a normal distribution without being a function of a sample from a normally distributed population. A statistic based on fewer that 30 realizations of a normal random variable may be normally distributed. So the two choices above imply restrictions that do not apply. By normally distributed statistic, I simply mean a random variable S with a normal distribution. (Since a statistic S os a function of sample values, it is a function of random variables, hence the statistic S is a random varible itself.)

I think you are incorrectly narrowing the scenario for hypothesis testing in your mind. You think of it only in terms of using a statistic S that is the mean value of some realizations. A statistic can be any function of the sample values. For example, we can define a statistic S1 to be the minimum value that appears in 60 independent realizations of a random variable. (See "order statistics".)

The fact that a statistic is a random variable causes mathematical statistics to have a recursive nature. A statistic S is a function of realizations of a random variable X, so S is a random variable itself. It has its own probability distribution. The distribution usually has its own mean, variance etc. In general, these parameters need not be the same as those of the distribution of X. (There is also no requirement that a statistic be a function of independent samples of a random variable even though the most of commonly used statistics are. )

 

[nomadreid]

Many thanks, Stephen Tashi. Your explanation helped clear up a number of issues. And yes, I very much appreciate precise terminology.