Shortcut for alpha =5% in a two-tailed test

  • Thread starter nomadreid
  • Start date
  • Tags
    Alpha Test
In summary, the conversation discusses the use of a level of significance of 5% for normally distributed data in a two-tailed test. It is suggested that there is no need to standardize the data and look at probability tables, but rather to reject values outside of two standard deviations from the mean. However, this approach may not always result in a 5% significance level and may require further considerations, such as the use of the Student's t correction for small sample sizes. The definition of the statistic itself may also involve some standardization of the data.
  • #1
nomadreid
Gold Member
1,665
203
If I adopt a level of significance of 5% for data which is normally distributed, and I am adopting a two-tailed test, then I see no reason to mess with standardizing my data and looking at the probability tables, but rather to simply reject anything outside of two standard deviations away from the mean. Am I missing some concept here?
 
Physics news on Phys.org
  • #2
nomadreid said:
If I adopt a level of significance of 5% for data which is normally distributed, and I am adopting a two-tailed test, then I see no reason to mess with standardizing my data and looking at the probability tables, but rather to simply reject anything outside of two standard deviations away from the mean. Am I missing some concept here?

You're not expressing your idea correctly. A hypothesis test is based on the distribution of a statistic, not "the data". A statistic (e.g. as used in the F-test, t-test etc.) that is a function of normally distributed data need not be normally distributed itself.
 
  • #3
nomadreid said:
If I adopt a level of significance of 5% for data which is normally distributed, and I am adopting a two-tailed test, then I see no reason to mess with standardizing my data and looking at the probability tables, but rather to simply reject anything outside of two standard deviations away from the mean. Am I missing some concept here?

Yes. Reject anything outside of two standard deviations away from the mean does not necessarily imply 5% significance.
 
  • #4
Thank you, Imalooser and Stephen Tashi. On both, I would be grateful for a bit of clarification. First, Stephen Tashi, you are correct, so what I should have written (I think) is: suppose I have a test statistic which is normally distributed, and I perform a two-tailed test, whereby I assume that the level of significance is split symmetrically between the two tails. In this case, may I then reject the hypothesis for values which fall outside of two standard deviations from the mean?
(side note: my background is in pure maths -- pure enough so that I have escaped until now needing any statistics; so although I am a beginner in statistics, replies using mathematical concepts are no problem.)
Now, Imalooser, I understand that the 5% significance level can include values which are within the interval with endpoints being two standard deviations from the mean, but I am a little shaky on the converse as you stated it. If, given the assumptions I mentioned above, I reject the values outside of that interval, that would give me an area of less than 5%. Your reply implies, however, that this wouldn't imply a level of significance of less than 5%. Could you give me a counter-example so that I can clearly see the difference?
Thanks in advance for further replies.
 
  • #5
nomadreid said:
Thank you, Imalooser and Stephen Tashi. On both, I would be grateful for a bit of clarification. First, Stephen Tashi, you are correct, so what I should have written (I think) is: suppose I have a test statistic which is normally distributed, and I perform a two-tailed test, whereby I assume that the level of significance is split symmetrically between the two tails. In this case, may I then reject the hypothesis for values which fall outside of two standard deviations from the mean?
(side note: my background is in pure maths -- pure enough so that I have escaped until now needing any statistics; so although I am a beginner in statistics, replies using mathematical concepts are no problem.)
Now, Imalooser, I understand that the 5% significance level can include values which are within the interval with endpoints being two standard deviations from the mean, but I am a little shaky on the converse as you stated it. If, given the assumptions I mentioned above, I reject the values outside of that interval, that would give me an area of less than 5%. Your reply implies, however, that this wouldn't imply a level of significance of less than 5%. Could you give me a counter-example so that I can clearly see the difference?
Thanks in advance for further replies.

If the test statistic is normally distributed then yes you are almost right. It's actually 1.97, not 2.

An example of a test statistic that is not normally distributed is a sample from a normal distribution of size less than thirty. Then the statistic is significantly not normally distributed and it is necessary to use the Student's t correction.

>If, given the assumptions I mentioned above, I reject the values outside of that interval, that would give me an area of less than 5%.

If the accept region has 95% of the area then the reject region must have 5%. The two regions must sum to 100%.

But yes, if you know what you are doing then no one can stop you from taking shortcuts. A teacher may want you to cross all the is and dot all the ts to prove you know what you're doing.
 
  • #6
nomadreid said:
suppose I have a test statistic which is normally distributed, and I perform a two-tailed test, whereby I assume that the level of significance is split symmetrically between the two tails. In this case, may I then reject the hypothesis for values which fall outside of two standard deviations from the mean?

Yes. By the definition of the "5% significance level", you must. You would also have to do it for the 4.55% significance level.

However, in regard to your original question:

I see no reason to mess with standardizing my data and looking at the probability tables,

The definition of the statistic itself may involve some standardization of the data, such as dividing the sample values by an estimator of the standard deviation. You can't avoid that if you are using the statistic.

I think you are asking is whether the single value of a normally distributed statistic need be "standardized" in order compute various probabiliites. What would this mean in general? You have a sample of N things and you have one statistic computed from those N things. What data would you use to estimate "the standard deviation of the statistic" since you have only one value of the statistic?
 
  • #7
Again, thanks to both Imalooser and Stephen Tashi.
First, Imalooser, your point about the normal distribution being only applicable for a size greater than 30 is understood, and I should have included that in my stated assumptions. Thanks for the correction. (By the way, despite the elementary nature of my questions, I am not a student; I simply wish to figure out a few statistics concepts on my own. Hence I am grateful for this Forum.)
Second: Stephen Tashi: actually, I was assuming (always dangerous) that I had sufficient information from the sampling distribution in order to figure out standard deviations and means, not merely the one statistic. Otherwise put, I assumed that I would have more than 30 samples, each one giving me its value of that statistic. In this case I could figure out whether the given statistic was at least two standard deviations from the mean. But one phrase you used puzzled me: you wrote of a "normally distributed statistic". Since a statistic is a single number, do you mean:
(a) a statistic calculated from a normally distributed sample (from a population which is assumed to be normally distributed), or
(b) a statistic which is one of N>30 statistics, so that the N statistics are themselves normally distributed?
Thanks again for any further help.
 
  • #8
nomadreid said:
I was assuming (always dangerous) that I had sufficient information from the sampling distribution in order to figure out standard deviations and means, not merely the one statistic. Otherwise put, I assumed that I would have more than 30 samples, each one giving me its value of that statistic. In this case I could figure out whether the given statistic was at least two standard deviations from the mean. But one phrase you used puzzled me: you wrote of a "normally distributed statistic". Since a statistic is a single number,

In statistics, a "statistic" is not a single number. (In layman's terms a "statistic" is a single number as in "2.4 children per family". Even statisticians lapse into this kind of speech, so perhaps we must regard "statistic" as being an ambiguous term. However, it has only one "high class" definition in mathematical statistics.)

Since you have a background in pure math, you should appreciate the need for precise terminology. Technically a statistic is a function, not a single value. For example, the mean of a sample of 60 indpendent realizations of a random variable X is a statistic and it is defined by the formula that adds up the 60 realizations and divides by 60.

You could consider an individual realization of a random variable X as a statistic. However, unless you know that X is normally distributed, you don't know that the statistic defined by one realization is a normal random variable. By contrast, the Central Limit Theorem tells you that the statistic defined by the mean of (say) 60 independent samples of X has an approximately normal distribution even if X doesn't.

do you mean:
(a) a statistic calculated from a normally distributed sample (from a population which is assumed to be normally distributed), or
(b) a statistic which is one of N>30 statistics, so that the N statistics are themselves normally distributed?
Thanks again for any further help.

I mean neither of the above. A statistic may have a normal distribution without being a function of a sample from a normally distributed population. A statistic based on fewer that 30 realizations of a normal random variable may be normally distributed. So the two choices above imply restrictions that do not apply. By normally distributed statistic, I simply mean a random variable S with a normal distribution. (Since a statistic S os a function of sample values, it is a function of random variables, hence the statistic S is a random varible itself.)

I think you are incorrectly narrowing the scenario for hypothesis testing in your mind. You think of it only in terms of using a statistic S that is the mean value of some realizations. A statistic can be any function of the sample values. For example, we can define a statistic S1 to be the minimum value that appears in 60 independent realizations of a random variable. (See "order statistics".)

The fact that a statistic is a random variable causes mathematical statistics to have a recursive nature. A statistic S is a function of realizations of a random variable X, so S is a random variable itself. It has its own probability distribution. The distribution usually has its own mean, variance etc. In general, these parameters need not be the same as those of the distribution of X. (There is also no requirement that a statistic be a function of independent samples of a random variable even though the most of commonly used statistics are. )
 
Last edited:
  • #9
Many thanks, Stephen Tashi. Your explanation helped clear up a number of issues. And yes, I very much appreciate precise terminology.
 

What is a two-tailed test?

A two-tailed test is a statistical hypothesis test where the alternative hypothesis includes both the possibility of a positive and negative effect. This means that the test is looking for any significant difference or relationship in both directions, rather than just one specific direction.

What is alpha in a two-tailed test?

In statistics, alpha (α) is the level of significance that is used to determine the critical value in a hypothesis test. In a two-tailed test, an alpha level of 5% means that there is a 5% chance of rejecting the null hypothesis when it is actually true. This is also known as the type I error rate.

What does it mean to have a shortcut for alpha = 5%?

Having a shortcut for alpha = 5% means that there is a specific value that can be used to determine the critical value in a two-tailed test without having to calculate it manually. This value is typically found in a statistical table or can be determined using a statistical software program.

Why is it important to use a two-tailed test with alpha = 5%?

Using a two-tailed test with alpha = 5% allows for a more comprehensive analysis of the data by considering both positive and negative effects. This can provide a more accurate understanding of the relationship between variables and reduce the likelihood of making a type I error.

What happens if alpha is set too high or too low in a two-tailed test?

If alpha is set too high (e.g. 10%), it increases the likelihood of making a type I error and incorrectly rejecting the null hypothesis. On the other hand, if alpha is set too low (e.g. 1%), it decreases the likelihood of making a type I error but increases the likelihood of making a type II error (failing to reject the null hypothesis when it is actually false).

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
2K
  • Calculus and Beyond Homework Help
Replies
1
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
7
Views
336
  • Set Theory, Logic, Probability, Statistics
Replies
26
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
20
Views
3K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
815
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
9
Views
1K
Back
Top