Some questions about hypothesis testing

In summary, the concept of alpha is used in hypothesis testing to determine the probability of rejecting the null hypothesis when it is actually true. It is represented by the symbol alpha and is calculated as the maximum of the probabilities of rejecting the null hypothesis, taken over all possible ways that the null hypothesis can be true. This definition is more sophisticated than the common definition of alpha as the probability of rejecting the null hypothesis when it is true."
  • #1
Artusartos
247
0
I think I’m a bit confused about what [tex]\alpha[/tex] is. For the definition that is given, what theta really means when they say [tex]\alpha = max_{\theta \in w_0} P_{\theta} [(X_1, … , X_n) \in C]. I just think that it’s not really clear in my head, so is it ok if anybody explains/gives and example about this?

(my question is about the second attachment)

Thanks in advance
 

Attachments

  • 20121201_075858[1].jpg
    20121201_075858[1].jpg
    50.8 KB · Views: 412
  • 20121201_075910[1].jpg
    20121201_075910[1].jpg
    43.5 KB · Views: 392
Physics news on Phys.org
  • #2
Perhaps the example of a "one tailed" test is the best intuitive explanation for that. Suppose we have a coin and our "null hypothesis" is "the coin is not biased toward heads". This hypothesis is different that the hypothesis "the coin is fair" because "not biased toward heads" would include the cases where the probability of the coin landing heads was 0.0 or 0.01 or 0.499 etc.

Hypothesis testing is a procedure where you define some feature of experimental data as "the test statistic" and define some "acceptance region" for that feature. If the feature of the particular data that you observe is outside the "acceptance region", you "reject" the null hypothesis. (Hypothesis testing isn't a mathematical proof that the null hypothesis is true or that it is false and it doesn't compute the probability that the null hypothesis is true or the probability that the null hypothesis is false. It is simply a procedure.)

To apply hypothesis testing to the above example, you must pick a feature of the data. Let's say the data is 100 independent flips of the coin. Let the feature be the total number of heads that occurred. Let's say the "acceptance region" is "80 or fewer flips of the coin produced heads".

The intuitive idea of "alpha" is to answer the question "What is the probability that I will reject the null hypothesis when it is actually true?". In this example, the question is "What is the probability that the observed number of heads will be more than 80 when the coin is actually "not biased toward heads". However, we can't compute a probability for this event because the statement the statement that the coin is "not biased toward heads" doesn't give us a specific probability to compute with. We don't know whether to assume that the probability of a head is 0.0 or 0.01 or .499 etc. Intuitively, if we assume the probability of heads is 0.5 then we are letting the coin be "as prone as possible to produce a head, without actually being biased toward producing a head". Assuming the probability of a head is 0.5 gives us a specific probability to use and it captures the idea of "the situation where the null hypothesis would be most likely to produce a false result".

Your book's definition is more sophisticated than the definition found in many introductory texts. Many texts define alpha as "the probability of rejecting the null hypothesis when it is true". Your text defines it as "the maximum of the probabiilities of rejecting the null hypothesis taken over all the possible ways the null hypothesis can be true". That is the more general definition of alpha.
 
  • #3
Stephen Tashi said:
Perhaps the example of a "one tailed" test is the best intuitive explanation for that. Suppose we have a coin and our "null hypothesis" is "the coin is not biased toward heads". This hypothesis is different that the hypothesis "the coin is fair" because "not biased toward heads" would include the cases where the probability of the coin landing heads was 0.0 or 0.01 or 0.499 etc.

Hypothesis testing is a procedure where you define some feature of experimental data as "the test statistic" and define some "acceptance region" for that feature. If the feature of the particular data that you observe is outside the "acceptance region", you "reject" the null hypothesis. (Hypothesis testing isn't a mathematical proof that the null hypothesis is true or that it is false and it doesn't compute the probability that the null hypothesis is true or the probability that the null hypothesis is false. It is simply a procedure.)

To apply hypothesis testing to the above example, you must pick a feature of the data. Let's say the data is 100 independent flips of the coin. Let the feature be the total number of heads that occurred. Let's say the "acceptance region" is "80 or fewer flips of the coin produced heads".

The intuitive idea of "alpha" is to answer the question "What is the probability that I will reject the null hypothesis when it is actually true?". In this example, the question is "What is the probability that the observed number of heads will be more than 80 when the coin is actually "not biased toward heads". However, we can't compute a probability for this event because the statement the statement that the coin is "not biased toward heads" doesn't give us a specific probability to compute with. We don't know whether to assume that the probability of a head is 0.0 or 0.01 or .499 etc. Intuitively, if we assume the probability of heads is 0.5 then we are letting the coin be "as prone as possible to produce a head, without actually being biased toward producing a head". Assuming the probability of a head is 0.5 gives us a specific probability to use and it captures the idea of "the situation where the null hypothesis would be most likely to produce a false result".

Your book's definition is more sophisticated than the definition found in many introductory texts. Many texts define alpha as "the probability of rejecting the null hypothesis when it is true". Your text defines it as "the maximum of the probabiilities of rejecting the null hypothesis taken over all the possible ways the null hypothesis can be true". That is the more general definition of alpha.

Thank you so much for your help, but I think I'm a bit confused again...

There is an example in my textbook (which I attached)...

It says that [tex]\alpha = P_{H_0} [S \leq k][/tex] and [tex]\alpha = P_{p_0} [S \leq k][/tex]

I have two questions:

1) In my previous attachments the textbook defined alpha as the maximum of [tex]P_{\theta} [(X_1, ... , X_n) \in C][/tex] So in this example, are they saying that [tex]\theta = H_0[/tex] when they are saying [tex]\alpha = P_{H_0} [S \leq k][/tex]?

2) How did [tex]H_0[/tex] turn into [tex]p_0[/tex] when they wrote [tex]P_{p_0} [S \leq k][/tex]
instead of [tex]\alpha = P_{H_0} [S \leq k][/tex]

Thanks in advance
 

Attachments

  • 20121201_160139.jpg
    20121201_160139.jpg
    47.7 KB · Views: 391
Last edited:
  • #4
Artusartos said:
1) So in this example, are they saying that [tex]\theta = H_0[/tex] when they are saying [tex]\alpha = P_{H_0} [S \leq k][/tex]?
The null hypothesis [itex] H_0 [/itex] is a statement, not a number. So it wouldn't make sense to say [itex] \theta = H_0[/itex]. As I understand their notation [itex]P_{ H_0}[S \leq k ] [/itex] means "The probability that S is less than or equal to k under the assumption that [itex] H_0 [/itex] is true" and [itex] P_{p_0}[ S \leq k ] [/itex] means the "The probability that S is less than or equal to k under the assumption that the probability of success is [itex] p_0 [/itex]. So both expressions refer to the same probability. (Your text didn't do a good job of definining those notations.)

In that example it is not necessary to speak of the maximum probability computer over the set of all [itex] \theta [/itex]. The null hypothesis in the example is stated as [itex] \theta = p = p_0 [/itex]. So the null hypothesis only deals with a single value of [itex] \theta [/itex].

If they had chosen to state the null hypothesis as [itex] \theta = p \ge p_0 [/itex] then we would have done the same computation for [itex] \alpha [/itex] as the example did, since [itex] p = p_0 [/itex] is the value of [itex] p [/itex] that maximizes the probability that [itex] S \leq k [/itex] among all the possible values of [itex] p [/itex] that are allowed when the null hypothesis is true.

It's an interesting question whether the null hypothesis in the example should be "The new treatment has the same effectiveness as the old treatment" or whether it makes more sense to make it say "The new treatment is no more effective than the old treatment". Since the example proposes a "one tailed" acceptance region, I think it makes more sense to phrase the null hypothesis the second way.

Trying to prove whether a particular type of acceptance region (one tailed, two tailed, or a even bunch of isolated intervals) is "best" involves defining what "best" means. The only way I know to approach that topic in frequentist statistics is to compare the "power" of tests that use difference acceptance regions. The power of a test is defined by a function, not by a single number, so comparing the power of two tests is not straightforward either.
 
  • #5
Stephen Tashi said:
The null hypothesis [itex] H_0 [/itex] is a statement, not a number. So it wouldn't make sense to say [itex] \theta = H_0[/itex]. As I understand their notation [itex]P_{ H_0}[S \leq k ] [/itex] means "The probability that S is less than or equal to k under the assumption that [itex] H_0 [/itex] is true" and [itex] P_{p_0}[ S \leq k ] [/itex] means the "The probability that S is less than or equal to k under the assumption that the probability of success is [itex] p_0 [/itex]. So both expressions refer to the same probability. (Your text didn't do a good job of definining those notations.)

In that example it is not necessary to speak of the maximum probability computer over the set of all [itex] \theta [/itex]. The null hypothesis in the example is stated as [itex] \theta = p = p_0 [/itex]. So the null hypothesis only deals with a single value of [itex] \theta [/itex].

If they had chosen to state the null hypothesis as [itex] \theta = p \ge p_0 [/itex] then we would have done the same computation for [itex] \alpha [/itex] as the example did, since [itex] p = p_0 [/itex] is the value of [itex] p [/itex] that maximizes the probability that [itex] S \leq k [/itex] among all the possible values of [itex] p [/itex] that are allowed when the null hypothesis is true.

It's an interesting question whether the null hypothesis in the example should be "The new treatment has the same effectiveness as the old treatment" or whether it makes more sense to make it say "The new treatment is no more effective than the old treatment". Since the example proposes a "one tailed" acceptance region, I think it makes more sense to phrase the null hypothesis the second way.

Trying to prove whether a particular type of acceptance region (one tailed, two tailed, or a even bunch of isolated intervals) is "best" involves defining what "best" means. The only way I know to approach that topic in frequentist statistics is to compare the "power" of tests that use difference acceptance regions. The power of a test is defined by a function, not by a single number, so comparing the power of two tests is not straightforward either.

Thank you so much for your time.
 

1. What is hypothesis testing?

Hypothesis testing is a statistical method used to determine whether a certain hypothesis or claim about a population is supported by the data. It involves comparing a sample of data to a null hypothesis, which represents the assumption that there is no significant difference or relationship in the population.

2. How do you come up with a hypothesis?

A hypothesis is typically derived from previous research, observations, or theories about a particular phenomenon. It should be specific, testable, and based on existing evidence or knowledge.

3. What is the difference between a null hypothesis and an alternative hypothesis?

The null hypothesis (H0) represents the assumption that there is no significant difference or relationship in the population, while the alternative hypothesis (Ha) represents the opposite - that there is a significant difference or relationship.

4. What is a p-value and how is it interpreted?

A p-value is the probability of obtaining a result at least as extreme as the one observed in the data, assuming the null hypothesis is true. It is typically compared to a predetermined significance level (usually 0.05) to determine if the result is statistically significant. If the p-value is less than the significance level, the null hypothesis is rejected and the alternative hypothesis is supported.

5. Can hypothesis testing prove causation?

No, hypothesis testing can only determine the likelihood of a certain hypothesis being true based on the data. It cannot prove causation, as there may be other factors or variables that are influencing the relationship between the variables being tested. Additional research and experimentation may be needed to establish causation.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
11
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
792
  • Set Theory, Logic, Probability, Statistics
Replies
7
Views
338
  • Set Theory, Logic, Probability, Statistics
Replies
8
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
4K
Back
Top