Using a one-sided P-value test in a selection

In summary, the conversation discusses the use of a one-sided P-value test in hypothesis selection and the limitations of using P-values. It also mentions the importance of considering the power of the test and the potential for error. Additionally, it touches on the differences between confidence intervals and significance levels and their relationship to likelihood ratios. The conversation concludes by mentioning the role of sample dependence in alpha and beta values.
  • #1
natski
267
2
Hey guys,

I am currently using a one-sided P-value test in a selection between two hypotheses:

H0: null hypothesis
H1: "signal" hypothesis

I know that there are only two possible hypotheses. I have read much in the literature about how the P-value is a bit crappy and a Bayesian model selection can offer the true probabilities, whereas the P-value only let's you reject H0 at a certain significance level.

However, if I reject H0 at say 95% probability, then since there are only 2 possible hypotheses, does this not mean that H1 must be accepted at 95% confidence level too? Afterall, what other possibility is there?

Natski
 
Physics news on Phys.org
  • #2


I'm not really a statistician but the idea isn't the confidence interval. The power of the test matters when deciding what test to use. There are two types of errors you can run into, you can reject H_0 when H_0 is true (with probability alpha) and accept H_0 when H_0 is false (probability beta).

When you say confidence interval, you are talking about alpha. 95% confidence interval means that the probability of you rejecting H_0 when it is true is 0.05. But this doesn't really say much about the probability of you accepting H_0 when H_0 is false. The Neyman Pearson Lemma tells you for a given alpha, the lowest beta can be obtained by using likelihood ratios. P-value comes in here, you can work out what significance level you need to reject H_0 (say you have p-value 0.02, then you can accept H_0 if you take alpha to be less than 0.02).

Again I have no idea about Bayesian statistics but I believe the alpha and beta are not just dependent on the test you use but also dependent on the sample.
 

Related to Using a one-sided P-value test in a selection

What is a one-sided P-value test?

A one-sided P-value test is a statistical method used to determine the likelihood of obtaining a certain result by chance alone. It is commonly used to test the significance of a difference or relationship in a single direction, rather than a two-sided test which considers differences in both directions.

When should a one-sided P-value test be used?

A one-sided P-value test should be used when there is a clear hypothesis or research question that predicts a specific direction of difference or relationship. For example, if a researcher hypothesizes that a new treatment will result in an improvement in a specific outcome, a one-sided test would be appropriate to use.

How is a one-sided P-value test calculated?

A one-sided P-value test is calculated by determining the probability of obtaining a result at least as extreme as the observed result, assuming the null hypothesis is true. This is typically done by comparing the observed result to a distribution, such as the normal distribution, and calculating the area under the curve that represents the likelihood of obtaining that result by chance alone.

What is the difference between a one-sided and two-sided P-value test?

The main difference between a one-sided and two-sided P-value test is the directionality of the test. A one-sided test only considers differences in one direction, while a two-sided test considers differences in both directions. This can affect the interpretation of the results and the conclusion about the significance of the difference or relationship.

What are the potential limitations of using a one-sided P-value test?

One potential limitation of using a one-sided P-value test is that if the observed result falls in the opposite direction of the predicted result, the test may not detect it as significant. Additionally, if there are multiple possible directions of difference or relationship, a one-sided test may not provide a comprehensive analysis. It is important for researchers to carefully consider their hypothesis and choose the appropriate type of test for their specific research question.

Similar threads

  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
22
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
3
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
3
Views
854
  • Set Theory, Logic, Probability, Statistics
Replies
5
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
8
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
10
Views
3K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
1K
Back
Top