Scientific Inference P3: Balancing predictive success with falsifiability - Comments

[bapowell]

bapowell submitted a new PF Insights post

Scientific Inference P3: Balancing predictive success with falsifiability

Continue reading the Original PF Insights Post.

 

[Greg Bernhardt]

This series really is phenomenal!

 

[stevendaryl]

I second Greg's comment.

But it occurred to me that we really have no basis at all for assigning $P(H)$, the a priori probability of a hypothesis. I suppose that at any given time, there are only a handful of hypotheses that have actually been developed to the extent of making testable predictions, so maybe you can just weight them all equally?

 

[stevendaryl]

I second Greg's comment.

But it occurred to me that we really have no basis at all for assigning $P(H)$, the a priori probability of a hypothesis. I suppose that at any given time, there are only a handful of hypotheses that have actually been developed to the extent of making testable predictions, so maybe you can just weight them all equally?

In the article, it's not $P(H)$ but $P(H | \mathcal{M})$, but I'm not sure that I understand the role of $\mathcal{M}$ here.

 

[Hornbein]

I second Greg's comment.

But it occurred to me that we really have no basis at all for assigning $P(H)$, the a priori probability of a hypothesis. I suppose that at any given time, there are only a handful of hypotheses that have actually been developed to the extent of making testable predictions, so maybe you can just weight them all equally?

There should be a basis for assigning the prior. It's just not part of the math.

You could collect data that 1% of the population has AIDS. That would be your prior for an individual having the condition.

 

[stevendaryl]

There should be a basis for assigning the prior. It's just not part of the math.

You could collect data that 1% of the population has AIDS. That would be your prior for an individual having the condition.

Okay, I was thinking of a different type of "hypothesis": a law-like hypothesis such as Newton's law of gravity, or the hypothesis that AIDS is caused by HIV. I don't know how you would assign a prior to such things.

 

[bapowell]

In the article, it's not $P(H)$ but $P(H | \mathcal{M})$, but I'm not sure that I understand the role of $\mathcal{M}$ here.

You can think of the hypothesis as being the value of a certain parameter, like the curvature of the universe. The model is the underlying theory relating that parameter to the observation, and should include prior information like the range of the parameter.

 

[anorlunda]

You can think of the hypothesis as being the value of a certain parameter, like the curvature of the universe. The model is the underlying theory relating that parameter to the observation, and should include prior information like the range of the parameter.

Suppose we observe something that is totally unrelated to the underlying theory. What is M in that case? What is P(H|M) in that case?

Edit: I should have asked what is P(O|M) in that case?

 

[bapowell]

M can be thought of as the underlying theory, which in practice is a set of equations relating the observable quantities to a set of parameters (together with constraints on those parameters, like the ranges of permitted values). If an observation is made that is not well-accommodated by the model M, then we will find low posterior probabilities for the parameters of the model, p(H|O). This is a signal that we either need to consider additional parameters within M, or consider a new M altogether.

 

[Stephen Tashi]

This is all crazy - or at least it's related to mental illness according to an article on the the current Science News website: https://www.sciencenews.org/article/bayesian-reasoning-implicated-some-mental-disorders