Understand the Importance of Reference Priors for Signal Search

In summary: You might argue that it is a common mistake, so let's use ##f## for density, but I don't think that is a common mistake and I'm not aware of any alternative notation for density.)In summary, the importance of a reference prior in a Bayesian analysis is to minimize the differences between the prior and posterior distributions of a parameter of interest. This is especially crucial when there is not a lot of data available, as the prior can heavily influence the posterior. However, when there is a large amount of data, the impact of the prior diminishes and the data itself becomes the main driver of the posterior distribution. It is important to understand the difference between probability and density in this context and to use the appropriate notation.
  • #1
ChrisVer
Gold Member
3,378
464
Hi, a very basic question: what is a good intuitive way to understand the importance of a reference prior? In the context of a signal search.
Bellow, I also try to give the way I understand the approach in a Bayesian analysis (roughly):
1. You have your likelihood model [itex]L = p(x_{obs} | \lambda , b+\mu s)[/itex], with the expected events ([itex]b,s[/itex] background,signal) and several priors [itex]\pi[/itex] (for your parameter of interest [itex]\mu[/itex] and other nuisance parameters [itex]\lambda[/itex] such as the uncertainties).
2. The posterior pdf is what you need in order to study the parameter of interest [itex]\mu[/itex]. By Bayes' Theorem, this is:
[itex] p ( \mu | x_{obs} ) = \frac{ L \pi(\lambda) \pi(\mu,\lambda)}{p(x_{obs})}[/itex]
The denominator several times can be taken out as it's only making sure that the normalization is correct for a pdf. Also I consider [itex]\pi(\mu,\lambda) = \pi(\mu) \pi(\lambda)[/itex], which tells that the two parameters are independent.
3. You run several experiments, and from the outcome of each experiment you "update" your knowledge on the parameter of interest. Aka in the end you build up the posterior probability (once you integrate out the extra dimensions from the NPs):
[itex] p( \mu | x_{obs} ) \propto \int d \lambda ~ p(\mu | x_{obs} , \lambda ) \pi(\lambda) [/itex]

So far I think I understand everything, maybe with some misconceptions which could potentially be pointed out.. When though one starts speaking about reference priors I am somewhat lost. Based on a few searches, I think the main target of the reference prior is to minimize the differences between the posterior and the prior. However I don't quiet understand how is that important as:
"I can give in any prior which I like (with reasonable limitations), and it's up to the observation/experiment to tell me how it evolves with the extra information. By reasonable I mean that for example it can't be 0 at ranges where the posterior is non-zero (as Bayes' theorem would result in 0 for the posterior)."
How could different distributions of a prior end up in updating it to different distributions for the posterior?
 
Physics news on Phys.org
  • #2
ChrisVer said:
Hi, a very basic question: what is a good intuitive way to understand the importance of a reference prior? In the context of a signal search.
Bellow, I also try to give the way I understand the approach in a Bayesian analysis (roughly):
1. You have your likelihood model [itex]L = p(x_{obs} | \lambda , b+\mu s)[/itex], with the expected events ([itex]b,s[/itex] background,signal) and several priors [itex]\pi[/itex] (for your parameter of interest [itex]\mu[/itex] and other nuisance parameters [itex]\lambda[/itex] such as the uncertainties).
2. The posterior pdf is what you need in order to study the parameter of interest [itex]\mu[/itex]. By Bayes' Theorem, this is:
[itex] p ( \mu | x_{obs} ) = \frac{ L \pi(\lambda) \pi(\mu,\lambda)}{p(x_{obs})}[/itex]
The denominator several times can be taken out as it's only making sure that the normalization is correct for a pdf. Also I consider [itex]\pi(\mu,\lambda) = \pi(\mu) \pi(\lambda)[/itex], which tells that the two parameters are independent.

Your number 2 is wrong by any standard I'm aware of. If you are using a probability density function, then it should be something like ##f_{\mu\vert X}(\cdot \vert x_{obs}) ##. Crucially this is a probability density, not a probability. (Look to the CDF for the probability.)
ChrisVer said:
...
How could different distributions of a prior end up in updating it to different distributions for the posterior?

There are a lot of different issues here. In some sense, this is the whole point of Bayesian Inference.

Have you tried working out some very simple finite cases? I would almost always start with finite, then consider countably infinite, and after all that maybe consider the continuous / uncountable case.

E.g. suppose you have a coin that is either 50:50 heads: tails or 70:30 heads tails or 90:10 heads tails. Now suppose you run 5 trials and the results are ___.
Now suppose instead you run 50000 trials and the results are ___. Try working this through with a uniform prior, vs something heavily skewed toward 50:50 case. You should be able to clearly see a big impact in the choice of prior for the former.

Loosely speaking: the prior has a big impact if you don't have much observations / data. If you have a lot of data / observations, then you can 'overwhelm your prior' if such data and the choice of prior has minimal impact on the posterior -- except if you zero things out as being impossible in the prior, then there is no opportunity to overwhelm. (There are a lot of subtleties in the continuous case, though. A lot of people equate zero probability with impossibility -- and that is in general wrong.)
 
Last edited:
  • Like
Likes WWGD
  • #3
StoneTemplePython said:
Your number 2 is wrong by any standard I'm aware of.
How so? I mean it's Bayes' theorem and I called them pdfs not probabilities (?)
 
  • #4
ChrisVer said:
How so? I mean it's Bayes' theorem and I called them pdfs not probabilities (?)

This may be a superficial issue. ##P## seems to alway refer to cumulative probability (read: from CDF) and ##p## for probability at a point (read from PMF or in a transition matrix, and such).

Put differently: by any standard I'm aware of, ##p## (and pr) is reserved for probabilities not densities. This may just be a convention, but I also see a lot people confuse densities with probabilities early on, so I'm fairly convinced that the convention is useful.
 

1. What is a reference prior?

A reference prior is a probability distribution that is used as a starting point for Bayesian inference. It represents the researcher's prior beliefs about the parameters of interest before any data is collected.

2. Why is it important to use reference priors for signal search?

Using reference priors for signal search allows for a more objective and transparent approach to data analysis. It ensures that the results are not influenced by subjective prior beliefs and helps to avoid biases in the analysis.

3. How are reference priors determined?

Reference priors can be determined using various methods such as maximum entropy, maximum likelihood, or Jeffreys' prior. The choice of method may depend on the specific research question and context.

4. Can reference priors be updated as more data is collected?

Yes, reference priors can be updated using Bayes' theorem as more data is collected. This allows for a more accurate representation of the researcher's beliefs and can improve the precision of the results.

5. How do reference priors differ from traditional prior distributions?

Traditional prior distributions are often chosen based on convenience or subjective beliefs, while reference priors are determined in a more objective and systematic manner. Reference priors also tend to be less informative and allow for more flexibility in the analysis.

Similar threads

  • Programming and Computer Science
Replies
2
Views
680
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
1K
  • Calculus and Beyond Homework Help
Replies
1
Views
633
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
  • Calculus and Beyond Homework Help
Replies
1
Views
547
  • High Energy, Nuclear, Particle Physics
Replies
1
Views
934
  • Set Theory, Logic, Probability, Statistics
Replies
6
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
Back
Top