How to Express Non-regular Prior Distributions by Mathematical Formula

  • I
  • Thread starter nizi
  • Start date
  • Tags
    Support
In summary, the article discusses methods for expressing non-regular prior distributions within a mathematical framework. It highlights the importance of these distributions in Bayesian analysis, particularly when dealing with complex data structures or models that do not adhere to standard assumptions. The paper presents various mathematical techniques and transformations that can be employed to accurately represent non-regular priors, ensuring that they can be effectively integrated into statistical models. Key considerations include the implications for inference and the need for careful calibration to maintain the validity of the Bayesian approach.
  • #1
nizi
17
1
This time I target the following two-class Bayesian logistic regression as statistical models.

$$y_n \sim \mathrm{Bernoulli}(q_n)$$
$$q_n = \sigma (\beta_0 + x_n \beta_1)$$

where ##n## is the index of the data and ##\sigma## is the logistic function.

Since I assume ##\beta_0 + x_n \beta_1## as the linear predictor (the independent variable of the logistic activation function), I have two parameters ##\beta_0## and ##\beta_1##, and I want to express that their prior distributions independently follow a non-regular uniform distribution, the support of which each probability density function is the set of real numbers.
Is it appropriate to write

$$\beta_0, \beta_1 \sim \mathrm{Uniform}(-\infty, \infty), i.i.d.$$

in this case?
I'm particularly concerned about the appropriateness of writing i.i.d.
If there is a more appropriate way to express this, please let me know.
 
Last edited:
Physics news on Phys.org
  • #2
The iid is fine. But this prior will require some justification. The uniform distribution will be really bad here because it has no tails so the posteriors are almost guaranteed to be unreliable. And starting with this prior is numerically impossible. And approximating it will probably have poor convergence.

If you do use this prior you will need to provide rock-solid theoretical arguments why the priors must come from a uniform distribution and none other. And if you have that much information then surely the non-regular version is not correct.
 
  • Like
Likes WWGD
  • #3
I appreciate your thought-provoking response very much.
My question has been answered.
What I was particularly concerned about is whether I can use the term "i.i.d." here, even thought ##x_n## has certain units, e.g. kg.
This is because the units of ##\beta_0## and ##\beta_1## are different in this case.

As you mentioned, it's impossible to use this improper prior in a numerical simulation, so it's only an mathematical expression.
 
  • Like
Likes Dale

FAQ: How to Express Non-regular Prior Distributions by Mathematical Formula

What is a non-regular prior distribution?

A non-regular prior distribution is a type of prior distribution in Bayesian statistics that does not conform to the standard regularity conditions, such as smoothness or differentiability. These priors can include distributions with discontinuities, spikes, or other irregular features that make them challenging to handle with conventional mathematical tools.

Why would one use a non-regular prior distribution?

Non-regular prior distributions are used when the prior knowledge or beliefs about a parameter are inherently non-smooth or involve specific constraints. For instance, if prior information suggests that a parameter is highly likely to be within a narrow range or exactly at specific values, a non-regular prior can more accurately represent this information compared to a smooth, regular prior.

How can one express a non-regular prior distribution mathematically?

Non-regular prior distributions can be expressed using piecewise functions, Dirac delta functions, or indicator functions. For example, a prior that places all its mass at a single point can be expressed using a Dirac delta function, while a prior that is uniform over a specific interval and zero elsewhere can be represented using an indicator function.

What are some examples of non-regular prior distributions?

Examples of non-regular prior distributions include the spike-and-slab prior, which combines a Dirac delta function (spike) with a continuous distribution (slab), and uniform priors over bounded intervals with sharp cutoffs. Another example is a prior that assigns a non-zero probability to a finite set of discrete points and zero probability elsewhere.

What challenges arise when using non-regular prior distributions?

Non-regular prior distributions can pose several challenges, including difficulties in computation and integration, especially in the context of Bayesian inference. Standard numerical methods may not be directly applicable, and specialized techniques or approximations might be required. Additionally, ensuring that the resulting posterior distribution remains well-defined and interpretable can be more complex.

Similar threads

A Covariance of Posterior Predictive Distribution
Replies
1
Views
1K
I How to calculate expectation and variance of kernel density estimator?
Replies
9
Views
3K
I Some notes on stochastic physics
Replies
1
Views
1K
How Does Bayesian Estimation Determine Point Estimates with Prior Distributions?
Replies
3
Views
3K
How to Estimate the Intrinsic Distribution
Replies
5
Views
4K
I Micromass' big October challenge
4
Replies
125
Views
18K
Challenge Math Challenge - April 2020
5
Replies
150
Views
18K
Show convergence of product of n uniform distributions
Replies
1
Views
3K
Mathematica How to type mathematical equations using LaTeX
Replies
3
Views
1K
Programs How good is this applied math program?
Replies
13
Views
2K

Hot Threads

Recent Insights

Back
Top