I How to Express Non-regular Prior Distributions by Mathematical Formula

  • I
  • Thread starter Thread starter nizi
  • Start date Start date
  • Tags Tags
    Support
AI Thread Summary
The discussion centers on expressing non-regular prior distributions for parameters in a two-class Bayesian logistic regression model. The proposed prior for the parameters β0 and β1 is a uniform distribution over all real numbers, but concerns are raised about its appropriateness due to potential issues with reliability and convergence in posterior estimates. It is noted that using such a prior necessitates strong theoretical justification, as the uniform distribution lacks tails. The term "i.i.d." is questioned in the context of differing units for the parameters, highlighting the complexity of the model. Ultimately, while the mathematical expression is acknowledged, its practical application in numerical simulations is deemed problematic.
nizi
Messages
17
Reaction score
1
This time I target the following two-class Bayesian logistic regression as statistical models.

$$y_n \sim \mathrm{Bernoulli}(q_n)$$
$$q_n = \sigma (\beta_0 + x_n \beta_1)$$

where ##n## is the index of the data and ##\sigma## is the logistic function.

Since I assume ##\beta_0 + x_n \beta_1## as the linear predictor (the independent variable of the logistic activation function), I have two parameters ##\beta_0## and ##\beta_1##, and I want to express that their prior distributions independently follow a non-regular uniform distribution, the support of which each probability density function is the set of real numbers.
Is it appropriate to write

$$\beta_0, \beta_1 \sim \mathrm{Uniform}(-\infty, \infty), i.i.d.$$

in this case?
I'm particularly concerned about the appropriateness of writing i.i.d.
If there is a more appropriate way to express this, please let me know.
 
Last edited:
Physics news on Phys.org
The iid is fine. But this prior will require some justification. The uniform distribution will be really bad here because it has no tails so the posteriors are almost guaranteed to be unreliable. And starting with this prior is numerically impossible. And approximating it will probably have poor convergence.

If you do use this prior you will need to provide rock-solid theoretical arguments why the priors must come from a uniform distribution and none other. And if you have that much information then surely the non-regular version is not correct.
 
I appreciate your thought-provoking response very much.
My question has been answered.
What I was particularly concerned about is whether I can use the term "i.i.d." here, even thought ##x_n## has certain units, e.g. kg.
This is because the units of ##\beta_0## and ##\beta_1## are different in this case.

As you mentioned, it's impossible to use this improper prior in a numerical simulation, so it's only an mathematical expression.
 

Similar threads

Replies
1
Views
2K
Replies
5
Views
4K
Replies
125
Views
19K
4
Replies
150
Views
19K
Replies
3
Views
2K
Replies
13
Views
3K
Replies
4
Views
4K
Back
Top