How to Express Non-regular Prior Distributions by Mathematical Formula

  • Context: Undergrad 
  • Thread starter Thread starter nizi
  • Start date Start date
  • Tags Tags
    Support
Click For Summary
SUMMARY

This discussion focuses on expressing non-regular prior distributions in Bayesian logistic regression models, specifically using the uniform distribution for parameters β₀ and β₁. The participants agree that while the notation β₀, β₁ ∼ Uniform(-∞, ∞), i.i.d. is mathematically valid, it poses significant challenges in practical applications due to the uniform distribution's lack of tails, leading to unreliable posteriors. The consensus emphasizes the necessity of providing robust theoretical justification for using such priors, as improper priors complicate numerical simulations and convergence.

PREREQUISITES
  • Understanding of Bayesian statistics and logistic regression
  • Familiarity with prior distributions and their implications
  • Knowledge of the logistic function and its application in regression models
  • Experience with statistical simulation techniques
NEXT STEPS
  • Research the implications of using improper priors in Bayesian analysis
  • Explore alternative prior distributions for Bayesian logistic regression
  • Learn about the convergence issues in Bayesian numerical simulations
  • Study the theoretical foundations of uniform distributions in statistical modeling
USEFUL FOR

Statisticians, data scientists, and researchers involved in Bayesian modeling, particularly those working with logistic regression and prior distribution selection.

nizi
Messages
17
Reaction score
1
This time I target the following two-class Bayesian logistic regression as statistical models.

$$y_n \sim \mathrm{Bernoulli}(q_n)$$
$$q_n = \sigma (\beta_0 + x_n \beta_1)$$

where ##n## is the index of the data and ##\sigma## is the logistic function.

Since I assume ##\beta_0 + x_n \beta_1## as the linear predictor (the independent variable of the logistic activation function), I have two parameters ##\beta_0## and ##\beta_1##, and I want to express that their prior distributions independently follow a non-regular uniform distribution, the support of which each probability density function is the set of real numbers.
Is it appropriate to write

$$\beta_0, \beta_1 \sim \mathrm{Uniform}(-\infty, \infty), i.i.d.$$

in this case?
I'm particularly concerned about the appropriateness of writing i.i.d.
If there is a more appropriate way to express this, please let me know.
 
Last edited:
Physics news on Phys.org
The iid is fine. But this prior will require some justification. The uniform distribution will be really bad here because it has no tails so the posteriors are almost guaranteed to be unreliable. And starting with this prior is numerically impossible. And approximating it will probably have poor convergence.

If you do use this prior you will need to provide rock-solid theoretical arguments why the priors must come from a uniform distribution and none other. And if you have that much information then surely the non-regular version is not correct.
 
  • Like
Likes   Reactions: WWGD
I appreciate your thought-provoking response very much.
My question has been answered.
What I was particularly concerned about is whether I can use the term "i.i.d." here, even thought ##x_n## has certain units, e.g. kg.
This is because the units of ##\beta_0## and ##\beta_1## are different in this case.

As you mentioned, it's impossible to use this improper prior in a numerical simulation, so it's only an mathematical expression.
 
  • Like
Likes   Reactions: Dale

Similar threads

  • · Replies 9 ·
Replies
9
Views
3K
  • · Replies 1 ·
Replies
1
Views
2K
Replies
3
Views
3K
  • · Replies 5 ·
Replies
5
Views
4K
  • · Replies 125 ·
5
Replies
125
Views
20K
  • · Replies 150 ·
6
Replies
150
Views
20K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 13 ·
Replies
13
Views
3K
  • · Replies 4 ·
Replies
4
Views
5K
  • · Replies 13 ·
Replies
13
Views
10K