Bayesian estimation via MCMC

In summary, the conversation discusses using a model M with adjustable parameters to predict experimental values and perform Bayesian parameter estimation. The speaker is looking for freely available MCMC software that can handle their specific approach, which involves solving complex systems of differential equations. They mention using R with MCMCpack but are unsure if it can handle their needs, and also ask for beginner-friendly resources on using MCMC techniques.
  • #1
witziger_Fuchs
2
0
Hi folks.

I have the following question.I have a model M containing 20 adjustable parameters k = {k_j}.
I also have 40-50 measured temporal profiles e = {e_i} at my disposal.

I can use M to predict the experimental values after solving complex systems of differential equations.Consequently, I get m(k) = {m_i(k)} which I can compare to e = {e_i}.Now, I want to perform a Bayesian parameter estimation of the system.I am going to define a (first) prior distribution for the parameters k: p_0(k)
Afterwards, I want to get the posterior probability distribution of k: f_p(k) = p(k|e) = L(e|k)*p_0(k)/p(e).
(Whereby p(e) represents, of course, a very complex multi-dimensional integral of "L(e|k)*p_0(k)".Naturally, I cannot compute analytically the solution.
It also stands to reason that an approximate calculation of f_p(k) (and integration of "L(e|k)*p_0(k)") would be computationally intractable.I read that Macrov-Chain-Monte-Carlo (MCMC) methods should be used for computing quantities of interest characterising the posterior (such as the points of highest probability density and high probability density regions, whose bounds can serve as error bars).
To be frank, I am a novice in that field. Do you know any MCMC software freely available to academic researchers which could carry out all these operations, given a "black box" m(k) relying on solving differential equation systems?
If so, are you also aware of any beginner-friendly introduction into the concrete application of these techniques?

I'd be very grateful for your answers.Kind regards.
 
Physics news on Phys.org
  • #2
I use R with the MCMCpack for my Bayesian estimation needs. It has always served me well, however I have never tried the kind of differential equation black box approach like you are describing. I have essentially only used it to do Bayesian linear regression were the residuals were assumed to be normally distributed and the Bayesian approach estimates the posterior of the linear model parameters and the residual variance.

I know that MCMCpack has built in routines for more complicated models, but I just haven't used them. You may need to try something more specialized, such as BUGS, but I have no experience with that.
 

1. What is the purpose of Bayesian estimation via MCMC?

Bayesian estimation via MCMC (Markov Chain Monte Carlo) is a statistical method used to estimate the parameters of a Bayesian model. It allows for the construction of posterior distributions of the parameters, which are used to infer the uncertainty of the estimated values. This method is particularly useful for complex models where traditional methods may not be feasible.

2. How does MCMC work?

MCMC is a simulation-based approach that uses a Markov chain to generate samples from the posterior distribution. The chain moves from one state to another based on a set of transition probabilities, and after a certain number of iterations, the samples will converge to the posterior distribution. These samples can then be used to estimate the parameters of the model.

3. What are some advantages of using Bayesian estimation via MCMC?

One of the main advantages of MCMC is its ability to handle complex models and data sets. It also allows for the incorporation of prior knowledge and uncertainty into the analysis. Additionally, MCMC provides a way to estimate the entire posterior distribution, rather than just a point estimate, which can be useful for decision making.

4. What are some potential challenges of using MCMC?

MCMC can be computationally intensive, especially for large data sets or complex models. It also requires careful tuning of parameters, such as the number of iterations and the proposal distribution, to ensure convergence and accurate results. Additionally, MCMC may not perform well when there are strong correlations between parameters in the model.

5. Are there any alternatives to MCMC for Bayesian estimation?

Yes, there are other methods for Bayesian estimation, such as variational inference and approximate Bayesian computation. These methods may be more efficient for certain types of models and data sets, but they also have their own limitations. MCMC remains a popular and well-established approach for Bayesian estimation due to its flexibility and ability to handle a wide range of problems.

Similar threads

MHB Bayesian parameter estimation via MCMC?
    • Set Theory, Logic, Probability, Statistics
Replies
1
Views
2K
I How to calculate expectation and variance of kernel density estimator?
    • Set Theory, Logic, Probability, Statistics
Replies
9
Views
2K
I Relation with Hessian and Log-likelihood
    • Set Theory, Logic, Probability, Statistics
Replies
12
Views
1K
Python Laplace approximation in Bayesian inference
    • Programming and Computer Science
Replies
1
Views
922
A Covariance of Posterior Predictive Distribution
    • Set Theory, Logic, Probability, Statistics
Replies
1
Views
964
MHB Uniformly Most Powerful Test for Weibull Distribution
    • Set Theory, Logic, Probability, Statistics
Replies
1
Views
868
A Finding Global Minima in Likelihood Functions
    • General Math
Replies
1
Views
1K
I Thoughts about this Bayesian Mechanics paper?
    • Quantum Interpretations and Foundations
Replies
3
Views
2K
Bernoulli and Bayesian probabilities
    • Calculus and Beyond Homework Help
Replies
4
Views
1K
A Understand the Importance of Reference Priors for Signal Search
    • Set Theory, Logic, Probability, Statistics
Replies
3
Views
1K

Hot Threads

Recent Insights

Back
Top