Finding Global Minima in Likelihood Functions

In summary: Other than that, there are many global optimizers that can be used but for a problem like this, you can also consider using Bayesian optimization methods. These methods can provide a posterior distribution instead of just a point estimate, which can be helpful in some cases. However, if the function is computationally expensive, you may want to consider other options such as MCMC sampling optimization. Keep in mind that the number of parameters may increase in the future, so it's important to choose an optimization method that can handle higher dimensions.
  • #1
tworitdash
107
26
I have a likelihood function that has one global minima, but a lot of local ones too. I attach a figure with the likelihood function in 2D (it has two parameters). I have added a 3D view and a surface view of the likelihood function. I know there are many global optimizers that can be used to obtain the location of the global minimum point in the likelihood function. However, I want to know what basic optimizer principles that I can use (that I can also derive and implement myself) for a problem like this. If you see the 3D view, you may find many local minima. I am also open to suggestions that involve Bayesian type of optimization where I will get a posterior and not just a point estimate. I am open to that as well. I have tried MCMC type sampling optimization, however, they are computationally expensive. The number of parameters may increase later.
 

Attachments

  • cKJT2.png
    cKJT2.png
    19.1 KB · Views: 78
  • 5KeQE.png
    5KeQE.png
    29.2 KB · Views: 79
Mathematics news on Phys.org
  • #2
Is it literally just this function you want to optimize?

You already did it, by drawing a graph. More formally if that's unsatisfying, for low dimensions and fast evaluation functions you can just evaluate the function at every point on a fine grid and pick the point with the best value. If you want a little extra precision you can run any optimizer from there to find the local extremum near that point.
 
  • Like
Likes WWGD and tworitdash

1. What is a global minimum in a likelihood function?

A global minimum in a likelihood function is the lowest possible value that the function can take across all possible values of the parameters. It represents the best fit for the data and is the most likely set of parameter values that generated the observed data.

2. Why is finding global minima important in likelihood functions?

Finding global minima is important in likelihood functions because it allows us to determine the most likely set of parameter values that generated the observed data. This is crucial for making accurate predictions and understanding the underlying mechanisms behind a phenomenon.

3. How is the global minimum different from a local minimum in a likelihood function?

A global minimum is the overall lowest value of a likelihood function, whereas a local minimum is the lowest value within a specific range of parameter values. Local minima can occur in likelihood functions with multiple peaks, but only the global minimum represents the best fit for the data.

4. What methods are commonly used to find global minima in likelihood functions?

There are several methods commonly used to find global minima in likelihood functions, including gradient descent, simulated annealing, and genetic algorithms. These methods vary in complexity and effectiveness, and the choice of method often depends on the specific problem and the available computing resources.

5. Can global minima in likelihood functions be guaranteed to be found?

No, there is no guarantee that global minima in likelihood functions can be found. In fact, it is often impossible to know if the global minimum has been found or if there may be an even lower value that has not yet been discovered. This is why multiple methods and careful analysis are often necessary to ensure the best possible fit for the data.

Similar threads

I Gradient descent algorithm and optimizers
    • General Math
Replies
5
Views
843
Solve the Global Minima Problem in Two Variable Functions
    • Calculus and Beyond Homework Help
Replies
3
Views
996
I What Does the Likelihood Function Tell Us?
    • Set Theory, Logic, Probability, Statistics
Replies
16
Views
1K
A Fisher information from likelihood function for quantum circuits
    • Quantum Physics
Replies
0
Views
283
MHB Finding MLE for $\theta$ of Statistical Product Model
    • Set Theory, Logic, Probability, Statistics
Replies
6
Views
1K
I Finding global minima of nth degree polynomials
    • Calculus
Replies
2
Views
808
A Correlation between parameters in a likelihood fit
    • Set Theory, Logic, Probability, Statistics
Replies
3
Views
903
Finding Absolute Minima & Maxima of a Function
    • Calculus and Beyond Homework Help
Replies
8
Views
1K
Find global minimum of f(x1,x2, xn), with many local minima
    • General Math
Replies
1
Views
2K
I Optimization problem with multiple outputs: impossible?
    • General Math
Replies
6
Views
1K

Hot Threads

Recent Insights

Back
Top