What is Maximum likelihood: Definition and 64 Discussions

In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of a probability distribution by maximizing a likelihood function, so that under the assumed statistical model the observed data is most probable. The point in the parameter space that maximizes the likelihood function is called the maximum likelihood estimate. The logic of maximum likelihood is both intuitive and flexible, and as such the method has become a dominant means of statistical inference.If the likelihood function is differentiable, the derivative test for determining maxima can be applied. In some cases, the first-order conditions of the likelihood function can be solved explicitly; for instance, the ordinary least squares estimator maximizes the likelihood of the linear regression model. Under most circumstances, however, numerical methods will be necessary to find the maximum of the likelihood function.
From the vantage point of Bayesian inference, MLE is a special case of maximum a posteriori estimation (MAP) that assumes a uniform prior distribution of the parameters. In frequentist inference, MLE is a special case of an extremum estimator, with the objective function being the likelihood.

View More On Wikipedia.org
Recent contents Top users
  1. A

    A Constructing PDFs for Max Likelihood Density Estimation Problem

    I have the following constrained optimization problem corresponding to the maximum likelihood density estimation: $$ \begin{aligned} &\text{maximize} && L(f) \\ &\text{subject to} && f \in H \\ &&& \int_a^b f(x) \mathop{}\!\mathrm{d} x = 1 \\ &&& f(x) \geq 0...
  2. X

    A Minimum-variance bound for the extended maximum likelihood estimation

    I am fitting a mass spectrum using pdf(M)=Ns×S(M)+Nb×B(M; a, b) to determine the yield with the extended maximum likelihood fit, where Ns and Nb are the number of signal and background events, S(M) is the function for the signal, B(M;a, b) is the function for the background with parameters a and...
  3. M

    I Maximum likelihood to fit a parameter of this model

    Hi PF! Given random time series data ##y_i##, we assume the data follows a EWMA (exponential weighted moving average) model: ##\sigma_t^2 = \lambda\sigma_{t-1}^2 + (1-\lambda)y_{t-1}^2## for ##t > 250##, where ##\sigma_t## is the standard deviation, and ##\sigma_{M=250}^2 =...
  4. F

    A Try to swap between mean and partial derivatives on a product

  5. ORF

    Binned Maximum Likelihood fit in python?

    Hi, I have been using Python for a while now, but so far for Least-squares fits using curve_fit from Scipy. I would like to start using Likelihood method to fit binned and unbinned data. I found some documentation in Scipy of how to implement unbinned likelihood fit, but I have not managed to...
  6. F

    MHB Max Likelihood: True/False? w/Proofs

    Can someone help please? are these statements true or false? With some proofs.
  7. F

    MHB Maximum Likelihood: True/False + Proves | Help

    Is this statement true or false with some proofs? Please help
  8. A

    A Correlation between parameters in a likelihood fit

    Hello community! I am facing a conceptual problem with the correlation matrix between maximum likelihood estimators. I estimate two parameters (their names are SigmaBin0 and qqzz_norm_0) from a multidimensional likelihood function, actually the number of parameters are larger than the two I am...
  9. B

    Python Minimization likelihood function with parameters

    Hallo at all! I'm learning statistic in python and I have a problem to show you. I have this parametric function: $$P(S|t, \gamma, \beta)=\langle s(t) \rangle \left( \frac{\gamma-\beta}{\gamma\langle s(t) \rangle -\beta}\right)^2\left( 1- \frac{\gamma-\beta}{\gamma\langle s(t) \rangle...
  10. F

    I Understanding 2 equivalent formulations of both data set measures

    I have two independant experiments have measured ##\tau_{1},\sigma_{1}## and ##\tau_{2},\sigma_{2}## with ##\sigma_{i}## representing errors on measures. From these two measures, assuming errors are gaussian, we want to get the estimation of Ï and its error (i.e with a combination of two...
  11. C

    MHB Log likelihood and Maximum likelihood

    I'm not sure how to get this first derivative (mainly where does the 4 come from?) I know x̄ is the sample mean (which I think is 1/2?) Can someone suggest where to start with finding the log-likelihood? I know the mass function of a binomial distribution is: Thanks!
  12. W

    A PCA and Maximum Likelihood?

    Hi, I am looking into a text on PCA obtained through path diagrams ( a diagram rep of the relationship between factors and the dependent and independent variables) and correlation matrices . There is a "reverse" exercise in which we are given a correlation matrix there is mention of the use of...
  13. F

    I Relation with Hessian and Log-likelihood

    I would like to demonstrate the equation (1) below in the general form of the Log-likelihood : ##E\Big[\frac{\partial \mathcal{L}}{\partial \theta} \frac{\partial \mathcal{L}^{\prime}}{\partial \theta}\Big]=E\Big[\frac{-\partial^{2} \mathcal{L}}{\partial \theta \partial...
  14. S

    MHB Maximum Likelihood Estimation Formula Simplification and Solution for p?

    Hey guys ! My mother language is not English by the way. Sorry for spelling and gramme. :) I'm curious to see if you can help me with my problem.I have already tried for almost a week and did not get to a solution. I also know, that the Maximum likelihood estimation is part of statistics and...
  15. F

    I Maximum Likelihood to find the original data or estimation directly

    I make confusions in the using of Maximum Likelihood to find (approximately) the original signal knowing the data observed and the using of Maximum Likelihood to find estimations of parameters of the PSF 1) Find (up to some point) the original signal : I start from this general definition (in...
  16. M

    MHB Finding the Maximum Likelihood Estimator for a Density Function

    Hey! :o We have the density function $f_x(x)=\frac{2c^2}{x^3}, x\geq 0, c\geq 0$. I want to calculate the maximum Likelihood estimator for $c$. We have the Likelihood Function $$L(c)=\prod_{i=1}^nf_{X_i}(x_i;c)=\prod_{i=1}^n\frac{2c^2}{x_i^3}$$ The logarithm of the Likelihood function is...
  17. Avatrin

    I Why is the maximum likelihood estimation accurate?

    Hi I've been googling maximum likelihood estimation. While I do understand how to compute it, I don't understand why maximizing the likelihood function will give us a good estimate of the actual parameter. In some cases, like the normal distribution, it seems almost obvious. However, in the...
  18. F

    I Maximum likelihood w/ histogram, zero probability samples

    I'm trying to replicate a machine learning experiment in a paper.The experiment used several signal generators and "trains" a system to recognize the output from each one. They way this is do is by sampling the output of each generator, and then building histograms from each trial. Later you...
  19. T

    Maximum likelihood of a statistical model

    Homework Statement I look at the distribution ##(Y_1,Y_2,...,Y_n)## where ##Y_i=μ+(1+φ x_i)+ε_i## where ##-1<φ<1## and ##-1<x_i<1## . x's are known numbers. ε's are independent and normally distributed with mean 0 and variance 1. I need to find the the maximum likelihood estimator for μ and...
  20. pluviosilla

    I Why is the Maximum Likelihood Function a product?

    Why is the Maximum Likelihood function a product? Explanations of how the Maximum Likelihood function is constructed usually just mention that events are independent and so the probability of several such events is just the product of the separate probabilities. I get the logic w.r.t...
  21. D

    Maximum Likelihood and Fisher Information

    Homework Statement Let X1, X2,...Xn be a random sample from pdf, f(x|θ) = θx-2 where 0 < θ ≤ x < ∞ Find the MLE of θMy attempt: Likelihood fxn: L(θ|x) = ∏θx-2 = θn∏ θx-2 And to find MLE, I take Log of that function and partial derivative (w.r.t θ, of log L(θ|x) and set that = 0, and get...
  22. D

    Find MLE of θ: Maximizing Likelihood fxn

    Homework Statement Let X1, X2,...Xn be a random sample from pdf, f(x|θ) = θx-2 where 0 < θ ≤ x < ∞ Find the MLE of θMy attempt: Likelihood fxn: L(θ|x) = ∏θx-2 = θn∏ θx-2 And to find MLE, I take Log of that function and partial derivative (w.r.t θ, of log L(θ|x) and set that = 0, and get...
  23. Julio1

    MHB Maximum Likelihood Estimators for Uniform Distribution

    Find maximum likelihood estimators of an sample of size $n$ if $X\sim U(0,\theta].$ Hello MHB :)! Can any user help me please :)! I don't how follow...
  24. D

    MHB What is the Maximum Likelihood Estimator for Uniform Distribution Endpoints?

    I need help on this problem, anyone know how to do it? Suppose you have n independent observations from a uniform distribution over the interval [𝜃1, 𝜃2]. a. Find the maximum likelihood estimator for each of the endpoints θ1 and θ2. b. Based on your result in part (a), what would you expect...
  25. F

    MHB Show that the maximum likelihood estimator is unbiased

    Consider a density family $f(x,{\mu})=c_{{\mu}}x^{{\mu}-1}\exp(\frac{-(\ln(x))^2)^2}{2}$ , where $c_{{\mu}}=\frac{1}{{\sqrt{2{\pi}}}}\exp(-{\mu}^2/2)$ For a sample $(X_{1},...,X_{n})$ fnd the maximum likelihood estimator and show it is unbiased. You may find the substitution $y=\ln x$ helpful...
  26. P

    Finding maximum likelihood estimator

    Homework Statement The independent random variables X_1, ..., X_n have the common probability density function f(x|\alpha, \beta)=\frac{\alpha}{\beta^{\alpha}}x^{\alpha-1} for 0\leq x\leq \beta. Find the maximum likelihood estimators of \alpha and \beta. Homework Equations log...
  27. I

    Maximum Likelihood Estimator

    Homework Statement 1. An experiment consists of giving a sequences of patients a risky treatment until two have died, and then recording N, the number who survived. If p is the proportion killed by the treatment, then the distribution of N is: P(N=n)=(n+1)(1-p)n p2 1)Find a general formula...
  28. S

    Question to Maximum Likelihood

    Hello, I have a question to Maximum Likelihood Estimation. The typical form of MLE looks like this: X = Hθ + W. W is gaussion with N(0, C). θml = (HTC-1H)-1HTC-1X I think θml can only be calculated after a lot of measurements are made, that is, there are plenty of samples of H and X...
  29. G

    Maximum Likelihood Estimator Problem

    Homework Statement 1. Suppose the data consist of a single number X, and the model is that X has the following probability density: f(x|θ) =(1+ xθ)/2 for -1≤ x ≤1; =0 otherwise. Supposing the possible values of θ are 0 ≤ θ ≤ 1; find the maximum likelihood estimate (MLE) of θ, and find...
  30. Y

    MLE Homework: Distance Measurement with Odometer Error

    Homework Statement John wants to measure the distance from his home to his office, so he drives to work several times and measures the distance on his car's odometer. Unfortunately, the odometer records distance only to the nearest mile. (Johns odometer changes abruptly from one digit to the...
  31. S

    Maximum Likelihood Estimator + Prior

    1.Suppose that X~B(1,∏). We sample n times and find n1 ones and n2=n-n1zeros a) What is ML estimator of ∏? b) What is the ML estimator of ∏ given 1/2≤∏≤1? c) What is the probability ∏ is greater than 1/2? d) Find the Bayesian estimator of ∏ under quadratic loss with this prior 2. The attempt at...
  32. S

    How to Find Maximum Likelihood Estimators for Sample Data?

    Homework Equations L(x,p) = \prod_{i=1}^npdf l= \sum_{i=1}^nlog(pdf) Then solve \frac{dl}{dp}=0 for p (parameter we are seeking to estimate) The Attempt at a Solution I know how to do this when we are given a pdf, but I'm confused how to do this when we have a sample.
  33. P

    Comparing Least Squares and Maximum Likelihood?

    Hi, Below is my attempt at a comparison between the two above-mentioned methods of estimation. Does anything in the table lack in validity and/or accuracy? Should any properties, advantages/disadvantages be eked out? Any suggestions/comments would be most appreciated! MLE...
  34. K

    MLE of μ for X1,...Xn with Known σ2i

    is it possible to estimate all parameters of an n-observation (X1,...Xn) with same mean, μ, but different variances (σ21,σ22,...,σ2n)? if we assume that σ2i are known for all i in {1,...n}, what is the mle of of μ?
  35. G

    Finding maximum likelihood function for 2 normally distributed samples

    Homework Statement The question is about how to combine to different samples done with 2 different methods of the same phenomena. Method 1 gives normally distributed variables X_1,X_2,...X_{n_1}, with \mu and \sigma^2_1 Method 1 gives normally distributed variables Y_1,Y_2,...,Y_{n_2}...
  36. B

    Maximum Likelihood Estimator, Single Observation

    Homework Statement An observation X has density function: f(x,/theta)=6x/(t^3)*(t-x) where t is a parameter: 0<x<t. Given the single observation X, determine the maximum likelihood estimator for t. Homework Equations Included below The Attempt at a SolutionFor a sample size of n...
  37. B

    Maximum Likelihood Estimator

    Homework Statement An observation X has density function: f(x,/theta)=6x/(t^3)*(t-x) where t is a parameter: 0<x<t. Given the single observation X, determine the maximum likelihood estimator for t. Homework Equations Included below The Attempt at a Solution For a sample size of...
  38. C

    Find Maximum Likelihood Estimator of Gamma Distribution

    Given f(x; β) = [ 1/( β^2) ] * x * e^(-x/ β) for 0 < x < infinity EX = 2β and VarX = 2(β^2) Questions: Find the Maximum likelihood estimator of β (I call it β''), then find Bias and variance of this β'' 1/ First, I believe this is a gamma distribution with alpha = 2. Is that right? 2/...
  39. K

    Statistical Analysis - Maximum Likelihood Fit

    Homework Statement I have a set of data from the DAMA experiment in which a detector attempted to measure collisions with 'WIMP's [Weakly Interacting Massive Particles] as a candidate for dark matter. The detector records the time in days of a collision event. After binning the data and...
  40. S

    Linear regression and maximum likelihood estimates

    Homework Statement Suppose that data (x1,y1),(x2,y2),.?.,(xn,yn) is modeled with xi being non random and Yi being observed values of random variables Y1,Y2,...Yn which are given by Yi = a + b(xi-xbar) + σεi Where a, b, σ are unknown parameters and εi are independent random variables each...
  41. P

    Maximum Likelihood Estimate

    I actually have two questions, both of which are on the same topic Homework Statement Consider X = number of independent trials until an event A occurs. Show that X has the probability mass function f(x) = (1-p)^x p, x = 1,2,3,..., where p is the probability that A occurs in a single trial...
  42. P

    Maximum Likelihood Estimator Question

    Homework Statement A bag contains sequentially numbered lots (1,2...N). Lots are drawn at random (each lot has the same probability of being drawn). Two lots are drawn without replacement and are observed to be X_1 = 17 and X_2 = 30. What is the MLE of N, the number of lots in a bag...
  43. P

    Maximum Likelihood Estimator Question

    Homework Statement Lifetimes of components are Gamma distributed. The parameters of the Gamma are shape = a scale = λ The pdf is: f(x) = (λ^a).x^(a-1).e^(-λx)/Γ(a) In this case, it is known that a = 3. Obtain the MLE of λ. Homework Equations The Attempt at a Solution Hi...
  44. S

    Maximum likelihood estimator and UMVUE

    Homework Statement Let X_{1}, ... , X_{n} be a random sample from f\left(x; \theta\right) = \theta x^{\theta - 1} I_{(0, 1)}\left(X\right), where \theta > 0. a. Find the maximum-likelihood estimator of \theta/\left(1 + \theta\right). b. Is there a function of \theta for which there...
  45. T

    Obtain maximum likelihood estimates

    Problem : It is assumed that the arrival of the number of calls X per hour follow a poisson distribution with parameter \lambda, A random sample X1 = x1 , X2 = x2,...xn is taken .Obtain the maximum likelihood estimates of the average arrival rate. Please can you provide me with some...
  46. T

    Maximum Likelihood Estimator of Bin(m,p)

    Homework Statement Let \displaystyle X_1 ,..., X_n \stackrel {\text{i.i.d.}}{\sim} \text{Bin} (m,p) where m is known. Find the MLE \hat{p}_n of p and hence show that \hat{p}_n is unbiased. The Attempt at a Solution Can anyone check my attempt please? \displaystyle L(m,p) =...
  47. S

    Maximum likelihood estimator of mean difference

    Homework Statement A sample of size n_{1} is to be drawn from a normal population with mean \mu_{1} and variance \sigma^{2}_{1}. A second sample of size n_{2} is to be drawn from a normal population with mean \mu_{2} and variance \sigma^{2}_{2}. What is the maximum likelihood estimator of...
  48. Y

    Maximum Likelihood Estimator for a function

    Homework Statement Consider the following density function: f(x) = ABB/xB+1; A<= x, zero elsewhere, where A > 0 and B> 0 Homework Equations The Attempt at a Solution f(x1,...,xn)= ABnBn(x1...xn)B+1 ln f(x1,...,xn)= Bn ln A + n ln B + (B+1)ln((x1...xn) After differentiating...
  49. A

    Expectation of an Uniform distribution maximum likelihood estimator

    Hi had this question on my last "Statistical Inference" exam. And I still have some doubts about it. I determined that the maximum likelihood estimator of an Uniform distribution U(0,k) is equal to the maximum value observed in the sample. That is correct. So say my textbooks. After that the...
  50. R

    Find ξ Value with Maximum Likelihood

    http://bbs.mathchina.com/usr1PvjRWKew/4/22/graph_1261406609.jpg I can't understand the equation, may I know is there any good idea to get the value of ξ?
Back
Top