Correlation between parameters in a likelihood fit

[Aleolomorfo]

TL;DR Summary

Which is the right way to estimate the correlation between parameters estimated with a likelihood function?

Hello community!

I am facing a conceptual problem with the correlation matrix between maximum likelihood estimators.

I estimate two parameters (their names are SigmaBin0 and qqzz_norm_0) from a multidimensional likelihood function, actually the number of parameters are larger than the two I am focusing my attention now. I need to evaluate the correlation between that two parameters.
I know the best way to evaluate the correlation between two parameters from a likelihood fit is starting from the Hessian matrix. Performing that method I get a correlation of -0.14.

Then I tried a different approach: studying the SigmaBin0 vs qqzz_norm_0 values when one of the parameters is the Parameters Of Interest (POI) and the other is profiled, and viceversa. I mean, I scan the likelihood function along SigmaBin0 while "watching" the qqzz_norm_0 profiled values, and then I run another scan along qqzz_norm_0 while "watching" the SigmaBin0 profiled values. My expectation is to find the same trend in both cases, but what I find is the right plot in attachment. The vertical line is the former case, instead the horizontal line is the latter one. If I apply the definition of correlation (ratio of the covariance wrt the standard deviations) I get a correlation of -0.29.
On the other hand, if I consider the two trends independently and apply again the definition of correlation I got a correlation of -1 (central and left plot in attachment).

In my mind all these approaches should be equivalent and giving the same value of the correlation, but it is not the case. So there might be some bug in my reasoning. Can someone help me to sort out my ideas, please?

 

[Dale]

Have you thought about using a Bayesian approach? With a Bayesian approach you get a posterior sample of the joint distribution of all of your parameters. So you can just directly calculate the correlation.

That said, I am not sure what you mean by:

I scan the likelihood function along SigmaBin0 while "watching" the qqzz_norm_0 profiled values

So I cannot really assess what problems you run into. But this procedure does not seem clear to me so I am not surprised that it gives different results than the Hessian approach.

 

[Stephen Tashi]

I mean, I scan the likelihood function along SigmaBin0 while "watching" the qqzz_norm_0 profiled values, and then I run another scan along qqzz_norm_0 while "watching" the SigmaBin0 profiled values.

It isn't clear to me what you mean by "scan" and "profile". These are not standard terms in statistics although they may be familiar to people in your particular field of study.

I am facing a conceptual problem with the correlation matrix between maximum likelihood estimators.

For a "correlation between two estimators" to make sense, we have to define the estimators as random variables. We could imagine that you have multiple independent data sets $D_1,D_2,D_3,...$ and from each data set $D_i$ we get one pair of maxiumum likihood estimators $(S_i, q_i)$ (estimating different two parametrs). Then we can regard these pairs of values as random samples from two random variables. Is that what you are doing?

 

[Dale]

For a "correlation between two estimators" to make sense, we have to define the estimators as random variables.

I agree. That is another reason to use the Bayesian approach here. This information is very natural and easy to obtain in that approach.