How To Do Fisher Forecasting For Constraining Parameters

In summary: To take one example, if you make use of supernova data, and allow the absolute magnitude of the supernovae to be a free parameter, then you cannot measure ##H_0## from the supernova data at all. You can get ##H_0## only if you combine the supernova data with something else (such as an estimate of the absolute magnitude of the supernovae, or of ##H_0## from a different source such as the CMB).In summary, the Fisher Forecast is a method used in experimental physics to determine the uncertainties of parameters from a set of observables. It involves
  • #1
xdrgnh
417
0
I'm doing a Fisher Forecast to satisfy my experimental physics requirement. I never done anything like this before so I'm kind of in the dark of how to proceed. I have done some readings on Fisher Forecasting and would like to outline how I would do it and then ask if you can tell me if I'm right.

  1. I have a list of observables and how they vary with parameters I want to know the errors bars for.
  2. I also have a list of upcoming experiments which can constrain these parameters with there expected noise levels and instrumental uncertainty.
  3. I have covariant matrices from experiments done in the past that constrain these parameters.
  4. I then form my fisher matrix with the following equation
    q94SK.png

  1. Once I have those matrices from all of my experiments for each parameter I then invert those matrices, add them onto the covariant matrices from past experiments. I then invert that to get my Fisher Information Matrix
  2. In order to find the uncertainties for each of my parameters once I have the Fisher Information Matrix I compute
    D6Szb.png
    where the 11 can be ij and the all I need to worry about are the diagonal elements of this inverse.
And then I'm done I think. Can anyone tell me if it is really this simple or am I missing a whole bunch of stuff. I never did data analysis before and I need to be fully done by the end of this month.

Thanks you
 
Space news on Phys.org
  • #2
Oh and the model in equation is a modified friedmann equation and I'm assuming all of my errors are gaussian in nature.
 
  • #3
xdrgnh said:
I'm doing a Fisher Forecast to satisfy my experimental physics requirement. I never done anything like this before so I'm kind of in the dark of how to proceed. I have done some readings on Fisher Forecasting and would like to outline how I would do it and then ask if you can tell me if I'm right.

  1. I have a list of observables and how they vary with parameters I want to know the errors bars for.
  2. I also have a list of upcoming experiments which can constrain these parameters with there expected noise levels and instrumental uncertainty.
  3. I have covariant matrices from experiments done in the past that constrain these parameters.
  4. I then form my fisher matrix with the following equationView attachment 205166
  1. Once I have those matrices from all of my experiments for each parameter I then invert those matrices, add them onto the covariant matrices from past experiments. I then invert that to get my Fisher Information Matrix
  2. In order to find the uncertainties for each of my parameters once I have the Fisher Information Matrix I compute View attachment 205167 where the 11 can be ij and the all I need to worry about are the diagonal elements of this inverse.
And then I'm done I think. Can anyone tell me if it is really this simple or am I missing a whole bunch of stuff. I never did data analysis before and I need to be fully done by the end of this month.

Thanks you
Right, the diagonal elements of the inverse of the Fisher information matrix are the uncertainties on the individual variables. This is because the inverse of the Fisher Information Matrix is the Covariance matrix, whose diagonal elements are just the variances.

In practice you have to be careful with this kind of calculation, because if your Fisher information matrix has any really small eigenvalues it can make the calculations unstable. Typically this means that you are using a model with more variables than your experiment can constrain. There are ways to deal with this, but it's always a bit finicky.
 
  • Like
Likes xdrgnh
  • #4
kimbyd said:
Right, the diagonal elements of the inverse of the Fisher information matrix are the uncertainties on the individual variables. This is because the inverse of the Fisher Information Matrix is the Covariance matrix, whose diagonal elements are just the variances.

In practice you have to be careful with this kind of calculation, because if your Fisher information matrix has any really small eigenvalues it can make the calculations unstable. Typically this means that you are using a model with more variables than your experiment can constrain. There are ways to deal with this, but it's always a bit finicky.
I'm trying to constrain 5 parameters and I will be doing all of this on Mathematica. In total from past experiments I have roughly 600 data points. From your experience how long does it take to perform a Fisher Forecast?
 
  • #5
xdrgnh said:
I'm trying to constrain 5 parameters and I will be doing all of this on Mathematica. In total from past experiments I have roughly 600 data points. From your experience how long does it take to perform a Fisher Forecast?
In terms of calculation time, it should take a couple of seconds at the most (likely much less). All of the time will be spent on figuring out precisely how to do it, and getting it to work.

The number of data points isn't as important as what those data points do to the uncertainties. You could have ten million data points and there could still be a problem if you have variables (or a combination of variables) that aren't affected by the data.

To take one example, if you make use of supernova data, and allow the absolute magnitude of the supernovae to be a free parameter, then you cannot measure ##H_0## from the supernova data at all. You can get ##H_0## only if you combine the supernova data with something else (such as an estimate of the absolute magnitude of the supernovae, or of ##H_0## from a different source such as the CMB).
 
  • Like
Likes xdrgnh
  • #6
kimbyd said:
In terms of calculation time, it should take a couple of seconds at the most (likely much less). All of the time will be spent on figuring out precisely how to do it, and getting it to work.

The number of data points isn't as important as what those data points do to the uncertainties. You could have ten million data points and there could still be a problem if you have variables (or a combination of variables) that aren't affected by the data.

To take one example, if you make use of supernova data, and allow the absolute magnitude of the supernovae to be a free parameter, then you cannot measure ##H_0## from the supernova data at all. You can get ##H_0## only if you combine the supernova data with something else (such as an estimate of the absolute magnitude of the supernovae, or of ##H_0## from a different source such as the CMB).
Well I'm using Supernova data, GRB data, BAO data and CMB data. I haven't begun yet because my adviser first wants me to outline to her the Fisher Forecast method. Once she thinks I understand it well enough she will give me papers for upcoming experiments and then I'll be on my way to Fisher Forecasting Tuesday of this upcoming week. Those variables that are not effected by the data, are those the nuisance parameters you are supposed to marginalize?
 
  • #7
That sounds like a pretty good combination of data that should work pretty well to constrain most cosmological parameters.

As for nuisance parameter, I think you're misunderstanding slightly. The absolute magnitude parameter is a nuisance parameter, because it's not one that helps you understand the expansion of the universe that's being measured. And yes, you have to marginalize over it.

What I was describing, however, was a degeneracy in the data. And while I described a degeneracy that was due to a nuisance parameter and a parameter of interest (##H_0##), they can happen in any parameters. Ideally you won't have this problem. If you do, it can be tricky to deal with. I'd forge ahead assuming it won't happen for now. Just bear in mind that it might, and the way you can tell is by looking at the eigenvalues of your Fisher matrix. If any of them are much, much smaller than the others, you've got a degeneracy (how much smaller depends upon the calculation method, but you're likely to have issues if one eigenvalue is more than about ##10^{10}## times another).
 

1. What is Fisher forecasting?

Fisher forecasting is a statistical method used in cosmology and particle physics to predict the uncertainties of model parameters based on observational data. It is named after British statistician Ronald Fisher and is commonly used to determine the ability of experiments to constrain certain parameters.

2. How is Fisher forecasting different from other statistical methods?

Fisher forecasting differs from other statistical methods in that it assumes the data is Gaussian distributed, and it provides a way to estimate the precision of parameter constraints without performing a full likelihood analysis. It also does not require prior knowledge of the underlying model or its parameters.

3. What are the steps involved in Fisher forecasting?

The first step in Fisher forecasting is to choose a model and its parameters. Next, the covariance matrix of the model is calculated using theoretical predictions. Then, the Fisher matrix is constructed by taking derivatives of the covariance matrix with respect to the parameters. Finally, the inverse of the Fisher matrix is used to estimate the uncertainties of the parameters.

4. What are the limitations of Fisher forecasting?

One of the limitations of Fisher forecasting is that it only provides estimates for the uncertainties of parameters, and does not give the full probability distribution or best-fit values. It also assumes that the data is Gaussian distributed, which may not always be the case. Additionally, it does not take into account any systematic errors or uncertainties in the theoretical predictions.

5. How can Fisher forecasting be useful in scientific research?

Fisher forecasting can be useful in scientific research as it provides a quick and efficient way to estimate the precision of parameters and determine the feasibility of experiments. It can also help in identifying which parameters have the most impact on the model and which ones can be constrained with current data. Additionally, it can be used to optimize experimental setups and design future experiments.

Similar threads

Replies
8
Views
2K
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
1K
Replies
5
Views
888
Replies
11
Views
2K
Replies
0
Views
184
  • Set Theory, Logic, Probability, Statistics
Replies
1
Views
734
Replies
4
Views
2K
Replies
1
Views
1K
  • Cosmology
Replies
5
Views
2K
Back
Top