How to optimize a parameter that is the index of a summation?

Click For Summary

Discussion Overview

The discussion revolves around optimizing a parameter, specifically the index of a summation, in the context of comparing experimentally measured values with simulated values derived from a Monte Carlo simulation. Participants explore methods for formulating this optimization problem in software like Matlab, focusing on the challenges of minimizing discrepancies across multiple detectors simultaneously.

Discussion Character

  • Exploratory
  • Technical explanation
  • Debate/contested
  • Mathematical reasoning

Main Points Raised

  • One participant describes the need to find an index ##n_0## that minimizes the difference between experimental and simulated values across multiple detectors, suggesting a potential use of ##\chi^2##-minimization or maximum-likelihood techniques.
  • Another participant suggests establishing a metric for distance to compare different approximations, proposing that if the sum follows a nice path, it could be minimized, but acknowledges that brute force searching may be necessary if the paths are not well-behaved.
  • A participant mentions that independent optimization for each detector yields differing values of ##n_0## and expresses a desire to find a single ##n_0## that minimizes differences for all detectors simultaneously.
  • There is a discussion about defining what "best" means in this context, with suggestions to consider various metrics such as the absolute sum of relative deviations or the maximal relative deviation.
  • One participant questions whether the proposed optimization procedure aligns with traditional maximum-likelihood techniques or ##\chi^2##-minimization, noting the challenges posed by the nature of the problem.
  • Another participant emphasizes the necessity of quantifying the compromise among detectors and suggests that traditional optimization techniques may not be applicable due to the problem's characteristics.

Areas of Agreement / Disagreement

Participants express differing views on the best approach to define and quantify the optimization metric. There is no consensus on whether the optimization can be framed as a maximum-likelihood technique or a ##\chi^2##-minimization, and the discussion remains unresolved regarding the best method to apply.

Contextual Notes

Participants highlight the complexity of the optimization problem, including the non-smooth nature of the dependent variable and the lack of a clear path for the summation, which complicates the application of standard optimization techniques.

Condereal
Messages
3
Reaction score
3
TL;DR
Finding the best parameter to satisfy a set of equations, but that parameter is the index of a summation.
Hi everyone!

So, the problem I'm having has more to do with "how to pose the problem to solve it in some software as Matlab or similar".

I have experimentally measured values ##\varepsilon_{exp}^i## with ##i=1,\cdots,6##, that is, I have 6 detectors.

Then, I know (from a Monte Carlo simulation) a set of values ##\psi^i(n)## with ##n\in\mathbb{N}## for each detector, that satisfy: $$\sum\limits_{n=0}^{n_{max}}\psi^i(n) = \varepsilon_{sim}^i\approx \varepsilon_{exp}^i$$ The thing is, I would like to find a number ##n_0\in\mathbb{N}## such that: $$\varepsilon_{exp}^i-\varepsilon_{sim}^i(n_0)=\varepsilon_{exp}^i-\sum_{n=n_0}^{n_{max}}\psi^i(n)\to 0$$ for all six equations at the same time, that is, for all ##i##. This is an optimization problem, and it screams ##\chi^2##-minimization or maximum-likelihood problem. Can anyone imagine a way of posing this problem in an environment like Matlab?

Every answer will be very much appreciated.
 
Physics news on Phys.org
You'll need some metric for the distance to compare different approximations. Then you can express everything in terms of that metric.

If your sum follows some nice path through your 6-dimensional space (or at least through the one-dimensional "distance" to the target) you could try to approximate it, minimize the now real parameter, and then look in the vicinity of that parameter for a local minimum of the discrete problem.

If your sum doesn't follow a nice path then searching through all cases is still an option.
 
Hi mfb, thank you for your answer!

Sadly, the paths ##\psi^i(n)## are not nice... I should probably try searching, by brute force, through all the cases. I did this but for each detector ##i## independently, setting up a tolerance. For example, I move ##n_0##, and if: $$100\frac{\lvert\varepsilon^i_{exp}-\varepsilon^i_{sim}(n_0)\rvert}{\varepsilon^i_{exp}}<t$$ for ##t=5\%## I choose that particular ##n_0## as my parameter. I obtain in that case, an ##n_0## for each ##i##, and they differ from each other. I would like to give the ##n_0## that minimizes all at once, but I wouldn't know how to start writing this in a script...!
 
In general there won't be an n_0 where all the relative deviations are minimal at the same time. You need some definition of "best" before you can try to find the best one. The absolute sum of relative deviations? The absolute sum of the squared relative deviations? The maximal relative deviation? Whatever you like. But you need to define what you want to optimize.
 
Yes, sorry, I expressed incorrectly the idea, that I wanted to minimize the difference for each detector with one ##n_0##, but in reality ##n_0## should give the smallest difference possible which "compromises" for all the detectors. So, your proposal would be to define a metric like "the absolute sum of relative deviation", and try for every ##n_0## until I find the minimum?

On the other hand:
Can this procedure be considered a usual maximum-likelihood technique, or a ##\chi^2##-minimization? In case not, do you think these techniques apply in my case?
I'm only asking because I've been suggested by a colleague to use a ##\chi^2##-minimization for this problem, but I really don't see how to do this.

Thank you for your time mfb, your answer was useful.
 
Condereal said:
the smallest difference possible which "compromises" for all the detectors.
You'll need to quantify that compromise. There is no way around that, and mathematics alone cannot tell you what will be the best metric for your task.
Condereal said:
Can this procedure be considered a usual maximum-likelihood technique, or a ##\chi^2##-minimization?
It can have some similarity, but with a single parameter and a dependent variable that doesn't vary smoothly nothing from that toolbox will help.
 

Similar threads

  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 0 ·
Replies
0
Views
2K
  • · Replies 3 ·
Replies
3
Views
923
  • · Replies 9 ·
Replies
9
Views
5K
  • · Replies 6 ·
Replies
6
Views
3K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 6 ·
Replies
6
Views
3K
Replies
2
Views
1K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 4 ·
Replies
4
Views
8K