Parameter Scaling for Optimization

  • Thread starter captain
  • Start date
  • #1
163
0

Main Question or Discussion Point

So I am still confused about how to applying scaling of parameters to a general optimization problem. Let's say I am trying to do maximum likelihood estimation. I understand how to find the scaling matrix (assuming we restrict it to diagonal form) and that the Hessian should be close to the identity matrix near the optimal parameter values. What I don't understand is that once you have your scaling matrix how do you directly use it in the optimization. In the case of MLE I feel that just scaling your parameters wouldn't yield the right results because your data that you are trying to fit would want to fit the actual parameters and not the scaled ones, unless the data itself was scaled, which I am not sure how to do in this formulation and such. Any help would be much appreciated. Thanks in advance.
 

Answers and Replies

  • #2
12,901
9,542
I think the scaling process serves the purpose, that no single range of data is preferred. E.g. if we have data around zero and others around a million, then the zeros will be lost in any calculation. So we scale the data, such that no range will be preferred. At the end of the process, we un-scale the result again, i.e. we multiply with the inverse of the scaling matrix in order to get the data back into the ranges they belong to.
 

Related Threads on Parameter Scaling for Optimization

  • Last Post
Replies
5
Views
978
  • Last Post
Replies
3
Views
1K
  • Last Post
Replies
2
Views
2K
  • Last Post
Replies
1
Views
1K
Replies
7
Views
2K
  • Last Post
Replies
2
Views
7K
  • Last Post
Replies
2
Views
2K
  • Last Post
Replies
2
Views
3K
  • Last Post
Replies
4
Views
950
  • Last Post
Replies
2
Views
2K
Top