Parameter Scaling for Optimization

[captain]

So I am still confused about how to applying scaling of parameters to a general optimization problem. Let's say I am trying to do maximum likelihood estimation. I understand how to find the scaling matrix (assuming we restrict it to diagonal form) and that the Hessian should be close to the identity matrix near the optimal parameter values. What I don't understand is that once you have your scaling matrix how do you directly use it in the optimization. In the case of MLE I feel that just scaling your parameters wouldn't yield the right results because your data that you are trying to fit would want to fit the actual parameters and not the scaled ones, unless the data itself was scaled, which I am not sure how to do in this formulation and such. Any help would be much appreciated. Thanks in advance.

 

[fresh_42]

I think the scaling process serves the purpose, that no single range of data is preferred. E.g. if we have data around zero and others around a million, then the zeros will be lost in any calculation. So we scale the data, such that no range will be preferred. At the end of the process, we un-scale the result again, i.e. we multiply with the inverse of the scaling matrix in order to get the data back into the ranges they belong to.