Levenberg-Marquardt Algorithm for more than one function

[Shauheen]

I have been looking how to use the Levenberg Marqurdt algorithm for minimizing the errors of two functions at the same time. I looked up this topic in the internet and the only useful thing I found was a thread between "I like Serena" and "thomas430" in summer 2011.

https://www.physicsforums.com/showthread.php?t=521670

I read "I like Serena"'s notes, but still have problem in finding out how I have to calculate the Jacobian matrix.

I am measuring two values over time in the lab. Then, a numerical forward model has been developed which models the physics and now, I need to solve the inverse problem to find the unknown parameters. I have solved this problem previously several times when only one error function needed to be minimized. Based on what I have learned about LMA (from Inverse Heat Transfer by Necati Ozicik), the gradient of the error (from one function) will result in multiplication of the Jacobian matrix by the difference between the experimental and model results. This gradient needs to become zero:

∇S(P)=2[-∂T(P)/∂P][Y-T(P)]=0

In which P is the vector of the unknown parameters, T is the model value vector and Y is the experimental value.
This problem, minimizes S(P)=Ʃ[Yi-Ti(P)]^2

Now, I have to find let say 3 parameters from the minimization of two errors from my experiments. In my experiment, I measure two values vs. time, let say Y and M. These two values are dependent. The number of the collected data are not the same for these two and I think they are both equally important. I could check on the typical range of errors to make them equally significant.
My question is how to minimize the error of the combination of these two and find my unknowns?

Let say if the error is like:
E(P)=Ʃ[Yi-Ti(P)]^2+Ʃ[Mi-Ui(P)]^2

Ui and Ti are the values from my model.
What would be the Jacobian matrix in this case? for one function, it would be like:
P(k+1)=P(k)+[(J(k)T.J(k)+μ(k)Ω(k)]^-1 (J(k))T[Y-T(P(k))]
in which k is the kth iteration.

How would be the formulation once two functions are involved?

I would really appreciate if anyone could help.

 

[I like Serena]

Hello and welcome to PF, Shauheen!

I'm not sure if I understand everything in your problem yet, but as I understand it now, you have a measured vector Y and a model T(P) for it, and you also have a measured vector M and a model U(P) for it.

You can turn these 2 measurement vectors into one but putting them on top of each other.
Let's say we let N=(Y1, ..., Yn, M1, ..., Mm), and let V(P)=(T1(P), ..., Tn(P),U1(P), ..., Um(P)).
Then that reduces your problem back to where you have one measurement vector and one model vector function.
The new Jacobian matrix is basically a combination of the 2 separate Jacobian matrices.

 

[Shauheen]

Hello and welcome to PF, Shauheen!

I'm not sure if I understand everything in your problem yet, but as I understand it now, you have a measured vector Y and a model T(P) for it, and you also have a measured vector M and a model U(P) for it.

You can turn these 2 measurement vectors into one but putting them on top of each other.
Let's say we let N=(Y1, ..., Yn, M1, ..., Mm), and let V(P)=(T1(P), ..., Tn(P),U1(P), ..., Um(P)).
Then that reduces your problem back to where you have one measurement vector and one model vector function.
The new Jacobian matrix is basically a combination of the 2 separate Jacobian matrices.

Thanks a lot I Like Serena for your kind welcome and reply to my question :)
Y and M are two different physical values. One is height change (I am tracking the height) and the other is mass. I start collecting each of these two data sets from time zero to t(final). Would that still be possible to put the two different physical value on top of each other in a matrix?
What would be my error then?

Physically in my mind, it is like plotting both curves vs. time and trying to match the two model to these curves with the same parameters, right?

I have attached for you a handwritten description about my problem and have attached it to this reply.

Thanks again for devoting your time.

 

[I like Serena]

Thanks a lot I Like Serena for your kind welcome and reply to my question :)
Y and M are two different physical values. One is height change (I am tracking the height) and the other is mass. I start collecting each of these two data sets from time zero to t(final). Would that still be possible to put the two different physical value on top of each other in a matrix?
What would be my error then?

Physically in my mind, it is like plotting both curves vs. time and trying to match the two model to these curves with the same parameters, right?

I have attached for you a handwritten description about my problem and have attached it to this reply.

Thanks again for devoting your time.

You can put the 2 vectors on top of each other.
The result is not so much a matrix, but just another vector.

The error function can be the same to what you already have:
$$E(P)=\sum_{i=1}^{n+m}[N_i-V_i(P)]^2=\sum_{i=1}^{n}[Y_i-T_i(P)]^2+\sum_{i=1}^{m}[M_i-U_i(P)]^2$$
where n is the number of Y measurements and m is the number of M measurements.When you have solved the system, useful properties will be:
$$s_Y = \sqrt{\frac{\sum[Y_i-T_i(P)]^2}{n-1}}$$
$$s_M = \sqrt{\frac{\sum[M_i-U_i(P)]^2}{m-1}}$$
where $s_Y$ is the standard deviation of the error in the Y measurements and $s_M$ is the standard deviation of the error in the M measurements.