Thread: empirical data View Single Post
HW Helper
P: 930

## empirical data

The tie to Levenberg-Marquardt is as follows. The (square) matrix $J^T J$ can often be ill-conditioned at intial guesses of the function coefficients making the equation $J^T J z = J^T (y-f)$ difficult if not impossible to solve. The (simpler) version of Levenberg-Marquardt modifies the problem by adding a small number $\lambda$ to the diagonals of $J^T J$, resulting in the following modified equation:

$$(J^T J + \lambda I) z = J^T (y-f)$$

which is more numerically stable the larger $\lambda$ is. The trick is to make $\lambda$ as small as possible to solve the problem, then gradually reduce it's magnitude to zero as the solution progresses.

One other note. The matrix $J^T J$ doesn't need to be computed at all if QR matrix factorization is used. Then, all that's needed is to factor $J = QR$ then solve the triangular system $R z = Q^T (y-f)$. See link below for more explanation.

http://www.alkires.com/teaching/ee10...torization.htm

The next question might be how to add $\lambda$ to the diagonals of $J^T J$ if $J^T J$ is never computed? Actually, it is done by augmenting J and (y-f) as follows:

$$J^* = \begin{bmatrix} J \\ \sqrt{\lambda} I \end{bmatrix} \qquad \qquad (y-f)^* = \begin{bmatrix} (y-f) \\ 0 \end{bmatrix}$$

Then factor $J^* = QR$ and solve $R z = Q^T (y-f)^*$.

This is probably more than you wanted to know........