Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Nonlinear Least Squares Fitting

  1. Jan 14, 2014 #1

    So I was hoping to get some help implementing a nonlinear least squares fitting algorithm. Technically this is an extension of my previous thread, however the problem I am having now is correctly computing the algorithm

    So the problem definition is this:

    Given two sets of n 3D points [itex]X_{i} = (X_{1},X_{2}...,X_{n})[/itex] and [itex]X^{'}_{i} = (X^{'}_{1},X^{'}_{2}...,X^{'}_{n})[/itex], where [itex]X^{'}_{i}[/itex] is the result of an applied rotation to [itex]X_{i}[/itex], find the corresponding rotation matrix and Euler Angles

    The formulas I am following are from these two links:
    Euler Angles Eqs 71 - 77 and NonLinear Least Squares Fitting

    I am going to attempt to follow the convention of those articles. So first off I set up matrices X and X' as such

    x_{1} & x_{2} & x_{3}... & x_{n} \\
    y_{1} & y_{2} & y_{3}... & y_{n} \\
    z_{1} & z_{2} & z_{3}... & z_{n} \end{bmatrix}[/itex]

    where [itex]x_{i},y_{i},z_{i}[/itex] are the components of [itex]X_{i} / X^{'}_{i}[/itex]

    Next I have set up my rotation matrices as follows:

    [itex] R_{x}(\theta_{x}) = \begin{bmatrix} 1 & 0 & 0 \\ 0 & cos(\theta_{x}) & -sin(\theta_{x}) \\ 0 & sin(\theta_{x}) & cos(\theta_{x}) \end{bmatrix} [/itex]

    [itex] R_{y}(\theta_{y}) = \begin{bmatrix} cos(\theta_{y}) & 0 & sin(\theta_{y}) \\ 0 & 1 & 0 \\ - sin(\theta_{-}) & 0 & cos(\theta_{y}) \end{bmatrix} [/itex]

    [itex] R_{z}(\theta_{z}) = \begin{bmatrix} cos(\theta_{z}) & -sin(\theta_{z}) & 0 \\ sin(\theta_{z}) & cos(\theta_{z}) & 0 \\ 0 & 0 & 1 \end{bmatrix} [/itex]

    Multiplied together:

    [itex]R_{t} = R_{z}*R_{y}*R_{x}[/itex]

    Next take matrix [itex]A[/itex] (which I take to the be rotation matrix [itex]R_{t}[/itex]) and turn it into a column vector called [itex]f[/itex]

    Then the Jacobian [itex]J[/itex] of [itex]f[/itex] with respect to [itex]\theta_{x}, \theta_{y}, \theta_{z}[/itex] is computed

    [itex]J*d\theta = df[/itex]

    This is where the first article stops. Moving to the next article,

    [itex]J[/itex] is [itex]A[/itex], [itex]d\theta[/itex] is [itex]d\lambda[/itex], and [itex]df[/itex] is [itex] d\beta [/itex] I presume. However, I will keep the original convention.

    Finally, you get [itex]J^{T}*J*d\theta = J^{T}*d\beta[/itex]

    so [itex] d\theta = (J^{T}*J)^{-1}*J^{T}*d\beta [/itex]

    So my questions are these:

    1. How do I form [itex]d\beta[/itex]? Right now I am taking my "guess" at the angles (will call this [itex]\theta_{xo,yo,zo}[/itex]) to compute [itex]R_{t0}[/itex].

    Then [itex]d\beta = X^{'} - R_{t0}*X[/itex], and it is turned into a column vector just like [itex]R_t[/itex] is turned into [itex]f[/itex]. Is this correct?

    2. When I compute my new angles, should it be

    [itex]\theta_{x,y,z} = \theta_{x,y,z} + d\theta[/itex] or [itex]\theta_{x,y,z} = \theta_{x,y,z} - d\theta[/itex]

    I have successfully programmed all of these steps into a VB.NET program, but the solution is not converging and I cannot figure out why.

    Any help greatly appreciated. Thanks
    Last edited: Jan 14, 2014
  2. jcsd
  3. Jan 14, 2014 #2
    So I am realizing I think how I am calculating [itex]d\beta[/itex] / [itex]d\ f[/itex] is incorrect, as this should yield an equation that is a 3x3 matrix which is transformed into the 1x9 column vector. Can someone please explain how to find [itex]d\ f[/itex]? I am assuming it has something to do with this equation [itex] A = X^{'}X^{T}(XX^{T})^{-1} [/itex]. Thanks
  4. Jan 15, 2014 #3
    So I think it figured it out, but it still seems strange. I took [itex] d\beta = R_{t}(\theta_{x},\theta_{y},\theta_{z}) - X^{'}X^{T}(XX^{T})^{-1} [/itex] where [itex] \theta_{x}, \theta_{y}, \theta_{z} [/itex] are updated at each iteration by [itex]\theta_{x,y,z} = \theta_{x,y,z} - d\theta_{x,y,z}[/itex], and it appears to be working.

    However, it seems like it should actually be [itex] d\beta = X^{'}X^{T}(XX^{T})^{-1} - R_{t}(\theta_{x},\theta_{y},\theta_{z}) [/itex] as this matches the definition on the wolfram page [itex]d\beta_{i} = y_{i} - f_{i}(\lambda_{1},\lambda_{2},... \lambda_{i})[/itex], but the solution refuses to converge unless I write it as above.

    Can anyone shed some light on this discrepancy and confirm I am calculating this correctly?
    Last edited: Jan 15, 2014
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted

Similar Discussions: Nonlinear Least Squares Fitting
  1. 3D Least Squares Fit (Replies: 3)

  2. Least squares fit (Replies: 7)