MHB Relation within Gauss-Newton method for minimization

i_a_n
Messages
78
Reaction score
0
If we study model fit on a nonlinear regression model $Y_i=f(z_i,\theta)+\epsilon_i$, $i=1,...,n$, and in the Gauss-Newton method, the update on the parameter $\theta$ from step $t$ to $t+1$ is to minimize the sum of squares $\sum_{i=1}^{n}[Y_i-f(z_i,\theta^{(t)})-(\theta-\theta^{(t)})^Tf'(z_i,\theta^{(t)})]^2$. Can we prove that (why) (part 1) the update is given in the following form: $\theta^{(t+1)}=\theta^{(t)}+[(A^{(t)})^TA^{(t)}]^{-1}(A^{(t)})^Tx^{(t)}$,(part 2) where $A^{(t)}$ is a matrix whose $i$-th row is $f'(z_i,\theta^{(t)})^T$, and $x^{(t)}$ is a column vector whose $i$-th entry is $Y_i-f(z_i,\theta^{(t)})$.
Any solution or hints? How to derive those relationships?

Thanks in advance!
 
Physics news on Phys.org
ianchenmu said:
If we study model fit on a nonlinear regression model $Y_i=f(z_i,\theta)+\epsilon_i$, $i=1,...,n$, and in the Gauss-Newton method, the update on the parameter $\theta$ from step $t$ to $t+1$ is to minimize the sum of squares $\sum_{i=1}^{n}[Y_i-f(z_i,\theta^{(t)})-(\theta-\theta^{(t)})^Tf'(z_i,\theta^{(t)})]^2$. Can we prove that (why) (part 1) the update is given in the following form: $\theta^{(t+1)}=\theta^{(t)}+[(A^{(t)})^TA^{(t)}]^{-1}(A^{(t)})^Tx^{(t)}$,(part 2) where $A^{(t)}$ is a matrix whose $i$-th row is $f'(z_i,\theta^{(t)})^T$, and $x^{(t)}$ is a column vector whose $i$-th entry is $Y_i-f(z_i,\theta^{(t)})$.
Any solution or hints? How to derive those relationships?

Thanks in advance!

How are those iterations (or updates) defined in the Gauss-Newton method?
 
Hi all, I've been a roulette player for more than 10 years (although I took time off here and there) and it's only now that I'm trying to understand the physics of the game. Basically my strategy in roulette is to divide the wheel roughly into two halves (let's call them A and B). My theory is that in roulette there will invariably be variance. In other words, if A comes up 5 times in a row, B will be due to come up soon. However I have been proven wrong many times, and I have seen some...
Thread 'Detail of Diagonalization Lemma'
The following is more or less taken from page 6 of C. Smorynski's "Self-Reference and Modal Logic". (Springer, 1985) (I couldn't get raised brackets to indicate codification (Gödel numbering), so I use a box. The overline is assigning a name. The detail I would like clarification on is in the second step in the last line, where we have an m-overlined, and we substitute the expression for m. Are we saying that the name of a coded term is the same as the coded term? Thanks in advance.
Back
Top