# Substitution to turn a non-linear least squares problem into a linear one

 P: 4 Substitution to turn a non-linear least squares problem into a linear one Hi, thanks for your answers. I do think, however, I did not point out two important things clearly enough: 1. I am aware of many iterative algorithms for this problem. However, I'm interested in a non-iterative solution. (Would you call this an ad-hoc solution?) Anyway, I want to have a fixed-run-time algorithm for the problem I mentioned, and we all know that linear problems can be solved this way. 2. I am also not interested in an approximation. I am interested whether the substitution will allow me to find the exact solution in a linear way. So let me summarize what I got to so far. Let's talke about $f(x) = 1 + ax + a^2 + b$. Using some data $x_i$ and $y_i$, I can define the residual $R = \sum_{i=1}^n ||f(x_i) - y_i||^2$ Inserting $f$ with $c$ substituted by $a^2 + b$, I find $R = \sum_{i=1}^n ||1 + ax_i + c - y_i||^2$ Now, I need to calculate the derivatives with respect to a and c. Not knowing whether total or partial derivatives are the way to go, let's try both. $\partial R / \partial a = \sum_{i=1}^n 2 (1 + ax_i + c - y_i) \cdot x_i = 0$ $\partial R / \partial c = \sum_{i=1}^n 2 (1 + ax_i + c - y_i) \cdot 1 = 0$ These are pretty easy to solve non-iteratively, because they are linear in a and c, although my original problem is not. That is my main concern. Will the solution I achieve from this system of equations be exact? In the manual examples I calculated, they were, but I cannot be sure this is always the case. If we go for total derivatives, we need to account for c depending on a, with $\partial c / \partial a = 2a$ and $\partial a / \partial c$ involving roots. $dR / da = \sum_{i=1}^n 2 (1 + ax_i + c - y_i) \cdot (x_i + \partial c / \partial a) = 0$ $dR / dc = \sum_{i=1}^n 2 (1 + ax_i + c - y_i) \cdot (1 + \partial a / \partial c)= 0$ These are now not linear any more, and much more complex to solve. Testing some values fulfilling $f(x_i) = y_i$ and the correct values for a and b will of course make the first term $1 + \dots - y_i$ equal 0, so the derivatives are 0 in both cases. So, still I am not sure which way is the right one...