Exponential curve fit using Apache Commons Math

[Duo Who Ow]

Homework Statement

I have the following data which I would like to model using an exponential function of the form y = A + Be^cx.

Using wolfram mathematica, solving for these coefficients was computed easily using the findfit function. I was tasked however to implement this using java and have come to this forum for help.

Data
X Y
10 88.822
20 45.607
30 26.407
40 18.004
50 12.504
60 10.878
70 8.229
80 8.032
90 6.674
100 6.927
110 6.146
120 6.18
130 6.19
140 5.454
150 5.698
160 5.249

Homework Equations

y = A + Be^cx

The Attempt at a Solution

[/B]
https://www.physicsforums.com/threads/exponential-least-squares-method.845240/
https://commons.apache.org/proper/commons-math/userguide/leastsquares.html
http://math.stackexchange.com/questions/1337601/fit-exponential-with-constant

From the information provided in the three links above, to solve my problem I would need to solve the three partial derivatives using an appropriate initial guess. From my limited understanding, I could potentially use a number around 5.5 as an initial guess for c which would reduce the iterations required to find an optimal solution.

However, I do not understand how to generate the 'model function' which is used by the least squares engine to evaluate the components. I have seen a few examples of generating the model function however I do not fully grasp how it is generated. A link below are additional examples. Hoping I can obtain assistance from you wonderful people. Thank you.

http://www.programcreek.com/java-ap...ath3.fitting.leastsquares.LeastSquaresProblem

PS - Sorry for the formatting, I could not find the relevant options to insert a table.

 

[Ray Vickson]

Homework Statement

I have the following data which I would like to model using an exponential function of the form y = A + Be^cx.

Using wolfram mathematica, solving for these coefficients was computed easily using the findfit function. I was tasked however to implement this using java and have come to this forum for help.

Data
X Y
10 88.822
20 45.607
30 26.407
40 18.004
50 12.504
60 10.878
70 8.229
80 8.032
90 6.674
100 6.927
110 6.146
120 6.18
130 6.19
140 5.454
150 5.698
160 5.249

Homework Equations

y = A + Be^cx

The Attempt at a Solution

[/B]
https://www.physicsforums.com/threads/exponential-least-squares-method.845240/
https://commons.apache.org/proper/commons-math/userguide/leastsquares.html
http://math.stackexchange.com/questions/1337601/fit-exponential-with-constant

From the information provided in the three links above, to solve my problem I would need to solve the three partial derivatives using an appropriate initial guess. From my limited understanding, I could potentially use a number around 5.5 as an initial guess for c which would reduce the iterations required to find an optimal solution.

However, I do not understand how to generate the 'model function' which is used by the least squares engine to evaluate the components. I have seen a few examples of generating the model function however I do not fully grasp how it is generated. A link below are additional examples. Hoping I can obtain assistance from you wonderful people. Thank you.

http://www.programcreek.com/java-ap...ath3.fitting.leastsquares.LeastSquaresProblem

PS - Sorry for the formatting, I could not find the relevant options to insert a table.

You can use TeX to format it as an array:
$$ \begin{array}{cc}
X & Y \\
10 & 88.822\\
20 & 45.607\\
30 & 26.407\\
\vdots & \vdots \\
150 & 5.698\\
160 & 5.249
\end{array}
$$
If you want, you can have horizontal and/or vertical dividing lines:
$$ \begin{array}{c|c}
X & Y \\ \hline
10 & 88.822\\
20 & 45.607\\
30 & 26.407\\
\vdots & \vdots \\
150 & 5.698\\
160 & 5.249 \\ \hline
\end{array}
$$

Right-click on the tables above and choose "display as tex ... " to see the commands I used.

About your difficulties: the first link above has all the details about what is happening. Some of the posts even derive the least-square equations in detail.

Of course, when it comes to actually solving the coupled nonlinear equations numerically, that is a whole different story. Again, some of the links provide access to details about the various solution algorithms available.

You will need to be more specific about what is puzzling you. Exactly what is it you do not understand?

 

[Duo Who Ow]

Hi Ray

Thanks for responding, your previous post was very insightful and accessible to even myself.

Excuse my initial post as I was excited finding this forum and the solution to my problem. From my limited understanding, the three partial derivatives need to be solved numerically using an initial guess which can be derived from y ≈ A + Bx + Cx² where A = a + b, B=bc and C = bc²/2.

The specific problem I face is implementing the 'model function' in the leastsquarebuilder in Apache Maths Common to determine the coefficients utilising least squares. If I am correct, the model function will be the three partial derivatives and my 'start' will be the initial values provided from the equation above.

Cheers

 

[SrayD]

Hi, was wondering why you are talking about partial derivatives, when there is just one variable. Unless there is more to the problem, of course.
But, aside from that; you can take log of equation and fit that using least-squares. Seems easier; and no initial guess needed; like that quadratic.
Hope this helps.

 

[Duo Who Ow]

Hi SrayD

For an equation of y = Ae^bx, taking the log of both sides and performing a least square regression would be possible to calculate the coefficients A and b. However when the equation becomes y = A + Be^cx, calculating the coefficients using this method does not work. There is no well-known direct way of calculating the coefficients for this equation hence why partial derivatives and numerical approaches are performed. Correct me if I'm mistaken.

Cheers

 

[Stephen Tashi]

For an equation of y = Ae^bx, taking the log of both sides and performing a least square regression would be possible to calculate the coefficients A and b.

That procedure does not calculate the values of A and b that minimize the least squares error function $f(A,b) = \sum_{i=1}^N (y_i - (Ae^{bx_i}))^2$ . Instead it calculates the values of A and b that minimize the function $g(A,b) = \sum_{i=1}^N (\ln(y_i) -(\ln(A) + bx_i))^2$.

 

[Stephen Tashi]

There is no well-known direct way of calculating the coefficients for this equation hence why partial derivatives and numerical approaches are performed. Correct me if I'm mistaken.

You are correct.

The problem for me (and for many potential advisors) is that the mathematics is quite familiar, but the functionality of "Apache Maths Common" (http://commons.apache.org/proper/commons-math/javadocs/api-3.5/index.html) is not. (The convenience of using extensive program libraries is offset by the disagreeable task of learning the interfaces to such libraries.)

A conceptually simple plan for solving your code would be to use a library function that minimized or maximized a user defined function of several variables. Can you find such a library function in Apache Maths Common ?

The solution of the problem can be approached by setting the system of 3 simultaneous equations, which comes from setting each the 3 partial derivatives equal to zero and then solving that simultaneous system. However, you can avoid those complications if you find a single library function that can minimize a function of several variables.

 

[SrayD]

Hi SrayD

For an equation of y = Ae^bx, taking the log of both sides and performing a least square regression would be possible to calculate the coefficients A and b. However when the equation becomes y = A + Be^cx, calculating the coefficients using this method does not work. There is no well-known direct way of calculating the coefficients for this equation hence why partial derivatives and numerical approaches are performed. Correct me if I'm mistaken.

Cheers

Yes, you are right. My Apologies. Thought could use log sum formula to simplify; but it gets pretty messy.

 

[JJacquelin]

Hi, Duo Who Ow

you seem embarrassed by the "appropriate initial guess" necessary to start the iterative process currently used in the usual methods of non-linear regression.

Why not using a method where no initial guess is necessary ?

This is possible in the case of the fitting of the function y=a+be^c . This method is very simple because no iterative process is necessary. See the code in attachment.

With your data, the result is : a = 6.189453 ; b = 156.951617 ; c = - 0.065979

The graph below shows the result of the fitting.

Note : In this particular method, the criteria of fitting isn't the usual "mean least squares". The theory is given in

https://fr.scribd.com/doc/14674814/Regressions-et-equations-integrales (pages 15-17).

If you definitively want a "mean least squares" fitting, use any available convenient software with the above (a,b,c) values as initial guess. The iterative process in the software will give : a = 6.479641 ; b = 163.31383 ; c = -0.06931 . These values appear slightly different, but in practice the result is quite the same. The curves on the graph cannot be distinguished one to the other.

 

[Duo Who Ow]

Hi JJacquelin

I have actually come across your post on mathexchange in regards to estimating coefficients without iterations, it was extremely helpful. I was able to successfully generate the coefficients using excel and was planning to implement your method within my program but my skills were inadequate for the task. That is why I have tried to implement the typical method of non linear Levenberg Marquardt curve fitting to determine the coefficients as libraries and programs such as ImageJ have successful implemented them.

It's nice to see you around and thank you for your response.