Nonlinear Least Squares Minimization

Click For Summary
SUMMARY

The discussion focuses on solving a Nonlinear Least Squares (NLLS) minimization problem using the function F(x;a) = (1+a1*x)/(a2+a3*x) with parameters a1=2, a2=3, and a3=5, and six data points. The Levenberg-Marquardt algorithm is recommended as a primary method for this type of problem. Additionally, the BFGS algorithm and the simplex search method are suggested as alternative optimization techniques. Resources such as "Numerical Recipes" are cited for further reading on these algorithms.

PREREQUISITES
  • Understanding of Nonlinear Least Squares (NLLS) minimization
  • Familiarity with the Levenberg-Marquardt algorithm
  • Knowledge of BFGS optimization method
  • Basic concepts of simplex search method
NEXT STEPS
  • Study the implementation of the Levenberg-Marquardt algorithm in Python using SciPy
  • Explore the BFGS algorithm for nonlinear optimization in MATLAB
  • Learn about the simplex search method and its applications in optimization
  • Read "Numerical Recipes" for detailed explanations and code examples of optimization algorithms
USEFUL FOR

Mathematicians, data scientists, and engineers involved in optimization problems, particularly those working with nonlinear models and seeking to enhance their understanding of NLLS techniques.

swartzism
Messages
103
Reaction score
0
How should I go about solving this problem? This is only to get a better understanding of how NLLS works.

F(x;a) = (1+a1*x)/(a2+a3*x) (so n = 3)

I am choosing a1,a2,a3 to be 2,3,5 respectively. I am also picking 6 data points (so m = 6):

(0, 0), (-1/4, 1/4), (-1/2, 1/10), (1/4, 1/4), (1, 1), (1/2, 1/2).

I'm not even sure where to begin to attack this problem.

Any suggestions?
 
Physics news on Phys.org
The Levenberg-Marquardt algorithm is a widely used method.

Alternatively, you can treat NLLSQ as a general nonlinear optimization problem, and use standard optimization methods like the BFGS algorithm if the problem is fairly well behaved, or the simplex search method (which can be very slow, but will find a local minimum of pretty much anything!) if it isn't.

IIRC all these are in "Numerical Recipes", or Google for other explanations of the algorithms and computer code.
 

Similar threads

MHB Bilinear transformation not working
  • · Replies 5 ·
Replies
5
Views
2K
Undergrad Nonlinear Least Squares or OLS for Nonlinear Models?
  • · Replies 7 ·
Replies
7
Views
3K
Undergrad Definition of minimal ideal using symbolic logic notation
  • · Replies 3 ·
Replies
3
Views
2K
Graduate Anti-dual numbers and what are their properties?
  • · Replies 1 ·
Replies
1
Views
2K
Undergrad What is Nonlinear Least Squares in Regression Analysis?
  • · Replies 4 ·
Replies
4
Views
2K
Graduate Have I solved this iterative equation correctly?
  • · Replies 2 ·
Replies
2
Views
1K
Problem with Bernoulli calculation
  • · Replies 4 ·
Replies
4
Views
1K
Undergrad Formal derivation of statement from Peano Arithmetic system
  • · Replies 10 ·
Replies
10
Views
2K
Reproduce band structure Kagome Fermi-Hubbard using Python
  • · Replies 6 ·
Replies
6
Views
2K
Comp Sci An Introduction to Deep Learning and Modifying Code
  • · Replies 7 ·
Replies
7
Views
2K